BACKGROUND
vast amounts of research is dedicated to the identification of mutations and their impacts.
biologists usually make inferences about functions of novel sequences by comparing them to the functions of known sequences  <cit> .
many mutagenesis experiments are performed to improve the properties of proteins, particularly enzymes.
additionally detection of disease causal mutations attracted a lot of attention.
the result of all these efforts lies in publications, particularly in textual format.
consequently, locating and retrieving this information is a very cumbersome task.
some databases try to manually curate such information and provide it in publicly accessible form.
however, even for an expert curator, extracting this information manually is laborious.
hence, database curators now increasingly reach for text-mining procedures.
large-scale attempts resulted in high levels of performance in the realization of the automatic extraction of mutations  <cit> .
yet, finding their impacts, affected protein properties and magnitudes of effects remains challenging.
mema  <cit>  uses regular expressions to extract mutations and mutation-gene pairs.
it focuses on the co-occurrence of mutations and genes within a sentence and proximity parameters within an abstract.
the performance of the system is evaluated on a set of  <dig> abstracts.
the reported recall and precision for the mutation detection task are >67% and >96%, respectively.
mutext  <cit>  searches for mutation data using a pattern matching approach and further validates the extracted point mutations using two plausibility filters: a sequence filter and distance filter.
the performance of the system is evaluated on two corpora.
their algorithm detects  <dig> %- <dig> % of point mutations with a specificity of  <dig> %- <dig> %.
mutation grab  <cit>  takes a dictionary-based approach to identify protein and gene names while extracting point mutation terms using regular expressions, utilizing graph bigrams to disambiguate the extracted protein point mutations.
the authors evaluate the effectiveness of their approach on the articles describing three protein families, namely, tyrosine protein kinases, gpcrs and transmembrane ion.
mugex  <cit>  uses  <dig> regular expressions to detect mutations and statistical techniques to disambiguate between protein mutations and nucleotide mutations or cell lines.
gene-mutation pairs are detected through proximity measures.
the mutationfinder system  <cit>  extends mutext  <cit> ’s rules to extract and normalize point mutations.
in recent work  <cit> , the authors present a strategy to integrate information about phenotypic effect of snps from uniprotkb and pathways from reactom and biopax for visualization in cytoscape.
yip et al.
<cit>  uses  <dig> regular expressions to extract and retrieve single amino acid poly morphisms .
the system is assessed on a swiss-prot corpus with  <dig> pubmed references.
additionally, each pattern is evaluated separately.
the mstrap  system  <cit>  was developed with the aim of annotating mutations and representing them as instances of an ontology.
they further use mstrapviz to read the populated ontology and visualize the annotations on protein structures.
enzyminer  <cit>  tries to categorize pubmed abstracts based on the impact of a protein level mutation on the stability and activity of a given enzyme.
using different classification algorithms, enzyminer is able to narrow down search results; however, detailed information about the direction of the impacts, association of impacts to mutations and the kind of change in stability or functionality is not provided.
although enzyminer targets mutation impact information, it differs significantly from our approach, as we are concerned with sentence-level detection and semantic analysis of mutation impacts, not document classification.
in  <cit> , the authors introduced the first rule-based approach to extract mutation impacts on protein properties while categorizing the directionality of the impacts and grounding the impacts to the mutations.
the extracted information was populated to a domain ontology for further querying through a web service.
while in the aforementioned work, molecular properties and the michaelis constant , the rate constant  and the compound variable  are considered, the other protein properties, such as the remaining kinetic constants and protein stability, are ignored.
on the corpus of  <dig> documents on haloalkane dehalogenase, the authors report a recall of 34% and a precision of 86% for the mutation-impact relation extraction task.
a recent work on the extraction of kinetic information and associated information, namely, enzyme names, ec numbers and localization is presented in  <cit> .
the proposed rule- and dictionary-based approach in this system is applied to pubmed abstracts and the results are provided in kid, the kinetic database  <cit> .
kipar  <cit> , an information retrieval system, focuses on kinetic modeling of metabolic pathways using a rule-based approach.
however, all the existing approaches are unable to extract the protein properties affected by mutations.
in this paper, we present a rule-based approach to extract mutation series, modified protein properties and magnitudes of effects  <cit> .
in our system, the relation between the magnitudes of effects and the protein properties are detected and validated against the domain ontology.
to provide for effective querying and analysis, we populate a domain ontology with the extracted information.
table  <dig> summarizes the scope of our open mutation miner  system, compared to existing approaches.
further details on these tasks are provided in the following section.
methods
in order to comprehensively extract mutation impacts, the detection of several named entities and their relations, in particular mutations and protein properties, is required.
as an example, consider the following text segment   <cit> : “several single mutants , double mutants , and triple mutants  were prepared and expressed as glutathione s-transferase  fusion proteins in escherichia coli and purified by gsh-agarose affinity chromatography.
mutant q15k-w37r and mutant q15r-w37r showed comparable activity for nad and nadp with an increase in activity nearly 3fold over that of the wild type.“
in this example, we need to extract increase as an impact that is caused “comparably” by two mutation pairs, q15k-w37r and q15r-w37r.
in other words, the two aforementioned mutations have the same impact on the activity of an enzyme, glutathione s-transferase  that is residing in the host organism, escherichia coli.
we are also interested to know that activity as a kinetic property of the mutant enzyme is measured 3fold higher than the activity of the wild-type enzyme.
note that other entities, such as single mutations , exist in the text segment, but here we are only interested in the entities that are related to the identified impact.
the result of the system should be a set of detected entities, correctly normalized and grounded, and linked with each other.
after detecting organism mentions, which is handled by a separate module, the organismtagger  <cit> , the first step of impact analysis is to detect impact mentions.
however, extracting only impacts is not sufficient; we want to know which mutation caused the impact.
hence, the system needs to ground the detected impacts to mutations.
additionally, mutations can appear in the form of mutation series .
thus, the system must also be able to identify these complex mutation expressions.
finding out which protein properties were affected by the mutations and to what extent is necessary to identify advantageous mutations.
towards this end, we export the analysis results into an ontology  for further applications, including queries and summarization.
an overview of our system is presented in figure  <dig>  in what follows, we will provide a detailed description of each task.
impact ontology
our impact ontology is an extension to the ontology described in  <cit> , conceptualizing impacts and the mutations associated with them .
the use of the impact ontology facilitates advanced queries and impact extraction.
the ontology contains information about several concepts: text elements, biological entities and entity relations, e.g., sentence, mutationlmpact and measuredwith, respectively.
we extended the ontology with new classes, such as michaelismentenconstant, specificactivity, and maximalvelocity.
our ontology has a rich set of relationships between the concepts.
main concepts modeling impacts on a semantic level are:
mutation: an alteration or a change to a gene and developing a different offspring.
unitofmeasurement: a standard for measuring the physical quantity.
mutationlmpact: the expansion of an impact can be presented as a bifurcating tree: each bifurcating node represents a mutation effect on protein properties, whether the impact is measurable or not.
proteinproperty: a class for protein properties, which subsumes kinetic properties, protein function, and protein stability.
information about the effect of mutations on proteins can be modeled at different granularity levels.
for example, the effect can be on the structure, which consequently can affect various properties of the proteins.
for a finer level of granularity, we represent all these relations.
the relations between these entities, expressed as owl object properties, are listed in table  <dig>
each protein property is measured with specific units of measurement, for example michaelis menten constant is measured with units such as per second, per minute, etc.
however, in interpreting the mutation impacts, not only are these units of measurement utilized, but also ratio measurements can be used.
for example, the measured values of the affected protein property are compared with the measured values of the wild type or other mutated protein properties, and specified by percent, fold or orders of magnitude.
we decided to establish some restrictions on the units of measurement with which each protein property is measured, as well as the ratio measurement units.
these restrictions are encoded in the ontology based on global standards , where protein properties are measured by specific units of measurements.
these constraints are encoded as possible value fillers for the measuredwith slot for a specific protein property.
for instance, km can be measured with fold, per second and per minute, etc.
we also defined a datatype property for protein properties, called physicalquantity, referring to the value and the unit of measurement found in the text.
mutation extraction component
single point mutations can be expressed in single-letter standard format or through more complex representations.
we integrated one external mutation detection system and also developed our own approach.
mutationtagger
our mutationtagger, based on previous work  <cit> , extracts single point mutations using grammar rules and normalizes them to their single-letter format.
however, mutational mentions in the form of natural language, such as “met for val substitution found at position 270” are currently ignored by our system.
mutationfinder
the mutationfinder system  <cit>  accomplishes the task of single mutation detection and normalization by using regular expressions.
mutationfinder also tries to identify mutational changes expressed in natural language.
however, it still fails at extracting all mentions.
mutation series
in the simplest case, impacts are results of single mutations.
however, impacts are occasionally caused by mutation series.
for example, in our corpus of  <dig> full-text documents, 6% of the mutation mentions are mutation series.
during the manual inspection of several hundred documents containing mutation series, it became obvious that these mutation series  may have different appearances and representations.
they can be described using special symbols, such as “/” and “:”, or keywords, such as double mutants, triple mutants, etc.
table  <dig> summarizes these different forms.
therefore, mutations connected with these special characters or preceded by the keywords are considered as mutation series and detected through regular expressions.
to ensure that these detected mutation series have one identical internal representation, we further normalize them to the format, where all the mutations in a series are separated by the notation “/”.
protein properties extraction component
mutations can alter the structure of proteins that subsequently results in affecting their functions, either by gaining a function or losing one.
mutations may also affect the stability of proteins, where the ratio of the unfolded protein increases or decreases compared to the folded protein.
mutagenesis experiments are constantly performed to identify the importance of protein residues, either to find the source of a disease or a cure to one.
furthermore, studies are done to improve enzyme functions.
in our system, protein properties are expressed in rdf format  <cit>  and detected through gazetteering.
the extracted information can then be correlated to impacts in subsequent processing steps.
molecular function
understanding the role of mutations, in particular their contribution to diseases like cancer, requires identifying their impact on molecular functions.
causative mutations can drive cancers by activating a protein function or in-activating a function.
they can promote cancer progression by their resistance to drugs or, according to a recent study, switching of functions  <cit> .
detection of the functional impact of mutations has not only drawn attention in cancer study, but has also been an important matter in re-sequencing efforts.
to detect molecular functions, we use the concepts presented by the gene ontology.
we generate an rdf representation of molecular functions from a download of the gene ontology.
the gene ontology is provided in obo-xml format, where each node is one entry .
we first check for molecular_function namespaces, then, we extract the name and go id, as well as the synonyms of the entry.
using this information, we generate our rdf file.
for obtaining further information, molecular functions are specified by their gene ontology id .
the format of a triple is c <dig> rdfs:subclassof c <dig>  where rdfs:subclassof is an instance of rdf:property and states that c <dig>  here recognized as the gene ontology id, is an instance of rdfs:class and a subclass of c <dig>  an instance of rdfs:class, “molecular_function”.
the resulting rdf is then used for gazetteering using an lkb gazetteer component  <cit> .
kinetic constants
depending on their interests, enzyme and protein engineers apply recombinant dna technology to improve enzyme kinetic values and stability or identify the roles of residues.
consider a study on the role of asn <dig> in humans  <cit> : “to examine the role of asn <dig> in the catalytic mechanism of human xr, mutant forms  were prepared.
the two mutations increased km for the substrate  and kd for nadph , but only the n107l mutation significantly decreased kcat value.“
here, two prepared mutations, n107d and n107l, affect three kinetic values, michaelis menten constant , turn-over number  and dissociation constant , of the protein.
to capture these kinetic properties, we manually compiled them from the scientific literature.
the list of these properties is by no means exhaustive.
however, property synonyms add complexity to later tasks where the relations are extracted and validated against the ontology.
a simple rdf schema allows us to deal with different term representations of a concept and to resolve all aliases of the same concept.
a triple is defined as c <dig> rdfs:subclassof c <dig>  where c <dig> is an instance of rdfs:class, “proteinproperty”.
the rdfs:label is an instance of rdf:property, where rdfs:domain is rdfs:resource and the rdfs:range is literal.
normalizing all aliases to one single representation can also be helpful when populating the output ontology.
consider half-life as an example, it can appear in different variations, e.g., t <dig> , t1/ <dig> and half-lives.
all these variations are represented as labels in the aforementioned rdf, thus, in case any of them matches, the mention is normalized to half-life.
kinetic values
knowing the magnitude of protein properties affected by mutations enables biologists to better compare the mutation impacts of their interests.
as an example, consider this bio-engineering study that was conducted on quinoprotein glucose dehydrogenase to improve the thermal stability of the enzyme  <cit> : “the halflife at 55°c of ser415cys  was approx 36-fold greater than that of the wild-type enzyme  and 4-fold greater than that of the ser231lys variant .”
the ser residue at position  <dig> is chosen for constructing different variants of the enzyme and compared with the s231k variant.
analyzing which variant results in the most thermostable enzyme requires the extraction of the magnitudes.
half-life of s415c is measured as  <dig> min, whereas s231k was measured as  <dig> min, and the measured half-lives of the two mutations are also compared to that of the wild-type enzyme.
the magnitudes of protein properties are expressed in signed numbers, decimals and ranges of values for a single parameter.
since the existing gate generic tokeniser  <cit>  can only detect digits, we developed a simple tokeniser to capture possible representations of magnitudes.
to ensure that we extract the reported ranges of magnitudes, we collected possible range representations from the literature and expressed them through grammar rules.
after detecting all possible values, we check which values express a physical quantity using the patterns and discard all other values.
units of measurement
units of measurement are expressed in various formats, in mass or molar concentration , in different systems  and different scales .
finding how a magnitude is measured requires detecting units of measurement.
using the same approach as for kinetic properties, the list of units of measurement was collected from the literature and encoded in an rdf schema.
the rdf schema is limited to one subclass hierarchy and assigns the units of measurement to their identified concept in the owl-dl ontology.
consider the unit of measurement, per second, the same concept as persecond is encoded in the owl-dl ontology .
if any of the representations of per second is detected in the document, the class persecond is assigned to it, facilitating the ontology population step.
physical quantities
we use the information about the units of measurement to extract physical quantities.
usually, units of measurement follow values, except for a few with no specific units of measurement, such as ph.
more succinctly:
after reviewing the literature, we designed a set of patterns to capture physical quantities.
impact extraction component
mutations are considered as sources of species evolution.
some result in beneficial changes, while others have detrimental effects.
it is important to not only find impacts, but also to mark the origin mutation and altered protein properties for further analysis.
a system capable of analyzing mutation impacts requires information from many entities.
impact analysis consists of the following steps:
<dig>  finding impact expressions.
<dig>  finding mutations or mutation keywords.
<dig>  identifying the polarity of the impact to detect advantageous and disadvantageous impacts.
<dig>  grounding the impacts to mutations to find which mutations lead to a specific impact.
<dig>  finding the affected protein properties.
<dig>  finding the magnitude of the effect to help bio-engineers compare the effects and find the most favourable mutations.
for the first step, we use ontology based gazetteering, with the help of the morphological analyzer  <cit> , to capture term variations.
using some heuristics , we attempt to ground the impacts to the detected mutations.
possible kinetic values are found using a custom tokeniser and validated by some rules.
the magnitude of an impact is detected through heuristics and validated against the domain ontology.
the last task solved by the system is to find the protein properties changed by a mutation, which is also done through additional heuristics.
impact gazetteer list generation
to identify the polarity of the impacts, we use the developed owl ontology encoding the type information of the impacts.
using an onto-gazetteer nlp component  <cit> , the text matches the gazetteer list entries, and the impact type class in the ontology is assigned to the text.
the impact gazetteer lists for positive, negative, neutral and non-measurable impacts, consisting of  <dig> words, were also compiled from the literature.
furthermore, the impact terms appear in different forms.
for example, activates, activate, activated, activating are all potential impact words; the problem of the term variation can be alleviated by stemming.
all the aforementioned variations of activate have the same root: “activate”.
the morphological analyzer  <cit>  provides the root of the impact words, and by matching the stemming result against the prepared impact gazetteer lists, all the various representations can be detected.
in the above example, by adding activate to our list, we can detect activates, activate, activated, activating.
impact detection
now that we gathered all the impact expressions, we will use this information to mark the impacts.
the scope of the impact should be limited to the part of a sentence expressing the impact.
consider the following example  <cit> : “the effects of the s136a and y149f mutations on the km values for nadp were low, but the k153m mutation caused increases of more than 53-fold in the values, which suggests that lys <dig> is involved in the coenzyme binding.”
three impacts are expressed in the above example:
• the effects of y149f on the km values for nadp was low
• the effects of k158q on the km values for nadp was low
• k153m caused increases of more than 53-fold in the values
however, this representation can not provide users with thorough information, in particular when a comparison between multiple impacts is made.
hence, we expand the scope of the impact to each sentence.
in case impact words are detected in a sentence, the sentence is marked as an impact sentence.
when multiple impact words are detected in a sentence, the sentence is marked multiple times as an impact sentence.
relying on impact word expressions alone to detect impact sentences would lead to many false positives.
as the next example illustrates, the impact expression ‘reduced’ exists in the sentence, however, the sentence does not express an impact of a mutation  <cit> : “the limited degree of flexibility in thermophilic enzymes results in reduced catalytic efficiency when compared to that of their mesophilic counterpart at low temperatures.”
on the other hand, extracting only the sentences containing mutation mentions and impact word expressions results in many false negatives  <cit> : “indeed, the n249y substitution increases by six-fold the turnover number measured at 65c with benzyl alcohol as substrate.
furthermore, the affinity for coenzymes is substantially lower than that of the wt protein .“
in the above example, the first impact the increase of the turnover number can be grounded to the mutation n249y.
however, the second impact on the affinity for coenzymes is embedded within the context.
if we only extract the impact sentences containing mutations, we would ignore the second impact.
therefore, to capture impact sentences effectively, we extract all the sentences containing impact word expressions, and further filter them if no mutations or special vocabularies describing a change to a protein exist in the sentence.
the impact expressions existing in one noun phrase are considered as one expression.
furthermore, if an impact expression appears in a verb phrase followed by another impact expression in a noun phrase, we consider them as one impact expression.
impact grounding
as discussed earlier, bio-engineers are interested in knowing what kind of effects an engineered mutation can lead to.
for this reason, the system must be able to accurately determine which mutation introduces a specific impact.
this is accomplished by a number of heuristics.
once the entities such as mutations, mutation series and impact words are identified and annotated, impact expressions are associated with mutations.
the algorithm for semantic assignment  is as follows: we first check if there exist any impact expressions in a given sentence, if yes all the mutations in the sentence are collected and analysed according to the following cases.
case 1: if the impact sentence contains one mutation, then all the impact expressions in the sentence are grounded to that detected mutation .
the detected mutation can be a single mutation or a mutation series.
case 2: if there exists more than one mutation:
<dig>  we check if the mutations are connected with conjunctions such as and and or; if yes, the impact is grounded to every detected mutation .
some of these mutations can be mutation series, such as n190v/w191s.
<dig>  mutations or mutation series are not connected with conjunctions such as and and or; in this case, the impact is grounded to the nearest detected mutation or mutations .
case 3: if no mutations are found in the sentence, the impact is grounded to the nearest mutation or mutations, making the simple assumption that the nearest mutation or mutation series invokes the impacts mentioned.
impacton relation detection
to help bio-engineers find their favourable mutation, we need to determine which protein properties are altered.
consider the following example  <cit> : “in this study, we have confirmed the roles of ser <dig>  tyr <dig> and lys <dig> of xr as the catalytic triad by drastic loss of activity resulting from the mutagenesis of s136a, y149f and k153m in rat xr.”
two prerequisite pieces of information, the impact expression, loss and the protein property changed by the mutation, activity are detected.
now we attempt to associate the appropriate pair.
we use a simple heuristic to detect which protein property is affected by a mutation:
<dig>  we first check if there exists one impact in a given sentence, if so the sentence is searched for protein properties.
we assume that all detected protein properties are altered by the impact.
<dig>  if multiple impacts are detected in a sentence, each impact is linked to the nearest protein property.
the impacton relation is represented with the sentence containing the impact expression and the protein property.
the result annotation of the above example is shown in table  <dig>
the ontology property ‘impacton’ connects impact expressions with the affected protein property.
measuredwith relation detection
at this stage, we find relations between the protein property affected by an impact and units of measurement and effect magnitudes .
consider the following two examples:   <cit>  “the mutant ssadh displays improved thermal stability, as indicated by the increase in tm from  <dig> to 93°c, which was determined by the apparent transition curves.”   <cit>  “except for thr416val/thr417val, which had a km value of  <dig> mm, the mutants had km values identical to  <dig> mm km value of the wild-type enzyme.“
in the first example, to know how much the thermal stability was improved by the mutation, we need to link the extracted protein property, thermal stability, with the physical quantity,  <dig> to 93°c.
in the second example, we need to detect that the km property of the protein affected by the double mutant, t416v/t417v, is measured as  <dig> mm, while other mutants had the same km value as that of the wild-type enzyme, which is measured as  <dig> mm.
to fulfil our objective of relating the protein properties with their units of measurement, we use simple proximity heuristics.
the detected relation candidates are then validated against the domain ontology : if the detected physical quantity is not among the possible value fillers of the slots for the aforementioned protein property, the relation candidate is discarded.
consider the following example  <cit> : “in addition, the half lives at 60°c of the r156e and n173d xylanases were respectively  <dig> and  <dig> min longer than that of the wild-type enzyme even in the absence of substrate.”
the protein property half lives is measured with minute, hour and fold; in the above example, the closest physical quantity is 60°c, and once the relation is validated against the ontology, it is discarded as degree celsius is not one of the fillers of half life and the correct filler  <dig> and  <dig> min is assigned.
to represent the relation, we mark the sentence as a measuredwith relation with two features, property name and physical quantity.
ontology population
to provide protein engineers and scientists with comprehensive information and a more expressive model, we populate our domain ontology with the extracted information, which can then be queried as a knowledge base.
since manually populating the ontology is a cumbersome task, we integrated the owlexporter  <cit>  component to automate this task.
two ontologies are required to export our extracted information, our domain impact ontology and a nlp ontology provided with the owlexporter component  <cit> .
the nlp ontology contains concepts such as document and sentence.
while populating our domain ontology, the owlexporter automatically populates the nlp ontology.
individuals of our domain concepts, such as mutation, mutationlmpact and proteinproperty are associated with the individuals of the nlp concepts, such as sentence.
we can then invoke more advanced queries, e.g., finding all the extracted impacts in a specific sentence.
in order to be able to export the entities and the relationships to our domain ontology, we need to assign owlexportclass and owlexportrelation annotation types to the document annotations.
this is achieved with additional jape grammar rules.
by assigning these two types of annotations to our document annotations, we inform the owlexporter about the annotations we want to export.
application
our system is implemented based on the general architecture for text engineering   <cit> , a java-based open source component framework for text processing.
our system can be run stand-alone, embedded in other applications, or deployed on a cloud computing infrastructure for large-scale document processing using the gate cloud parallelizer .
additionally, we provide a number of semantic access methods, described below.
web service invocation
to use our pipelines as a web service, we created owl service descriptions for the semantic assistants framework  <cit> .
two services are currently provided, one for mutation tagging and one for impact detection.
these services are described through metadata expressed in an owl ontology.
both services can then be deployed in a semantic assistants server.
the server allows any web client to send documents to the service through standard web service invocations and receive the results in xml format.
additionally, semantic assistants-enabled clients, like openoffice or the firefox web browser , can directly send documents to the services on behalf of a user.
querying impact information
presenting impact information in a structured format allows users to quickly access the relevant information  <cit> .
for example, an end user might be interested to search for impacts of a specific mutation, or all the altered properties of an impact.
towards this end, we export the extracted information to the ontology; consequently, we can simply query the ontological knowledge base for our desired information.
figure  <dig> simply queries for the mutations that increased the activity of a protein using the sparql query language; the results of this query are shown in figure  <dig>
RESULTS
we analyzed the performance of our approaches for mutation series and impact extraction in detail on different corpora.
first, the evaluation of the mutation series detection module is investigated.
then, the effectiveness of the impact extraction, as well as grounding to the correct mutation is measured on literature describing enzymes.
data
to evaluate the performance of the system for each task, we prepared two corpora: mutation series and impact.
mutation series detection corpus
we prepared a corpus containing  <dig> full-text pubmed articles on enzymes to assess the efficiency of the system in detecting mutation series.
we ensured that all these documents contain multiple mutation mentions.
these documents contain a total of  <dig> mutations and  <dig> mutation series.
the list of documents used for evaluation is provided in an additional file .
impact extraction corpora
we selected  <dig> pubmed ids and manually annotated them with the impact information.
for each impact mention, only the part of the sentence mentioning the mutation and the impact was selected.
thus, if a sentence expresses multiple impacts, all are annotated separately .
the impacts are grounded to the respective mutations and the ec number of experimented enzymes is specified.
the list of documents used for evaluation is provided in an additional file .
evaluation
first, the correctness of the mutation series extraction is assessed.
second, the effectiveness of the impact extraction, as well as grounding to the correct mutation, is measured on literature describing enzymes.
since the mutation series detection relies on correctly recognizing mutations, we first show the mutation detection result for each system, followed by the result of our mutation series detection.
quantitative evaluation metrics
the evaluation procedure is performed by comparing the manually annotated texts with the annotations generated by our system, measured with the metrics explained in this section.
the number of correctly identified items as a percentage of the number of items identified is specified as precision .
the number of correctly identified items as a percentage of the total number of correct items is defined as recall .
the f-measure  is used as a weighted  average of precision and recall.
finally, accuracy is the percentage of decisions that are correct.
the performance results are computed according to different criteria: strict  and lenient .
in “strict”, we measure all partially correct responses as incorrect.
in “lenient”, all partially correct responses are measured as correct  <cit> .
mutation detection evaluation
both mutationfinder  <cit>  and our omm mutationtagger were applied to  <dig> manually annotated documents; the comparative results of the systems are shown in table  <dig>
comparative evaluation of the mutation detection performance, using mutationfinder and omm mutationtagger.
mutation series evaluation
we also verified the correctness of the extracted mutation series.
the results are presented in table  <dig>
impact analysis evaluation
here, we analyse how correctly our system can detect all impacts expressed in a sentence.
we further investigate the performance of our developed grounding algorithm .
the performance of our system on our manually annotated corpus of  <dig> documents is assessed and the results are summarized in tables  <dig> and  <dig>  omm system results are provided in an additional file .
in our corpora, 5% of all point mutations are expressed in natural language; thus, in an experiment, we considered the results of both mutationtagger and mutationfinder for the impact detection and grounding tasks.
as can be seen in table  <dig>  this combination of both systems slightly increases recall at the expense of precision.
discussion
false negatives of impact detection are mainly due to author-defined mutation names.
for example, pmid  <dig>  reporting on the mutant of alcohol dehydrogenase, uses mssadh to refer to n249y in the document.
authors of the paper pmid  <dig> also assign no.
<dig> to a mutation containing  <dig> amino acid substitutions; t71a, k264e, l317s, t331a, r407l, s415g, k455i and e277g.
since we rely on mutation mentions and the keywords introduced earlier in impact detection section to detect impacts, these impacts are not detected.
tables from processed pdf files are converted into indistinct textual blocks, and in case they are reporting the impacts of mutations, our system detects them as impacts.
these mentions are not manually annotated, thus they are considered as false positives.
CONCLUSIONS
mutation impacts are essential for understanding the role of mutations.
the data regarding the mutations and impacts exists primarily in scientific publications.
in this paper, we described open mutation miner , a comprehensive, modular, open source text mining system for extracting and grounding mutation impacts, affected protein properties and magnitudes of effects.
the performance of our system is evaluated on multiple corpora.
furthermore, we created additional manual annotations for the biomedical literature.
our ontology population approach provides comprehensive information to a biologist and can be queried or further integrated with other systems.
further work will address mutation co-reference resolution; in journal papers, very often the authors use pronominal or nominal mutation references that hinders the grounding of impacts.
all occurrences of mutations, including nominal and pronominal references are required to be detected.
deletion and insertion mutations pose additional challenges to be addressed in a future version.
abbreviations used
amenda: automatic mining of enzyme data; brenda: braunschweig enzyme database; dl: description logic; ec: enzyme commission; frenda: full reference enzyme data; gate: general architecture for text engineering; go: gene ontology; jape: java annotation pattern language; kid: kinetic database; nlp: natural language processing; owl: web ontology language; rdf: resource description framework; si: systéme international d’unités; sparql: sparql protocol and rdf query language; xml: extensible markup language.
authors’ contributions
nn designed the impact extraction and grounding rules, implemented the system, contributed to the ontology design, annotated the documents, and performed the evaluation.
rw provided gate and nlp expertise, co-ordinated the overall project, contributed to the ontology and the semantic assistants framework.
both authors contributed to the manuscript.
competing interests
the authors declare that they have no competing interests.
supplementary material
additional file 1
list of documents used for evaluation the list of documents  used in the corpora for mutation series and impact analysis evaluation.
click here for file
additional file 2
manual annotations manual annotations of  <dig> documents used for evaluation, presented in tab-delimited format.
click here for file
additional file 3
omm system results the output of our omm system, listing all detected impacts that are grounded to their respective mutations.
click here for file
acknowledgements
vladislav ryzhikov contributed to the mutationtagger component.
thomas kappler contributed to the mutationtagger component and the impact ontology.
christopher j.o.
baker contributed to the impact ontology.
funding for the open mutation miner project is supplied in part by the natural sciences and engineering research council of canada  discovery grants  program.
this article has been published as part of bmc genomics volume  <dig> supplement  <dig>  2012: snp-sig 2011: identification and annotation of snps in the context of structure, function and disease.
the full contents of the supplement are available online at http://www.biomedcentral.com/bmcgenomics/supplements/13/s <dig>

 vast amounts of research is dedicated to the identification of mutations and their impacts. biologists usually make inferences about functions of novel sequences by comparing them to the functions of known sequences. many mutagenesis experiments are performed to improve the properties of proteins, particularly enzymes. additionally detection of disease causal mutations attracted a lot of attention. the result of all these efforts lies in publications, particularly in textual format. consequently, locating and retrieving this information is a very cumbersome task. some databases try to manually curate such information and provide it in publicly accessible form. however, even for an expert curator, extracting this information manually is laborious. hence, database curators now increasingly reach for text-mining procedures. large-scale attempts resulted in high levels of performance in the realization of the automatic extraction of mutations. yet, finding their impacts, affected protein properties and magnitudes of effects remains challenging. mema uses regular expressions to extract mutations and mutation-gene pairs. it focuses on the co-occurrence of mutations and genes within a sentence and proximity parameters within an abstract. the performance of the system is evaluated on a set of abstracts. the reported recall and precision for the mutation detection task are >67% and >96%, respectively. mutext searches for mutation data using a pattern matching approach and further validates the extracted point mutations using two plausibility filters: a sequence filter and distance filter. the performance of the system is evaluated on two corpora. their algorithm detects %- % of point mutations with a specificity of %- %. mutation grab takes a dictionary-based approach to identify protein and gene names while extracting point mutation terms using regular expressions, utilizing graph bigrams to disambiguate the extracted protein point mutations. the authors evaluate the effectiveness of their approach on the articles describing three protein families, namely, tyrosine protein kinases, gpcrs and transmembrane ion. mugex uses regular expressions to detect mutations and statistical techniques to disambiguate between protein mutations and nucleotide mutations or cell lines. gene-mutation pairs are detected through proximity measures. the mutationfinder system extends mutext ’s rules to extract and normalize point mutations. in recent work, the authors present a strategy to integrate information about phenotypic effect of snps from uniprotkb and pathways from reactom and biopax for visualization in cytoscape. yip et al. uses regular expressions to extract and retrieve single amino acid poly morphisms. the system is assessed on a swiss-prot corpus with pubmed references. additionally, each pattern is evaluated separately. the mstrap system was developed with the aim of annotating mutations and representing them as instances of an ontology. they further use mstrapviz to read the populated ontology and visualize the annotations on protein structures. enzyminer tries to categorize pubmed abstracts based on the impact of a protein level mutation on the stability and activity of a given enzyme. using different classification algorithms, enzyminer is able to narrow down search results; however, detailed information about the direction of the impacts, association of impacts to mutations and the kind of change in stability or functionality is not provided. although enzyminer targets mutation impact information, it differs significantly from our approach, as we are concerned with sentence-level detection and semantic analysis of mutation impacts, not document classification. in, the authors introduced the first rule-based approach to extract mutation impacts on protein properties while categorizing the directionality of the impacts and grounding the impacts to the mutations. the extracted information was populated to a domain ontology for further querying through a web service. while in the aforementioned work, molecular properties and the michaelis constant, the rate constant and the compound variable are considered, the other protein properties, such as the remaining kinetic constants and protein stability, are ignored. on the corpus of documents on haloalkane dehalogenase, the authors report a recall of 34% and a precision of 86% for the mutation-impact relation extraction task. a recent work on the extraction of kinetic information and associated information, namely, enzyme names, ec numbers and localization is presented in. the proposed rule- and dictionary-based approach in this system is applied to pubmed abstracts and the results are provided in kid, the kinetic database. kipar, an information retrieval system, focuses on kinetic modeling of metabolic pathways using a rule-based approach. however, all the existing approaches are unable to extract the protein properties affected by mutations. in this paper, we present a rule-based approach to extract mutation series, modified protein properties and magnitudes of effects. in our system, the relation between the magnitudes of effects and the protein properties are detected and validated against the domain ontology. to provide for effective querying and analysis, we populate a domain ontology with the extracted information. table summarizes the scope of our open mutation miner system, compared to existing approaches. further details on these tasks are provided in the following section. methods in order to comprehensively extract mutation impacts, the detection of several named entities and their relations, in particular mutations and protein properties, is required. as an example, consider the following text segment: “several single mutants, double mutants, and triple mutants were prepared and expressed as glutathione s-transferase fusion proteins in escherichia coli and purified by gsh-agarose affinity chromatography. mutant q15k-w37r and mutant q15r-w37r showed comparable activity for nad and nadp with an increase in activity nearly 3fold over that of the wild type.“ in this example, we need to extract increase as an impact that is caused “comparably” by two mutation pairs, q15k-w37r and q15r-w37r. in other words, the two aforementioned mutations have the same impact on the activity of an enzyme, glutathione s-transferase that is residing in the host organism, escherichia coli. we are also interested to know that activity as a kinetic property of the mutant enzyme is measured 3fold higher than the activity of the wild-type enzyme. note that other entities, such as single mutations, exist in the text segment, but here we are only interested in the entities that are related to the identified impact. the result of the system should be a set of detected entities, correctly normalized and grounded, and linked with each other. after detecting organism mentions, which is handled by a separate module, the organismtagger, the first step of impact analysis is to detect impact mentions. however, extracting only impacts is not sufficient; we want to know which mutation caused the impact. hence, the system needs to ground the detected impacts to mutations. additionally, mutations can appear in the form of mutation series. thus, the system must also be able to identify these complex mutation expressions. finding out which protein properties were affected by the mutations and to what extent is necessary to identify advantageous mutations. towards this end, we export the analysis results into an ontology for further applications, including queries and summarization. an overview of our system is presented in figure in what follows, we will provide a detailed description of each task. impact ontology our impact ontology is an extension to the ontology described in, conceptualizing impacts and the mutations associated with them. the use of the impact ontology facilitates advanced queries and impact extraction. the ontology contains information about several concepts: text elements, biological entities and entity relations, e.g., sentence, mutationlmpact and measuredwith, respectively. we extended the ontology with new classes, such as michaelismentenconstant, specificactivity, and maximalvelocity. our ontology has a rich set of relationships between the concepts. main concepts modeling impacts on a semantic level are: mutation: an alteration or a change to a gene and developing a different offspring. unitofmeasurement: a standard for measuring the physical quantity. mutationlmpact: the expansion of an impact can be presented as a bifurcating tree: each bifurcating node represents a mutation effect on protein properties, whether the impact is measurable or not. proteinproperty: a class for protein properties, which subsumes kinetic properties, protein function, and protein stability. information about the effect of mutations on proteins can be modeled at different granularity levels. for example, the effect can be on the structure, which consequently can affect various properties of the proteins. for a finer level of granularity, we represent all these relations. the relations between these entities, expressed as owl object properties, are listed in table each protein property is measured with specific units of measurement, for example michaelis menten constant is measured with units such as per second, per minute, etc. however, in interpreting the mutation impacts, not only are these units of measurement utilized, but also ratio measurements can be used. for example, the measured values of the affected protein property are compared with the measured values of the wild type or other mutated protein properties, and specified by percent, fold or orders of magnitude. we decided to establish some restrictions on the units of measurement with which each protein property is measured, as well as the ratio measurement units. these restrictions are encoded in the ontology based on global standards, where protein properties are measured by specific units of measurements. these constraints are encoded as possible value fillers for the measuredwith slot for a specific protein property. for instance, km can be measured with fold, per second and per minute, etc. we also defined a datatype property for protein properties, called physicalquantity, referring to the value and the unit of measurement found in the text. mutation extraction component single point mutations can be expressed in single-letter standard format or through more complex representations. we integrated one external mutation detection system and also developed our own approach. mutationtagger our mutationtagger, based on previous work, extracts single point mutations using grammar rules and normalizes them to their single-letter format. however, mutational mentions in the form of natural language, such as “met for val substitution found at position 270” are currently ignored by our system. mutationfinder the mutationfinder system accomplishes the task of single mutation detection and normalization by using regular expressions. mutationfinder also tries to identify mutational changes expressed in natural language. however, it still fails at extracting all mentions. mutation series in the simplest case, impacts are results of single mutations. however, impacts are occasionally caused by mutation series. for example, in our corpus of full-text documents, 6% of the mutation mentions are mutation series. during the manual inspection of several hundred documents containing mutation series, it became obvious that these mutation series may have different appearances and representations. they can be described using special symbols, such as “/” and “:”, or keywords, such as double mutants, triple mutants, etc. table summarizes these different forms. therefore, mutations connected with these special characters or preceded by the keywords are considered as mutation series and detected through regular expressions. to ensure that these detected mutation series have one identical internal representation, we further normalize them to the format, where all the mutations in a series are separated by the notation “/”. protein properties extraction component mutations can alter the structure of proteins that subsequently results in affecting their functions, either by gaining a function or losing one. mutations may also affect the stability of proteins, where the ratio of the unfolded protein increases or decreases compared to the folded protein. mutagenesis experiments are constantly performed to identify the importance of protein residues, either to find the source of a disease or a cure to one. furthermore, studies are done to improve enzyme functions. in our system, protein properties are expressed in rdf format and detected through gazetteering. the extracted information can then be correlated to impacts in subsequent processing steps. molecular function understanding the role of mutations, in particular their contribution to diseases like cancer, requires identifying their impact on molecular functions. causative mutations can drive cancers by activating a protein function or in-activating a function. they can promote cancer progression by their resistance to drugs or, according to a recent study, switching of functions. detection of the functional impact of mutations has not only drawn attention in cancer study, but has also been an important matter in re-sequencing efforts. to detect molecular functions, we use the concepts presented by the gene ontology. we generate an rdf representation of molecular functions from a download of the gene ontology. the gene ontology is provided in obo-xml format, where each node is one entry. we first check for molecular_function namespaces, then, we extract the name and go id, as well as the synonyms of the entry. using this information, we generate our rdf file. for obtaining further information, molecular functions are specified by their gene ontology id. the format of a triple is c rdfs:subclassof c where rdfs:subclassof is an instance of rdf:property and states that c here recognized as the gene ontology id, is an instance of rdfs:class and a subclass of c an instance of rdfs:class, “molecular_function”. the resulting rdf is then used for gazetteering using an lkb gazetteer component. kinetic constants depending on their interests, enzyme and protein engineers apply recombinant dna technology to improve enzyme kinetic values and stability or identify the roles of residues. consider a study on the role of asn in humans: “to examine the role of asn in the catalytic mechanism of human xr, mutant forms were prepared. the two mutations increased km for the substrate and kd for nadph, but only the n107l mutation significantly decreased kcat value.“ here, two prepared mutations, n107d and n107l, affect three kinetic values, michaelis menten constant, turn-over number and dissociation constant, of the protein. to capture these kinetic properties, we manually compiled them from the scientific literature. the list of these properties is by no means exhaustive. however, property synonyms add complexity to later tasks where the relations are extracted and validated against the ontology. a simple rdf schema allows us to deal with different term representations of a concept and to resolve all aliases of the same concept. a triple is defined as c rdfs:subclassof c where c is an instance of rdfs:class, “proteinproperty”. the rdfs:label is an instance of rdf:property, where rdfs:domain is rdfs:resource and the rdfs:range is literal. normalizing all aliases to one single representation can also be helpful when populating the output ontology. consider half-life as an example, it can appear in different variations, e.g., t, t1/ and half-lives. all these variations are represented as labels in the aforementioned rdf, thus, in case any of them matches, the mention is normalized to half-life. kinetic values knowing the magnitude of protein properties affected by mutations enables biologists to better compare the mutation impacts of their interests. as an example, consider this bio-engineering study that was conducted on quinoprotein glucose dehydrogenase to improve the thermal stability of the enzyme: “the halflife at 55°c of ser415cys was approx 36-fold greater than that of the wild-type enzyme and 4-fold greater than that of the ser231lys variant.” the ser residue at position is chosen for constructing different variants of the enzyme and compared with the s231k variant. analyzing which variant results in the most thermostable enzyme requires the extraction of the magnitudes. half-life of s415c is measured as min, whereas s231k was measured as min, and the measured half-lives of the two mutations are also compared to that of the wild-type enzyme. the magnitudes of protein properties are expressed in signed numbers, decimals and ranges of values for a single parameter. since the existing gate generic tokeniser can only detect digits, we developed a simple tokeniser to capture possible representations of magnitudes. to ensure that we extract the reported ranges of magnitudes, we collected possible range representations from the literature and expressed them through grammar rules. after detecting all possible values, we check which values express a physical quantity using the patterns and discard all other values. units of measurement units of measurement are expressed in various formats, in mass or molar concentration, in different systems and different scales. finding how a magnitude is measured requires detecting units of measurement. using the same approach as for kinetic properties, the list of units of measurement was collected from the literature and encoded in an rdf schema. the rdf schema is limited to one subclass hierarchy and assigns the units of measurement to their identified concept in the owl-dl ontology. consider the unit of measurement, per second, the same concept as persecond is encoded in the owl-dl ontology. if any of the representations of per second is detected in the document, the class persecond is assigned to it, facilitating the ontology population step. physical quantities we use the information about the units of measurement to extract physical quantities. usually, units of measurement follow values, except for a few with no specific units of measurement, such as ph. more succinctly: after reviewing the literature, we designed a set of patterns to capture physical quantities. impact extraction component mutations are considered as sources of species evolution. some result in beneficial changes, while others have detrimental effects. it is important to not only find impacts, but also to mark the origin mutation and altered protein properties for further analysis. a system capable of analyzing mutation impacts requires information from many entities. impact analysis consists of the following steps: finding impact expressions. finding mutations or mutation keywords. identifying the polarity of the impact to detect advantageous and disadvantageous impacts. grounding the impacts to mutations to find which mutations lead to a specific impact. finding the affected protein properties. finding the magnitude of the effect to help bio-engineers compare the effects and find the most favourable mutations. for the first step, we use ontology based gazetteering, with the help of the morphological analyzer, to capture term variations. using some heuristics, we attempt to ground the impacts to the detected mutations. possible kinetic values are found using a custom tokeniser and validated by some rules. the magnitude of an impact is detected through heuristics and validated against the domain ontology. the last task solved by the system is to find the protein properties changed by a mutation, which is also done through additional heuristics. impact gazetteer list generation to identify the polarity of the impacts, we use the developed owl ontology encoding the type information of the impacts. using an onto-gazetteer nlp component, the text matches the gazetteer list entries, and the impact type class in the ontology is assigned to the text. the impact gazetteer lists for positive, negative, neutral and non-measurable impacts, consisting of words, were also compiled from the literature. furthermore, the impact terms appear in different forms. for example, activates, activate, activated, activating are all potential impact words; the problem of the term variation can be alleviated by stemming. all the aforementioned variations of activate have the same root: “activate”. the morphological analyzer provides the root of the impact words, and by matching the stemming result against the prepared impact gazetteer lists, all the various representations can be detected. in the above example, by adding activate to our list, we can detect activates, activate, activated, activating. impact detection now that we gathered all the impact expressions, we will use this information to mark the impacts. the scope of the impact should be limited to the part of a sentence expressing the impact. consider the following example: “the effects of the s136a and y149f mutations on the km values for nadp were low, but the k153m mutation caused increases of more than 53-fold in the values, which suggests that lys is involved in the coenzyme binding.” three impacts are expressed in the above example: • the effects of y149f on the km values for nadp was low • the effects of k158q on the km values for nadp was low • k153m caused increases of more than 53-fold in the values however, this representation can not provide users with thorough information, in particular when a comparison between multiple impacts is made. hence, we expand the scope of the impact to each sentence. in case impact words are detected in a sentence, the sentence is marked as an impact sentence. when multiple impact words are detected in a sentence, the sentence is marked multiple times as an impact sentence. relying on impact word expressions alone to detect impact sentences would lead to many false positives. as the next example illustrates, the impact expression ‘reduced’ exists in the sentence, however, the sentence does not express an impact of a mutation: “the limited degree of flexibility in thermophilic enzymes results in reduced catalytic efficiency when compared to that of their mesophilic counterpart at low temperatures.” on the other hand, extracting only the sentences containing mutation mentions and impact word expressions results in many false negatives: “indeed, the n249y substitution increases by six-fold the turnover number measured at 65c with benzyl alcohol as substrate. furthermore, the affinity for coenzymes is substantially lower than that of the wt protein.“ in the above example, the first impact the increase of the turnover number can be grounded to the mutation n249y. however, the second impact on the affinity for coenzymes is embedded within the context. if we only extract the impact sentences containing mutations, we would ignore the second impact. therefore, to capture impact sentences effectively, we extract all the sentences containing impact word expressions, and further filter them if no mutations or special vocabularies describing a change to a protein exist in the sentence. the impact expressions existing in one noun phrase are considered as one expression. furthermore, if an impact expression appears in a verb phrase followed by another impact expression in a noun phrase, we consider them as one impact expression. impact grounding as discussed earlier, bio-engineers are interested in knowing what kind of effects an engineered mutation can lead to. for this reason, the system must be able to accurately determine which mutation introduces a specific impact. this is accomplished by a number of heuristics. once the entities such as mutations, mutation series and impact words are identified and annotated, impact expressions are associated with mutations. the algorithm for semantic assignment is as follows: we first check if there exist any impact expressions in a given sentence, if yes all the mutations in the sentence are collected and analysed according to the following cases. case 1: if the impact sentence contains one mutation, then all the impact expressions in the sentence are grounded to that detected mutation. the detected mutation can be a single mutation or a mutation series. case 2: if there exists more than one mutation: we check if the mutations are connected with conjunctions such as and and or; if yes, the impact is grounded to every detected mutation. some of these mutations can be mutation series, such as n190v/w191s. mutations or mutation series are not connected with conjunctions such as and and or; in this case, the impact is grounded to the nearest detected mutation or mutations. case 3: if no mutations are found in the sentence, the impact is grounded to the nearest mutation or mutations, making the simple assumption that the nearest mutation or mutation series invokes the impacts mentioned. impacton relation detection to help bio-engineers find their favourable mutation, we need to determine which protein properties are altered. consider the following example: “in this study, we have confirmed the roles of ser tyr and lys of xr as the catalytic triad by drastic loss of activity resulting from the mutagenesis of s136a, y149f and k153m in rat xr.” two prerequisite pieces of information, the impact expression, loss and the protein property changed by the mutation, activity are detected. now we attempt to associate the appropriate pair. we use a simple heuristic to detect which protein property is affected by a mutation: we first check if there exists one impact in a given sentence, if so the sentence is searched for protein properties. we assume that all detected protein properties are altered by the impact. if multiple impacts are detected in a sentence, each impact is linked to the nearest protein property. the impacton relation is represented with the sentence containing the impact expression and the protein property. the result annotation of the above example is shown in table the ontology property ‘impacton’ connects impact expressions with the affected protein property. measuredwith relation detection at this stage, we find relations between the protein property affected by an impact and units of measurement and effect magnitudes. consider the following two examples: “the mutant ssadh displays improved thermal stability, as indicated by the increase in tm from to 93°c, which was determined by the apparent transition curves.” “except for thr416val/thr417val, which had a km value of mm, the mutants had km values identical to mm km value of the wild-type enzyme.“ in the first example, to know how much the thermal stability was improved by the mutation, we need to link the extracted protein property, thermal stability, with the physical quantity, to 93°c. in the second example, we need to detect that the km property of the protein affected by the double mutant, t416v/t417v, is measured as mm, while other mutants had the same km value as that of the wild-type enzyme, which is measured as mm. to fulfil our objective of relating the protein properties with their units of measurement, we use simple proximity heuristics. the detected relation candidates are then validated against the domain ontology: if the detected physical quantity is not among the possible value fillers of the slots for the aforementioned protein property, the relation candidate is discarded. consider the following example: “in addition, the half lives at 60°c of the r156e and n173d xylanases were respectively and min longer than that of the wild-type enzyme even in the absence of substrate.” the protein property half lives is measured with minute, hour and fold; in the above example, the closest physical quantity is 60°c, and once the relation is validated against the ontology, it is discarded as degree celsius is not one of the fillers of half life and the correct filler and min is assigned. to represent the relation, we mark the sentence as a measuredwith relation with two features, property name and physical quantity. ontology population to provide protein engineers and scientists with comprehensive information and a more expressive model, we populate our domain ontology with the extracted information, which can then be queried as a knowledge base. since manually populating the ontology is a cumbersome task, we integrated the owlexporter component to automate this task. two ontologies are required to export our extracted information, our domain impact ontology and a nlp ontology provided with the owlexporter component. the nlp ontology contains concepts such as document and sentence. while populating our domain ontology, the owlexporter automatically populates the nlp ontology. individuals of our domain concepts, such as mutation, mutationlmpact and proteinproperty are associated with the individuals of the nlp concepts, such as sentence. we can then invoke more advanced queries, e.g., finding all the extracted impacts in a specific sentence. in order to be able to export the entities and the relationships to our domain ontology, we need to assign owlexportclass and owlexportrelation annotation types to the document annotations. this is achieved with additional jape grammar rules. by assigning these two types of annotations to our document annotations, we inform the owlexporter about the annotations we want to export. application our system is implemented based on the general architecture for text engineering, a java-based open source component framework for text processing. our system can be run stand-alone, embedded in other applications, or deployed on a cloud computing infrastructure for large-scale document processing using the gate cloud parallelizer. additionally, we provide a number of semantic access methods, described below. web service invocation to use our pipelines as a web service, we created owl service descriptions for the semantic assistants framework. two services are currently provided, one for mutation tagging and one for impact detection. these services are described through metadata expressed in an owl ontology. both services can then be deployed in a semantic assistants server. the server allows any web client to send documents to the service through standard web service invocations and receive the results in xml format. additionally, semantic assistants-enabled clients, like openoffice or the firefox web browser, can directly send documents to the services on behalf of a user. querying impact information presenting impact information in a structured format allows users to quickly access the relevant information. for example, an end user might be interested to search for impacts of a specific mutation, or all the altered properties of an impact. towards this end, we export the extracted information to the ontology; consequently, we can simply query the ontological knowledge base for our desired information. figure simply queries for the mutations that increased the activity of a protein using the sparql query language; the results of this query are shown in figure  we analyzed the performance of our approaches for mutation series and impact extraction in detail on different corpora. first, the evaluation of the mutation series detection module is investigated. then, the effectiveness of the impact extraction, as well as grounding to the correct mutation is measured on literature describing enzymes. data to evaluate the performance of the system for each task, we prepared two corpora: mutation series and impact. mutation series detection corpus we prepared a corpus containing full-text pubmed articles on enzymes to assess the efficiency of the system in detecting mutation series. we ensured that all these documents contain multiple mutation mentions. these documents contain a total of mutations and mutation series. the list of documents used for evaluation is provided in an additional file. impact extraction corpora we selected pubmed ids and manually annotated them with the impact information. for each impact mention, only the part of the sentence mentioning the mutation and the impact was selected. thus, if a sentence expresses multiple impacts, all are annotated separately. the impacts are grounded to the respective mutations and the ec number of experimented enzymes is specified. the list of documents used for evaluation is provided in an additional file. evaluation first, the correctness of the mutation series extraction is assessed. second, the effectiveness of the impact extraction, as well as grounding to the correct mutation, is measured on literature describing enzymes. since the mutation series detection relies on correctly recognizing mutations, we first show the mutation detection result for each system, followed by the result of our mutation series detection. quantitative evaluation metrics the evaluation procedure is performed by comparing the manually annotated texts with the annotations generated by our system, measured with the metrics explained in this section. the number of correctly identified items as a percentage of the number of items identified is specified as precision. the number of correctly identified items as a percentage of the total number of correct items is defined as recall. the f-measure is used as a weighted average of precision and recall. finally, accuracy is the percentage of decisions that are correct. the performance results are computed according to different criteria: strict and lenient. in “strict”, we measure all partially correct responses as incorrect. in “lenient”, all partially correct responses are measured as correct. mutation detection evaluation both mutationfinder and our omm mutationtagger were applied to manually annotated documents; the comparative results of the systems are shown in table comparative evaluation of the mutation detection performance, using mutationfinder and omm mutationtagger. mutation series evaluation we also verified the correctness of the extracted mutation series. the results are presented in table impact analysis evaluation here, we analyse how correctly our system can detect all impacts expressed in a sentence. we further investigate the performance of our developed grounding algorithm. the performance of our system on our manually annotated corpus of documents is assessed and the results are summarized in tables and omm system results are provided in an additional file. in our corpora, 5% of all point mutations are expressed in natural language; thus, in an experiment, we considered the results of both mutationtagger and mutationfinder for the impact detection and grounding tasks. as can be seen in table this combination of both systems slightly increases recall at the expense of precision. discussion false negatives of impact detection are mainly due to author-defined mutation names. for example, pmid reporting on the mutant of alcohol dehydrogenase, uses mssadh to refer to n249y in the document. authors of the paper pmid also assign no. to a mutation containing amino acid substitutions; t71a, k264e, l317s, t331a, r407l, s415g, k455i and e277g. since we rely on mutation mentions and the keywords introduced earlier in impact detection section to detect impacts, these impacts are not detected. tables from processed pdf files are converted into indistinct textual blocks, and in case they are reporting the impacts of mutations, our system detects them as impacts. these mentions are not manually annotated, thus they are considered as false positives.  mutation impacts are essential for understanding the role of mutations. the data regarding the mutations and impacts exists primarily in scientific publications. in this paper, we described open mutation miner, a comprehensive, modular, open source text mining system for extracting and grounding mutation impacts, affected protein properties and magnitudes of effects. the performance of our system is evaluated on multiple corpora. furthermore, we created additional manual annotations for the biomedical literature. our ontology population approach provides comprehensive information to a biologist and can be queried or further integrated with other systems. further work will address mutation co-reference resolution; in journal papers, very often the authors use pronominal or nominal mutation references that hinders the grounding of impacts. all occurrences of mutations, including nominal and pronominal references are required to be detected. deletion and insertion mutations pose additional challenges to be addressed in a future version. abbreviations used amenda: automatic mining of enzyme data; brenda: braunschweig enzyme database; dl: description logic; ec: enzyme commission; frenda: full reference enzyme data; gate: general architecture for text engineering; go: gene ontology; jape: java annotation pattern language; kid: kinetic database; nlp: natural language processing; owl: web ontology language; rdf: resource description framework; si: systéme international d’unités; sparql: sparql protocol and rdf query language; xml: extensible markup language. authors’ contributions nn designed the impact extraction and grounding rules, implemented the system, contributed to the ontology design, annotated the documents, and performed the evaluation. rw provided gate and nlp expertise, co-ordinated the overall project, contributed to the ontology and the semantic assistants framework. both authors contributed to the manuscript. competing interests the authors declare that they have no competing interests. supplementary material additional file 1 list of documents used for evaluation the list of documents used in the corpora for mutation series and impact analysis evaluation. click here for file additional file 2 manual annotations manual annotations of documents used for evaluation, presented in tab-delimited format. click here for file additional file 3 omm system results the output of our omm system, listing all detected impacts that are grounded to their respective mutations. click here for file acknowledgements vladislav ryzhikov contributed to the mutationtagger component. thomas kappler contributed to the mutationtagger component and the impact ontology. christopher j.o. baker contributed to the impact ontology. funding for the open mutation miner project is supplied in part by the natural sciences and engineering research council of canada discovery grants program. this article has been published as part of bmc genomics volume supplement 2012: snp-sig 2011: identification and annotation of snps in the context of structure, function and disease. the full contents of the supplement are available online at http://www.biomedcentral.com/bmcgenomics/supplements/13/s 
mutations as sources of evolution have long been the focus of attention in the biomedical literature. however, manually curating the rich and fast growing repository of biomedical literature is expensive and time-consuming. we present open mutation miner, the first comprehensive, fully open-source approach to automatically extract impacts and related relevant information from the biomedical literature. accessing the mutational information and their impacts on protein properties facilitates research in various domains, such as enzymology and pharmacology. while the detection of single-point mutations is well covered by existing systems, challenges still exist in grounding impacts to their respective mutations and recognizing the affected protein properties, in particular kinetic and stability properties together with physical quantities. it also extracts the affected protein properties, in particular kinetic and stability properties, as well as the magnitude of the effects and validates these relations against the domain ontology. the output of our system can be provided in various formats, in particular by populating an owl-dl ontology, which can then be queried to provide structured information. we assessed the performance of our work on manually annotated corpora and the results show the reliability of our approach. furthermore, access to the analysis results is provided through multiple interfaces, including web services for automated data integration and desktop-based solutions for end user interactions. we present an ontology model for mutation impacts, together with a comprehensive text mining system for extracting and analysing mutation impact information from full-text articles.
 mutations as sources of evolution have long been the focus of attention in the biomedical literature. accessing the mutational information and their impacts on protein properties facilitates research in various domains, such as enzymology and pharmacology. however, manually curating the rich and fast growing repository of biomedical literature is expensive and time-consuming. as a solution, text mining approaches have increasingly been deployed in the biomedical domain. while the detection of single-point mutations is well covered by existing systems, challenges still exist in grounding impacts to their respective mutations and recognizing the affected protein properties, in particular kinetic and stability properties together with physical quantities.  we present an ontology model for mutation impacts, together with a comprehensive text mining system for extracting and analysing mutation impact information from full-text articles. organisms, as sources of proteins, are extracted to help disambiguation of genes and proteins. our system then detects mutation series to correctly ground detected impacts using novel heuristics. it also extracts the affected protein properties, in particular kinetic and stability properties, as well as the magnitude of the effects and validates these relations against the domain ontology. the output of our system can be provided in various formats, in particular by populating an owl-dl ontology, which can then be queried to provide structured information. the performance of the system is evaluated on our manually annotated corpora. in the impact detection task, our system achieves a precision of %- %, a recall of %- %, and grounds the detected impacts with an accuracy of %-77%. the developed system, including resources, evaluation data and end-user and developer documentation is freely available under an open source license at http://www.semanticsoftware.info/open-mutation-miner.  we present open mutation miner, the first comprehensive, fully open-source approach to automatically extract impacts and related relevant information from the biomedical literature. we assessed the performance of our work on manually annotated corpora and the results show the reliability of our approach. the representation of the extracted information into a structured format facilitates knowledge management and aids in database curation and correction. furthermore, access to the analysis results is provided through multiple interfaces, including web services for automated data integration and desktop-based solutions for end user interactions. snp-sig 2011: identification and annotation of snps in the context of structure, function and disease vienna, austria 