BACKGROUND
semgen
in an effort to minimize errors due to natural language processing , we developed and evaluated a procedure for postprocessing extracted genetic information.
this processing is applied to the output of semgen , an nlp system for extracting semantic predications  from the text of medline citations  <cit> .
semgen applies in the domain of molecular genetics and has several components: journal descriptor indexing   <cit> , the medpost tagger  <cit> , the specialist lexicon  <cit> , the umls metathesaurus  <cit> , metamap  <cit> , and abgene  <cit> .
these components interact to first identify genetic phenomena and disorders and subsequently construct semantic relations among these entities: gene-gene interactions  as well as gene-disease associations .
processing consists of three phases: construction of an underspecified syntactic structure identification of relevant semantic concepts, and final interpretation of a semantic predication.
the system first calls jdi, a statistics-based labeled categorizer used to limit input text to the molecular genetics domain before proceeding with natural language processing.
after jdi has applied, text is sent both to the medpost tagger and to abgene, which assists in identifying gene names .
as the first step in creating an underspecified syntactic structure, the medpost tagger, drawing on the specialist lexicon, labels the words in input  with part-of-speech categories, as shown in .
we now show that apoa-ii promotes insulin resistance and has diverse effects on fat homeostasis
pron adv verb compl noun verb noun noun conj aux adj noun prep noun noun
the tagged list in  serves as input to an underspecified  parser which identifies phrases in an input string.
these correspond to low level nodes in a syntactic parse tree, and except for noun phrases , are left unlabeled.
in addition, words are assigned their role inside the noun phrase, as either modifier  or head.
informally, a head is the most important word in a noun phrase.
a schematic example is given in , where phrases are delimited by square brackets.
np  np   np np]
the syntactic structure in  serves as the basis for the next phase, identification of noun phrases expressing relevant semantic concepts: genetic phenomena and disorders.
genetic phenomena are defined broadly as those concepts that may have a bearing on molecular genetics and include genes, proteins, nucleotide sequences, mutations, polymorphisms, and chromosomes.
in this study we concentrated exclusively on genes.
for each gene name identified, semgen attempts to provide the corresponding official symbol and entrez gene id, although this is not always possible.
gene symbol resolution is a challenging nlp task, currently under active investigation .
for disorders, semgen considers a concept having one of the following umls semantic types to be relevant: 'pathologic function', 'disease or syndrome', 'neoplastic process', and 'congenital abnormality'.
during processing to identify relevant semantic concepts, semgen examines each noun phrase in structures such as  to determine whether it qualifies as a genetic phenomenon or as a disorder.
for disorder names, metamap is used exclusively to determine whether the phrase maps to a concept in the umls metathesaurus having a relevant semantic type.
in considering genetic phenomena, semgen first calls on metamap and the umls metathesaurus; however, since the metathesaurus is not complete for gene names, the output from abgene is also consulted.
abgene identifies gene and protein names in text using both statistical and empirical methods.
strategies include low frequency trigrams and rules generated both automatically and by hand.
in addition, the program relies on linguistic information, such as cue words, suffixes, and part-of-speech information.
semgen directs abgene to identify all gene names in the medline citation that contains the sentence currently being processed.
these are then available as a resource to semgen.
for example, abgene identifies the gene names in  from a citation  with title structural variants in the retinoid receptor genes in patients with schizophrenia and other psychiatric diseases.
"nurr <dig> gene", "rars", "rar/rxr", "retinoid receptors", "retinoid receptor genes", "rxrgamma", "rxr genes"
in processing input from this citation that contains the phrase nurr <dig> gene, semgen determines from metamap that nurr <dig> does not occur in the metathesaurus.
the abgene list  for the citation containing the current input sentence is then consulted, and nurr <dig> is successfully identified as a genetic phenomenon in the sentence containing this phrase.
the structure in  is an example of the syntactic parse  enhanced with identification of a gene and a disorder.
the phrase containing "apoa-ii" has now been expanded to include the entrez gene id, official symbol, gene name isolated by semgen as well as the original text.
the noun phrase containing "insulin resistance" has been expanded to include the umls concept,  as well as the original text.
np  np   np np]
in identifying relations, the final phase of processing, semgen relies on concepts identified in the previous phase, some of which serve as arguments of predications.
indicators constitute a crucial aspect of this phase.
these are syntactic elements that map to the predicate of a semantic predication.
verbs commonly serve this function.
for example, in , the verb "promote" indicates the semantic predicate predispose.
this forms the basis for construction of a predication with arguments "jnk" and "diabetes."
there appear to be multiple mechanisms through which jnk might promote diabetes.
other examples of verbs that indicate semantic relations between a gene and a disorder include "predispose"  and "influence" and "implicate" .
similarly, verbs indicating a relation between two genes are "stimulate" and "upregulate" , "block" and "inhibit" , and "mediate" .
in addition to verbs, prepositions may serve as indicators of semantic predicates.
for example, in , the preposition "in" indicates the predicate associated_with .
a clinic-based study of the lrrk <dig> gene in parkinson disease yields new mutations.
the preposition "for" can also indicate associated_with, as in .
saitohin represents a candidate gene for parkinson's disease.
it should be noted that in both  and  the preposition is the only syntactic cue that there is a semantic relation between the gene and the disorder.
in the molecular genetics domain, we have encountered prepositions as indicators only for the semantic predicate associated_with.
semgen has rules that encode indicators for the semantic predicates relevant to this domain.
in constructing a predication these rules identify predicates and are then supported by argument identification rules.
such rules apply several constraints: no argument can be used in more than one predication unless it is coordinate or the head of a relative clause.
for example, in , since parp- <dig> and nuclear factor kappa b are coordinate each is allowed to participate in a separate associated_with predication with inflammatory disorders.
in , tag is the head of relative clause  and is thus allowed to serve as the subject of an inhibit relation with p <dig> and also as the subject of an associated_with relation with insulinoma.
parp- <dig> and nuclear factor kappa b have both been suggested to play a crucial role in inflammatory disorders.
however, tag, which inactivates the key tumour suppressor p <dig>  is not known to be involved in the pathogenesis of human insulinoma.
subjects appear to the left of an indicator, and objects to the right for active constructions; for passives the order is reversed.
human obesity syndrome in , for example, will be interpreted as the object of a causes relation, because it occurs to the left of the passive indicator caused; mkks, to the right, will be the subject.
we and others have demonstrated that human obesity syndrome is caused by mutations in the gene mkks.
finally, arguments are constrained by semantic class: both arguments of a gene-gene interaction predicate  must be genetic phenomena.
the subject of a gene-disorder predicate  must be a genetic phenomenon, while its object is a disorder.
an example of semgen output is given in figure  <dig>  which provides the final interpretation for the example from  above.
gfinder
in order to exploit extracted semantic predications, we used the gfinder  application  <cit> .
this is a web application that enriches lists of gene ids with controlled functional annotations dynamically retrieved from several biomolecular databanks, including entrez gene  <cit> , gene ontology  <cit> , kegg  <cit> , swiss-prot  <cit> , pfam  <cit> , and omim  <cit> .
moreover, gfinder allows computational and statistical analyses on the functional and phenotypic annotations of user-selected groups of genes, aimed at highlighting those annotation categories that are significant in the genes selected.
overview
after establishing a baseline for semgen processing on the basis of a corpus of medline citations on diabetes, we evaluated the results of postprocessing this semgen output with a distance filtering procedure we developed.
we also compared these results to those obtained from applying our filtering method to genetic  and etiologic  relations obtained with co-occurrence processing applied to the same corpus on diabetes.
finally, we tested the usefulness of the filtered information obtained from semgen by looking at genes extracted from text on parkinson's disease and enhanced with annotations using gfinder.
RESULTS
postprocessing
the postprocessing procedure for increased accuracy is based on the intuition that syntactic complexity correlates with reliability in nlp.
one aspect of complexity is argument proximity to indicator, and anecdotal evidence suggests that arguments close to their indicator  are more likely to be correctly identified by semgen.
for example, in figure  <dig> the 'easy' sentence has arguments contiguous to the indicator "inhibited," and semgen identifies them correctly.
by contrast, in the 'hard' sentence, semgen did not correctly identify the arguments of the predication indicated by "influence" .
although both "creb" and "cbp" are of the appropriate semantic class to serve as arguments of this gene-gene interaction predication, they do not act in that capacity in this sentence.
based on examples such as those in figure  <dig>  we hypothesized that postprocessing based on argument distance could enhance semgen accuracy.
we further assumed that syntactic type of indicator  would have a bearing on reliability, since in addition to proximity, structural aspects such as these are an aspect of complexity.
in order to test this hypothesis, we implemented a procedure that first kept track of indicator category and argument distance in semgen output, and then devised a parametric filtering procedure based on these phenomena.
argument distance was computed as the number of phrases intervening between an argument and its indicator.
we established a baseline by calculating the precision of  <dig>  unfiltered semgen relations extracted from text on diabetes, as described in the methods section.
table  <dig> provides results.
predications both with and without an official gene symbol and entrez gene id are given, as well as the type of relation  and indicator responsible .
as noted earlier, no gene-gene relations with a preposition indicator were extracted by semgen.
precision for gene-gene relations in particular is not adequate for subsequent unsupervised processing.
we evaluated the performance of the postprocessing procedure using the same  <dig>  relations used to generate semgen baseline figures.
figure  <dig> shows results  only for the relations in which all genes have been mapped to the official gene symbol and entrez gene id.
graph a contains figures for gene-gene relations with verb indicators, while graphs b and c have results for gene-disease relations, b for verb indicators and c for prepositions.
"all" refers to no filtering ; values at this point correspond to the relevant line from table  <dig> .
the use of this filtering procedure implies an inverse relationship between precision and amount of information retained : as precision increases, more information is lost.
in all cases, argument distance correlates with this trade-off.
for example, if gene-gene relations from verb indicators are filtered for arguments at distance  <dig> from the indicator, precision increases from  <dig> %   to  <dig> %  ; however, information retained  drops to  <dig> % .
results demonstrate that proximity to verb indicator has a positive effect on accuracy.
in addition, semantic class of predicate  appears to influence reliability.
with regard to type of indicator, precision is higher for gene-disease relations with verb indicator  than for the same relations with preposition indicator .
although we did not conduct a formal error analysis to explicate this difference, it is likely dependent on the fact that "in" as an indicator is ambiguous, and hence prone to generating semgen errors.
another observation is that for verb indicators, precision is higher for gene-disease relations  than for gene-gene relations .
again, we can only provide an impressionistic explanation.
this result is probably due to the tendency for disorder names to be easier to identify than gene names.
proximity in co-occurrence processing
to assess the generality of distance based filtering, we applied this postprocessing to gene-gene and gene-disease relations identified with co-occurrence processing.
after using semgen to identify relevant entities in the medline citations on diabetes noted above, we computed gene-gene and gene-disorder co-occurrence, and then computed the distance  between each co-occurrence pair.
the results are shown in figure  <dig>  "all" indicates no filtering, and the other points denote "bins" of cumulative intervening content words .
results show that precision varies with distance, reaching best values for moderate to -intermediate distances for gene-gene co-occurrences and low distances for gene-disorder co-occurrence pairs.
filtering at such distances improves results when compared to no filtering.
for gene-disease co-occurrences, precision improved by  <dig> % when considering only co-occurrences with no more than  <dig> intervening content word:  <dig> % vs.  <dig> % precision .
whereas, for gene-gene co-occurrences improvement when considering only co-occurrences with up to  <dig> intervening content words was  <dig> %:  <dig> % vs.  <dig> % precision .
when the number of intervening content words is low , precision is decreased for gene-gene co-occurrences.
the effect of proximity filtering on overall precision of co-occurrence processing is not dramatic.
in addition, the improved precision results have a wide margin that partially overlaps with the unfiltered results, suggesting that in some cases the improvement could be due to variation.
however, these results do not limit the benefits of such filtering for semantic interpretation.
as noted above, the effectiveness of proximity processing with semantic interpretation is ultimately determined by the structure of english sentences.
co-occurrence processing is applied without reference to that structure, and hence subsequent proximity processing either has minimal effect or depresses precision .
exploiting extracted relations
as noted above, we used text on the molecular genetics of diabetes to evaluate the accuracy of our postprocessing filter.
we also conducted a second test to demonstrate the usefulness of filtered predications for subsequent unsupervised processing.
for this, we focused on parkinson's disease, whose polygenetic etiology is not fully understood  <cit> .
eighty-four etiologic relations on parkinson's related disorders containing an official gene symbol and entrez gene id were extracted with semgen from medline citations on parkinson's disease.
these were then subjected to postprocessing limited to verb indicators and a maximum argument distance of  <dig>  the  <dig> distinct remaining relations contained  <dig> unique genes .
five of these  were not associated with parkinson's disease or related disorders in several important genetic disorder databases, including the online mendelian inheritance in man , genetics home reference   <cit> , genetic association database   <cit> , or the parkinson disease mutation database   <cit> .
to obtain more information on those five genes, we loaded their entrez gene ids into gfinder, which enabled us to find the biological process categories in the gene ontology that were assigned to the selected genes.
we did not take into account annotations inferred from electronic annotation and those with negative evidence.
the annotations used, shown in figure  <dig>  are mainly related to cellular metabolism and its regulation, cell communication, and transport.
in particular, they include "development" ; "cell-cell signaling" ; "negative regulation of translation" ; "g-protein coupled receptor protein signaling pathway" , which is a parent of "dopamine receptor signaling pathway"; and "monoamine transport" , a sibling of "dopamine transport."
note that because of inheritance in the gene ontology hierarchy, all ancestor categories are also assigned to these genes.
we also manually assessed the correctness of the  <dig> relations by comparing them to the original sentences.
we then calculated precision before and after filtering, and recall and f score after filtering.
figure  <dig> shows the results for verb and preposition indicators at several argument distance values.
they illustrate that by considering only relations with verb indicators  and maximum argument distance of  <dig>  we obtained both high precision  and recall  for the selected relations.
the breadth of the confidence intervals is due to the small number of semantic relations assessed.
discussion
nlp to support research in molecular biology
semgen is one of several systems currently being developed to provide access to information in text  for molecular biology research .
of the systems that identify relations, various approaches  are being pursued.
due to the complexity of the content involved, most systems focus on a particular molecular biology phenomenon.
several applications address protein-protein interactions: bunescu et al.
<cit> , for example, use machine learning techniques, while corney et al.
<cit>  employ syntactic patterns for such relations.
blaschke et al.
<cit>  and blaschke & valencia  <cit>  also use syntactic templates, enhanced with proximity processing between arguments, to identify protein interactions.
huang et al.
<cit>  also call on syntactic patterns, which they discover with automatic methods, and temkin and gilder  <cit>  use a context free grammar to extract protein interactions from text.
hu et al.
<cit>  concentrate on protein phosphorylation using a system similar to semgen.
regarding function and structure, gaizauskas et al.
<cit>  employ both syntactic parsing and semantic templates to identify information about protein structure in text.
koike et al.
<cit>  exploit syntactic patterns for gene function, and daraselia et al.
<cit>  take advantage of a full parse for protein function.
friedman et al.
<cit>  use a semantic grammar to identify molecular pathways, while santos et al.
<cit>  combine statistical methods with partial and full parsing and concentrate on the wnt pathway.
blaschke et al.
<cit>  mine gene expression information from medline citations using a method similar to  <cit> .
finally, leroy et al.
<cit>  exploit a shallow parser to identify various relations in molecular biology.
proximity between arguments is also used in their method.
the postprocessing technique we developed selects the extracted semantic relations that are most likely to be correct based on distance of the arguments from the syntactic predicate .
other methods  <cit>  have employed a related notion, namely distance between arguments participating in a relation, where the relations are identified with templates or shallow parsing.
previous work has not discussed incremental improvements dependent on degree of proximity, nor discussed the recall-precision trade-off, nor compared proximity filtering to unfiltered results.
although we based our work on semgen, our filtering process could be applied to relations produced by other nlp methods.
the semantic content of the relations is not relevant.
the postprocessing technique could be transferred most straightforwardly to those systems that retrieve arguments using rules or patterns, since a verb or preposition  is available to interact with argument distance as a predictor of extraction accuracy.
when an indicator is not used, as in statistical systems, the technique could be slightly modified to use distance between arguments as the sole predictor of correctness.
however, in this case it is not likely to be dramatically effective, as suggested by our experiment with co-occurrence processing.
argument distance thresholds while postprocessing extracted relations
effectively exploiting extracted genetic and etiologic relations for subsequent applications depends on maintaining a balance between the highest possible precision and a sufficient level of retained information  for useful applications.
for gene-gene relations derived from verbs, for example, this can be obtained by allowing an argument distance of no more than  <dig> phrases from the indicator  precision, and  <dig> %  recall).
however, when filtered relations are used for automatic processing, high precision should take precedence over high recall : for verb indicators, an argument distance of  <dig> for genetic relations and  <dig> or  <dig> for etiologic relations.
for supervised applications less strict threshold values can be used.
considering all extracted relations, whether they include an official gene symbol or not, would increase precision for any distance threshold .
such relations could be useful for subsequent supervised applications; however, we limited this study to official gene symbols  because this allows automatic linking to structured biological data.
subsequent automatic processing based on this information could then unveil hidden biological knowledge  <cit> .
exploiting extracted information
the application example discussed above illustrates that the procedure we propose for filtering the results of automatically extracted gene-gene and gene-disease relations effectively selects useful information.
many of the genes identified, including cyp2d <dig>  lrrk <dig>  mapt, nr4a <dig>  park <dig>  park <dig>  park <dig>  pink <dig>  uchl <dig> , are associated with parkinson's disorders in genetic disorder reference databases, such as omim, ghr, gad, and mutpd.
furthermore, the process identified five genes not associated with parkinson's disease in those databases .
by uploading these genes into gfinder, we were able to highlight their biological process categories in the gene ontology .
three of these genes  are currently associated only with high level or general biological process categories that might or might not be related to parkinson's disease.
however, two of these genes  are associated with low level biological process categories clearly related to parkinson' disease: "g-protein coupled receptor protein signaling pathway" and "monoamine transport", respectively.
alterations in the biological processes of these categories, which are parent and sibling categories of "dopamine receptor signaling pathway" and "dopamine transport" respectively, may well be involved in parkinson's disease and suggest interesting avenues for further analysis.
in fact, motor symptoms in parkinson's disease are generally thought to result from deficiency or dysfunction of dopamine or dopaminergic neurons in the substantia nigra  <cit> .
future work
although current filtering usefully supports subsequent automatic analysis, there is room for improvement.
as a further measure of sentence complexity, an extended multi-parametric filtering could be implemented, which takes into account the total number of arguments on both the left and the right of an indicator.
it would also be possible to improve results by exploiting domain knowledge about genes and diseases to support statistical methods for constructing resources expressing functional characteristics such as involvement in biological processes, biochemical pathways, molecular functions, and co-occurring expression in similar tissues.
this information could then be consulted to exclude improbable semantic relations.
CONCLUSIONS
the genetic and etiologic relations extracted by semgen from the research literature are normalized semantic descriptions of complex genetic interactions.
the filtering method we implemented increases the precision of error-prone nlp by selecting the semantic relations most likely to be correct.
this information can then be used for further applications aimed at uncovering new biomedical knowledge.

 semgen in an effort to minimize errors due to natural language processing, we developed and evaluated a procedure for postprocessing extracted genetic information. this processing is applied to the output of semgen, an nlp system for extracting semantic predications from the text of medline citations. semgen applies in the domain of molecular genetics and has several components: journal descriptor indexing, the medpost tagger, the specialist lexicon, the umls metathesaurus, metamap, and abgene. these components interact to first identify genetic phenomena and disorders and subsequently construct semantic relations among these entities: gene-gene interactions as well as gene-disease associations. processing consists of three phases: construction of an underspecified syntactic structure identification of relevant semantic concepts, and final interpretation of a semantic predication. the system first calls jdi, a statistics-based labeled categorizer used to limit input text to the molecular genetics domain before proceeding with natural language processing. after jdi has applied, text is sent both to the medpost tagger and to abgene, which assists in identifying gene names. as the first step in creating an underspecified syntactic structure, the medpost tagger, drawing on the specialist lexicon, labels the words in input with part-of-speech categories, as shown in. we now show that apoa-ii promotes insulin resistance and has diverse effects on fat homeostasis pron adv verb compl noun verb noun noun conj aux adj noun prep noun noun the tagged list in serves as input to an underspecified parser which identifies phrases in an input string. these correspond to low level nodes in a syntactic parse tree, and except for noun phrases, are left unlabeled. in addition, words are assigned their role inside the noun phrase, as either modifier or head. informally, a head is the most important word in a noun phrase. a schematic example is given in, where phrases are delimited by square brackets. np np np np] the syntactic structure in serves as the basis for the next phase, identification of noun phrases expressing relevant semantic concepts: genetic phenomena and disorders. genetic phenomena are defined broadly as those concepts that may have a bearing on molecular genetics and include genes, proteins, nucleotide sequences, mutations, polymorphisms, and chromosomes. in this study we concentrated exclusively on genes. for each gene name identified, semgen attempts to provide the corresponding official symbol and entrez gene id, although this is not always possible. gene symbol resolution is a challenging nlp task, currently under active investigation. for disorders, semgen considers a concept having one of the following umls semantic types to be relevant:'pathologic function','disease or syndrome','neoplastic process', and'congenital abnormality'. during processing to identify relevant semantic concepts, semgen examines each noun phrase in structures such as to determine whether it qualifies as a genetic phenomenon or as a disorder. for disorder names, metamap is used exclusively to determine whether the phrase maps to a concept in the umls metathesaurus having a relevant semantic type. in considering genetic phenomena, semgen first calls on metamap and the umls metathesaurus; however, since the metathesaurus is not complete for gene names, the output from abgene is also consulted. abgene identifies gene and protein names in text using both statistical and empirical methods. strategies include low frequency trigrams and rules generated both automatically and by hand. in addition, the program relies on linguistic information, such as cue words, suffixes, and part-of-speech information. semgen directs abgene to identify all gene names in the medline citation that contains the sentence currently being processed. these are then available as a resource to semgen. for example, abgene identifies the gene names in from a citation with title structural variants in the retinoid receptor genes in patients with schizophrenia and other psychiatric diseases. "nurr gene", "rars", "rar/rxr", "retinoid receptors", "retinoid receptor genes", "rxrgamma", "rxr genes" in processing input from this citation that contains the phrase nurr gene, semgen determines from metamap that nurr does not occur in the metathesaurus. the abgene list for the citation containing the current input sentence is then consulted, and nurr is successfully identified as a genetic phenomenon in the sentence containing this phrase. the structure in is an example of the syntactic parse enhanced with identification of a gene and a disorder. the phrase containing "apoa-ii" has now been expanded to include the entrez gene id, official symbol, gene name isolated by semgen as well as the original text. the noun phrase containing "insulin resistance" has been expanded to include the umls concept, as well as the original text. np np np np] in identifying relations, the final phase of processing, semgen relies on concepts identified in the previous phase, some of which serve as arguments of predications. indicators constitute a crucial aspect of this phase. these are syntactic elements that map to the predicate of a semantic predication. verbs commonly serve this function. for example, in, the verb "promote" indicates the semantic predicate predispose. this forms the basis for construction of a predication with arguments "jnk" and "diabetes." there appear to be multiple mechanisms through which jnk might promote diabetes. other examples of verbs that indicate semantic relations between a gene and a disorder include "predispose" and "influence" and "implicate". similarly, verbs indicating a relation between two genes are "stimulate" and "upregulate", "block" and "inhibit", and "mediate". in addition to verbs, prepositions may serve as indicators of semantic predicates. for example, in, the preposition "in" indicates the predicate associated_with. a clinic-based study of the lrrk gene in parkinson disease yields new mutations. the preposition "for" can also indicate associated_with, as in. saitohin represents a candidate gene for parkinson's disease. it should be noted that in both and the preposition is the only syntactic cue that there is a semantic relation between the gene and the disorder. in the molecular genetics domain, we have encountered prepositions as indicators only for the semantic predicate associated_with. semgen has rules that encode indicators for the semantic predicates relevant to this domain. in constructing a predication these rules identify predicates and are then supported by argument identification rules. such rules apply several constraints: no argument can be used in more than one predication unless it is coordinate or the head of a relative clause. for example, in, since parp- and nuclear factor kappa b are coordinate each is allowed to participate in a separate associated_with predication with inflammatory disorders. in, tag is the head of relative clause and is thus allowed to serve as the subject of an inhibit relation with p and also as the subject of an associated_with relation with insulinoma. parp- and nuclear factor kappa b have both been suggested to play a crucial role in inflammatory disorders. however, tag, which inactivates the key tumour suppressor p is not known to be involved in the pathogenesis of human insulinoma. subjects appear to the left of an indicator, and objects to the right for active constructions; for passives the order is reversed. human obesity syndrome in, for example, will be interpreted as the object of a causes relation, because it occurs to the left of the passive indicator caused; mkks, to the right, will be the subject. we and others have demonstrated that human obesity syndrome is caused by mutations in the gene mkks. finally, arguments are constrained by semantic class: both arguments of a gene-gene interaction predicate must be genetic phenomena. the subject of a gene-disorder predicate must be a genetic phenomenon, while its object is a disorder. an example of semgen output is given in figure which provides the final interpretation for the example from above. gfinder in order to exploit extracted semantic predications, we used the gfinder application. this is a web application that enriches lists of gene ids with controlled functional annotations dynamically retrieved from several biomolecular databanks, including entrez gene, gene ontology, kegg, swiss-prot, pfam, and omim. moreover, gfinder allows computational and statistical analyses on the functional and phenotypic annotations of user-selected groups of genes, aimed at highlighting those annotation categories that are significant in the genes selected. overview after establishing a baseline for semgen processing on the basis of a corpus of medline citations on diabetes, we evaluated the results of postprocessing this semgen output with a distance filtering procedure we developed. we also compared these results to those obtained from applying our filtering method to genetic and etiologic relations obtained with co-occurrence processing applied to the same corpus on diabetes. finally, we tested the usefulness of the filtered information obtained from semgen by looking at genes extracted from text on parkinson's disease and enhanced with annotations using gfinder.  postprocessing the postprocessing procedure for increased accuracy is based on the intuition that syntactic complexity correlates with reliability in nlp. one aspect of complexity is argument proximity to indicator, and anecdotal evidence suggests that arguments close to their indicator are more likely to be correctly identified by semgen. for example, in figure the'easy' sentence has arguments contiguous to the indicator "inhibited," and semgen identifies them correctly. by contrast, in the'hard' sentence, semgen did not correctly identify the arguments of the predication indicated by "influence". although both "creb" and "cbp" are of the appropriate semantic class to serve as arguments of this gene-gene interaction predication, they do not act in that capacity in this sentence. based on examples such as those in figure we hypothesized that postprocessing based on argument distance could enhance semgen accuracy. we further assumed that syntactic type of indicator would have a bearing on reliability, since in addition to proximity, structural aspects such as these are an aspect of complexity. in order to test this hypothesis, we implemented a procedure that first kept track of indicator category and argument distance in semgen output, and then devised a parametric filtering procedure based on these phenomena. argument distance was computed as the number of phrases intervening between an argument and its indicator. we established a baseline by calculating the precision of unfiltered semgen relations extracted from text on diabetes, as described in the methods section. table provides results. predications both with and without an official gene symbol and entrez gene id are given, as well as the type of relation and indicator responsible. as noted earlier, no gene-gene relations with a preposition indicator were extracted by semgen. precision for gene-gene relations in particular is not adequate for subsequent unsupervised processing. we evaluated the performance of the postprocessing procedure using the same relations used to generate semgen baseline figures. figure shows results only for the relations in which all genes have been mapped to the official gene symbol and entrez gene id. graph a contains figures for gene-gene relations with verb indicators, while graphs b and c have results for gene-disease relations, b for verb indicators and c for prepositions. "all" refers to no filtering; values at this point correspond to the relevant line from table. the use of this filtering procedure implies an inverse relationship between precision and amount of information retained: as precision increases, more information is lost. in all cases, argument distance correlates with this trade-off. for example, if gene-gene relations from verb indicators are filtered for arguments at distance from the indicator, precision increases from % to %; however, information retained drops to %. results demonstrate that proximity to verb indicator has a positive effect on accuracy. in addition, semantic class of predicate appears to influence reliability. with regard to type of indicator, precision is higher for gene-disease relations with verb indicator than for the same relations with preposition indicator. although we did not conduct a formal error analysis to explicate this difference, it is likely dependent on the fact that "in" as an indicator is ambiguous, and hence prone to generating semgen errors. another observation is that for verb indicators, precision is higher for gene-disease relations than for gene-gene relations. again, we can only provide an impressionistic explanation. this result is probably due to the tendency for disorder names to be easier to identify than gene names. proximity in co-occurrence processing to assess the generality of distance based filtering, we applied this postprocessing to gene-gene and gene-disease relations identified with co-occurrence processing. after using semgen to identify relevant entities in the medline citations on diabetes noted above, we computed gene-gene and gene-disorder co-occurrence, and then computed the distance between each co-occurrence pair. the results are shown in figure "all" indicates no filtering, and the other points denote "bins" of cumulative intervening content words. results show that precision varies with distance, reaching best values for moderate to -intermediate distances for gene-gene co-occurrences and low distances for gene-disorder co-occurrence pairs. filtering at such distances improves results when compared to no filtering. for gene-disease co-occurrences, precision improved by % when considering only co-occurrences with no more than intervening content word: % vs. % precision. whereas, for gene-gene co-occurrences improvement when considering only co-occurrences with up to intervening content words was %: % vs. % precision. when the number of intervening content words is low, precision is decreased for gene-gene co-occurrences. the effect of proximity filtering on overall precision of co-occurrence processing is not dramatic. in addition, the improved precision results have a wide margin that partially overlaps with the unfiltered results, suggesting that in some cases the improvement could be due to variation. however, these results do not limit the benefits of such filtering for semantic interpretation. as noted above, the effectiveness of proximity processing with semantic interpretation is ultimately determined by the structure of english sentences. co-occurrence processing is applied without reference to that structure, and hence subsequent proximity processing either has minimal effect or depresses precision. exploiting extracted relations as noted above, we used text on the molecular genetics of diabetes to evaluate the accuracy of our postprocessing filter. we also conducted a second test to demonstrate the usefulness of filtered predications for subsequent unsupervised processing. for this, we focused on parkinson's disease, whose polygenetic etiology is not fully understood. eighty-four etiologic relations on parkinson's related disorders containing an official gene symbol and entrez gene id were extracted with semgen from medline citations on parkinson's disease. these were then subjected to postprocessing limited to verb indicators and a maximum argument distance of the distinct remaining relations contained unique genes. five of these were not associated with parkinson's disease or related disorders in several important genetic disorder databases, including the online mendelian inheritance in man, genetics home reference, genetic association database, or the parkinson disease mutation database. to obtain more information on those five genes, we loaded their entrez gene ids into gfinder, which enabled us to find the biological process categories in the gene ontology that were assigned to the selected genes. we did not take into account annotations inferred from electronic annotation and those with negative evidence. the annotations used, shown in figure are mainly related to cellular metabolism and its regulation, cell communication, and transport. in particular, they include "development"; "cell-cell signaling"; "negative regulation of translation"; "g-protein coupled receptor protein signaling pathway", which is a parent of "dopamine receptor signaling pathway"; and "monoamine transport", a sibling of "dopamine transport." note that because of inheritance in the gene ontology hierarchy, all ancestor categories are also assigned to these genes. we also manually assessed the correctness of the relations by comparing them to the original sentences. we then calculated precision before and after filtering, and recall and f score after filtering. figure shows the results for verb and preposition indicators at several argument distance values. they illustrate that by considering only relations with verb indicators and maximum argument distance of we obtained both high precision and recall for the selected relations. the breadth of the confidence intervals is due to the small number of semantic relations assessed. discussion nlp to support research in molecular biology semgen is one of several systems currently being developed to provide access to information in text for molecular biology research. of the systems that identify relations, various approaches are being pursued. due to the complexity of the content involved, most systems focus on a particular molecular biology phenomenon. several applications address protein-protein interactions: bunescu et al., for example, use machine learning techniques, while corney et al. employ syntactic patterns for such relations. blaschke et al. and blaschke & valencia also use syntactic templates, enhanced with proximity processing between arguments, to identify protein interactions. huang et al. also call on syntactic patterns, which they discover with automatic methods, and temkin and gilder use a context free grammar to extract protein interactions from text. hu et al. concentrate on protein phosphorylation using a system similar to semgen. regarding function and structure, gaizauskas et al. employ both syntactic parsing and semantic templates to identify information about protein structure in text. koike et al. exploit syntactic patterns for gene function, and daraselia et al. take advantage of a full parse for protein function. friedman et al. use a semantic grammar to identify molecular pathways, while santos et al. combine statistical methods with partial and full parsing and concentrate on the wnt pathway. blaschke et al. mine gene expression information from medline citations using a method similar to. finally, leroy et al. exploit a shallow parser to identify various relations in molecular biology. proximity between arguments is also used in their method. the postprocessing technique we developed selects the extracted semantic relations that are most likely to be correct based on distance of the arguments from the syntactic predicate. other methods have employed a related notion, namely distance between arguments participating in a relation, where the relations are identified with templates or shallow parsing. previous work has not discussed incremental improvements dependent on degree of proximity, nor discussed the recall-precision trade-off, nor compared proximity filtering to unfiltered results. although we based our work on semgen, our filtering process could be applied to relations produced by other nlp methods. the semantic content of the relations is not relevant. the postprocessing technique could be transferred most straightforwardly to those systems that retrieve arguments using rules or patterns, since a verb or preposition is available to interact with argument distance as a predictor of extraction accuracy. when an indicator is not used, as in statistical systems, the technique could be slightly modified to use distance between arguments as the sole predictor of correctness. however, in this case it is not likely to be dramatically effective, as suggested by our experiment with co-occurrence processing. argument distance thresholds while postprocessing extracted relations effectively exploiting extracted genetic and etiologic relations for subsequent applications depends on maintaining a balance between the highest possible precision and a sufficient level of retained information for useful applications. for gene-gene relations derived from verbs, for example, this can be obtained by allowing an argument distance of no more than phrases from the indicator precision, and % recall). however, when filtered relations are used for automatic processing, high precision should take precedence over high recall: for verb indicators, an argument distance of for genetic relations and or for etiologic relations. for supervised applications less strict threshold values can be used. considering all extracted relations, whether they include an official gene symbol or not, would increase precision for any distance threshold. such relations could be useful for subsequent supervised applications; however, we limited this study to official gene symbols because this allows automatic linking to structured biological data. subsequent automatic processing based on this information could then unveil hidden biological knowledge. exploiting extracted information the application example discussed above illustrates that the procedure we propose for filtering the results of automatically extracted gene-gene and gene-disease relations effectively selects useful information. many of the genes identified, including cyp2d lrrk mapt, nr4a park park park pink uchl, are associated with parkinson's disorders in genetic disorder reference databases, such as omim, ghr, gad, and mutpd. furthermore, the process identified five genes not associated with parkinson's disease in those databases. by uploading these genes into gfinder, we were able to highlight their biological process categories in the gene ontology. three of these genes are currently associated only with high level or general biological process categories that might or might not be related to parkinson's disease. however, two of these genes are associated with low level biological process categories clearly related to parkinson' disease: "g-protein coupled receptor protein signaling pathway" and "monoamine transport", respectively. alterations in the biological processes of these categories, which are parent and sibling categories of "dopamine receptor signaling pathway" and "dopamine transport" respectively, may well be involved in parkinson's disease and suggest interesting avenues for further analysis. in fact, motor symptoms in parkinson's disease are generally thought to result from deficiency or dysfunction of dopamine or dopaminergic neurons in the substantia nigra. future work although current filtering usefully supports subsequent automatic analysis, there is room for improvement. as a further measure of sentence complexity, an extended multi-parametric filtering could be implemented, which takes into account the total number of arguments on both the left and the right of an indicator. it would also be possible to improve results by exploiting domain knowledge about genes and diseases to support statistical methods for constructing resources expressing functional characteristics such as involvement in biological processes, biochemical pathways, molecular functions, and co-occurring expression in similar tissues. this information could then be consulted to exclude improbable semantic relations.  the genetic and etiologic relations extracted by semgen from the research literature are normalized semantic descriptions of complex genetic interactions. the filtering method we implemented increases the precision of error-prone nlp by selecting the semantic relations most likely to be correct. this information can then be used for further applications aimed at uncovering new biomedical knowledge. 
natural language processing is increasingly used for this purpose although it inherently involves errors. since this proximity filtering is based on syntactic structure, applying it to the results of co-occurrence processing is useful, but not as effective as when applied to the output of natural language processing. information based on the proximity postprocessing method we suggest is of sufficient quality to be profitably used for subsequent applications aimed at uncovering new biomedical knowledge. we also applied this strategy to relations identified with co-occurrence processing. although proximity filtering is only marginally effective for enhancing the precision of relations extracted with co-occurrence processing, it is likely to benefit methods based, even partially, on syntactic structure, regardless of the relation. for example, if gene-gene relations are filtered for arguments at a distance of phrase from the predicate, precision increases from % to %. based on the number of intervening phrases between an argument and its predicate, we defined a heuristic strategy to filter the extracted semantic relations according to their likelihood of being correct. in an effort to exploit semgen predications on the etiology of disease after increasing precision with postprocessing, a gene list was derived from extracted information enhanced with postprocessing filtering and was automatically annotated with gfinder, a web application that dynamically retrieves functional and phenotypic information from structured biomolecular resources.
 genomic functional information is valuable for biomedical research. however, such information frequently needs to be extracted from the scientific literature and structured in order to be exploited by automatic systems. natural language processing is increasingly used for this purpose although it inherently involves errors. a postprocessing strategy that selects relations most likely to be correct is proposed and evaluated on the output of semgen, a system that extracts semantic predications on the etiology of genetic diseases. based on the number of intervening phrases between an argument and its predicate, we defined a heuristic strategy to filter the extracted semantic relations according to their likelihood of being correct. we also applied this strategy to relations identified with co-occurrence processing. finally, we exploited postprocessed semgen predications to investigate the genetic basis of parkinson's disease.  the filtering procedure for increased precision is based on the intuition that arguments which occur close to their predicate are easier to identify than those at a distance. for example, if gene-gene relations are filtered for arguments at a distance of phrase from the predicate, precision increases from % to %. since this proximity filtering is based on syntactic structure, applying it to the results of co-occurrence processing is useful, but not as effective as when applied to the output of natural language processing. in an effort to exploit semgen predications on the etiology of disease after increasing precision with postprocessing, a gene list was derived from extracted information enhanced with postprocessing filtering and was automatically annotated with gfinder, a web application that dynamically retrieves functional and phenotypic information from structured biomolecular resources. two of the genes in this list are likely relevant to parkinson's disease but are not associated with this disease in several important databases on genetic disorders.  information based on the proximity postprocessing method we suggest is of sufficient quality to be profitably used for subsequent applications aimed at uncovering new biomedical knowledge. although proximity filtering is only marginally effective for enhancing the precision of relations extracted with co-occurrence processing, it is likely to benefit methods based, even partially, on syntactic structure, regardless of the relation. 