BACKGROUND
the amount of biomedical literature available online continues to grow rapidly today, creating a need for automatic processing using bioinformatics tools.
many information extraction  systems incorporating natural language processing  techniques have been developed for use in the biomedical field.
a key ie task in this field is the extraction of relations between named entities , such as protein-protein and gene-disease interactions.
semantic role labeling , also called shallow semantic parsing  <cit> , is a popular semantic analysis technique for extracting relations.
in srl, sentences are represented by one or more predicate-argument structures , also known as propositions  <cit> .
each pas is composed of a predicate  and several arguments  that have different semantic roles, including main arguments such as an agent that deliberately performs an action  and a patient that experiences an action , as well as adjunct arguments, such as time, manner, and location.
here, the term argument refers to a syntactic constituent of the sentence related to the predicate; and the term semantic role refers to the semantic relationship between a predicate  and an argument  of a sentence.
for example, in figure  <dig>  the sentence "il <dig> and il <dig> receptors activate stat <dig>  stat <dig>  and stat <dig> proteins in the human b cells" describes a molecular activation process.
it can be represented by a pas in which "activate" is the predicate, "il <dig> and il <dig> receptors" comprises the agent, "stat <dig>  stat <dig>  and stat <dig> proteins" comprises the patient, and "in the human b cells" is the location.
thus, the agent, patient, and location are the arguments of the predicate.
an important preliminary task in srl is to define the set of possible semantic roles for each verb sense, referred to as a roleset.
a roleset can be paired with a set of syntactic frames that shows all the acceptable syntactic expressions of those roles.
this is called a frameset  <cit> .
in  <dig>  the proposition bank project   <cit>  published a guide, propbank i  <cit> , which defined a format for pas annotation.
alongside propbank i, the project also released a corpus of pas's for  <dig>  verbs in the newswire domain to facilitate ml-based srl system development  <cit> .
the semantic arguments of individual verbs in the propbank i annotation are numbered from  <dig>  for a specific verb, arg <dig> is usually the argument corresponding to the agent  <cit> , while arg <dig> usually corresponds to the patient.
however, higher-numbered arguments, which occupy about 10% of the total arguments, have no consistent role definitions.
in addition to numbered arguments, there are also argms, which refer to annotation of modifiers.
the semi-regular and flexible assignment of numbered arguments to semantic roles found in propbank i facilitates formulation of the srl task as a classification problem with machine-learning  based systems.
that is, given a phrase, the sentence containing it, and the predicate, a system must classify the phrase's semantic role corresponding to the predicate.
for specific applications, however, the flexible argument assignment of propbank i annotation may be a disadvantage.
in some cases, developers may wish to limit the semantic roles of each argument.
take the frameset of "delete" for example.
table  <dig> shows the frameset definition.
as you can see in table  <dig>  the agent is defined as "entity removing", and the patient is defined as "thing being removed" in propbank i. however, in certain biomedical events, a developer might want to limit the agent to being a certain causal mechanism such as a mutation or alternative splicing and the patient to being an "exon, gene, chromosomal region,  cell".
an alternative to propbank, the pasbio  <cit>  project provides more detailed and restrictive framesets for  <dig> biomedical verbs.
the well-known biomedical text mining researchers cohen and hunter  <cit>  have found the pasbio annotation viable for representing the pas's of biomedical verbs.
several applications have been developed based on pasbio or following its spirit.
for example, shah et al.
<cit>  used the frameset definitions of pasbio to construct semantic patterns which can extract information about tissue-specific gene expression from biomedical literature.
later, shah and bork applied this approach to construct the lsat  database system  <cit> .
kogan et al.
<cit>  followed the pasbio annotation to built a domain-specific set of pass for the medical domain, which successfully extended pasbio to clinical texts.
all these systems mainly use handcrafted rules to identify and classify arguments into semantic roles.
unfortunately, due to the lack of an annotated corpus and inconsistent definitions between specific numbered arguments, no publicly available ml-based srl systems based on the pasbio standard have been developed.
to be able to apply ml to the biomedical srl problem, we constructed a biomedical domain specific proposition bank based on the more consistent propbank i annotation format.
the project, bioprop  <cit> , defined roles for  <dig> common biomedical verbs and provided an annotated corpus on which we developed an ml-based srl system, biosmile  <cit> .
this work was expanded upon with the release of our web-based search application, biosmile web search  <cit> , in february  <dig>
in this paper, we aim to build a bridge between bioprop and pasbio to facilitate pasbio-based srl system development.
using our system, one will first be able to roughly classify arguments' semantic roles according to bioprop, and then translate the pas's into pasbio annotation using a rule-based converter.
methods
the approaches applied in this work include:  named entity tagging,  semantic role labeling following bioprop's annotation format, and  rule-based conversion from bioprop to pasbio annotation.
named entity tagging
according to our observations, some bioprop arguments are equivalent to other pasbio arguments only under certain conditions, usually defined as the presence of a certain named entity  in a certain argument.
for example, arg <dig> of the verb "express" must be a gene or gene product in pasbio.
therefore, it is necessary to first tag all nes in the sentences.
to do this, we employ our previously developed ne recognition software, nerbio  <cit> , to tag five ne types: protein, dna, rna, cell line, and cell type.
we use a dictionary to find other ne types, such as extron and intron.
semantic role labeling
before conversion to the pasbio annotation format, a fundamental step is to identify the pas's of each sentence and annotate them using the bioprop format.
here, we briefly introduce how we constructed the bioprop-based srl system, biosmile, used for this task.
the first step was to construct a training corpus.
in our previous work, chou et al.
<cit> , we annotated pas's in genia's corpus of full parse trees, the genia treebank   <cit> , using propbank i framesets.
we then defined and added framesets for biomedical verbs to fit specific usages in biomedical literature.
however, all the new and modified framesets still conform strictly to the propbank annotation format.
a total of  <dig>  pas's were annotated for  <dig> biomedical verbs.
the second step we took was to formulate the srl problem as an ml-based sentence tagging problem.
the basic units of a sentence can be words, phrases, and constituents .
punyakanok et al.
<cit>  has shown that constituent-by-constituent  tagging is the best formulation for the srl problem; therefore, we adopted this formulation.
finally, we constructed a biomedical full parser based on the charniak parser  <cit>  with gtb as its training data which could automatically generate parse trees for sentences.
its performance is reported in additional file  <dig>
using bioprop as the training corpus, c-by-c formulation, and the parse trees generated by our biomedical full parser, we then constructed our srl system, biosmile, following the maximum entropy ml model  <cit> .
details of the features used in our srl system can be found in  <cit> .
development of conversion rules
there are two main differences between bioprop and pasbio pas framesets annotations:  pasbio developers usually define framesets to represent specific biological events.
therefore, for each argument, it is necessary to include information in addition to its semantic role, such as whether the argument should be a specific ne or contain specific keywords.
the order of arguments for a given verb sense in a bioprop frameset may not match that in a corresponding pasbio frameset.
to deal with these two differences, we build conversion rules verb by verb using our semi-automatic rule-generation tool which describe under which conditions each mapping is valid.
the algorithm used by the rule-generator compares corresponding framesets for a given verb sense, checks each argument in its pasbio frameset, and tries to find an argument in its bioprop frameset that has the same semantic role under a set of conditions.
when a match is found, the algorithm maps a link between the two frameset arguments, which includes a description of required conditions .
each conversion rule consists of two elements: predicates and transformations.
the predicate is the target verb.
the first part of each transformation is the condition, which specifies the criteria that the arguments should follow.
these criteria are defined as the composition of one or more logical predicates, which are concatenated by logical operators, such as and, and or.
two most common predicates are containsne and containskeywords.
the former is true if the argument contains at least one instance of the ne type ne.
the latter is true if the argument contains at least one specified keyword kw.
if there are no conditions for a transformation, this part can be omitted.
the second part is the mapping between a bioprop argument and a pasbio argument.
the mapping consists of three elements: the source argument, an arrow "→", and the destination argument.
for example, the transformation in figure  <dig> defines a mapping from argm-loc to arg <dig>  all the arguments that are not defined in the transformation source field are dropped.
as shown in figure  <dig>  the condition of the transformation "arg <dig> → arg1" is containsne, which is interpreted as the mapping arg <dig> → arg <dig> holds if arg <dig> contains at least one protein.
for a case in which arguments match, such as that in figure  <dig>  the conversion rules can be automatically generated as follows:
<dig>  for each argument pair, , if the argument phrase does not contain any recognized nes, a simple rule will be generated in the argument's "rule candidates" field: argumentb → argumentp
<dig>  if the argument contains recognized ne types , they will become the conditions imposed on the argument, and the following rule type will be generated: containsne ?argumentb → argumentp
users can modify the generated rules by editing the "rule candidates" field.
in addition to defining simple conditions, such as containsne, we also describe complex conditions using a format called the bracket form pattern, which can represent syntactic and semantic information as criteria.
the pattern can be applied when two or more pasbio arguments are covered by only one bioprop argument , and vice versa.
a bracket form  <cit>  is a representation of a parse tree using brackets , to show the tree's structure.
a simplified bracket form for the parse tree shown in figure  <dig>  with some internal bracket divisions omitted for clarity: )  )).
each constituent and its daughters are enclosed with brackets.
if we replace constituent words in the phrase with a wildcard symbol "", the above bracket form becomes:
)  ))
we can then use the bracket form as a pattern to match parse trees with the same structures.
to make these patterns more precise, we can add restrictions on the phrase constituents, such as limiting their semantic roles, head words and head words' upenn pos  <cit> .
to restrict a constituent's semantic role, one would insert a hyphen followed by the semantic role after the constituent type.
for example,  might become .
the head word can be defined as the most important word in a constituent  <cit> , and we identify it using collins'  <cit>  rule-based method.
head words of constituents are marked with an ampersand followed by the head word – e.g.
.
and the upenn pos of the head word is placed directly after, separated by a forward slash – e.g.
.
if we combine our above examples, we can make the pattern, ") ))", where the outside np must be arg <dig>  and the inside nps' head word must be "mrna" and "kilobase" with pos's "nns."
in our notation, a rule will appear as follows:
bracketformpattern ?
c <dig> → argument <dig>  c <dig> → argument <dig> ..., ci → argumenti,..., ck → argumentk;
"bracketformpattern" is a logical predicate which means the source argument, arguments, must match the bracket form pattern x for the transformations "ci → argumenti" to occur, where ci is any constituent of a source argument annotated by pasbio.
in the example in figure  <dig> for the verb "express", "arg1" in the bioprop column does not directly match any one pasbio argument, but instead overlaps two arguments, arg <dig> and arg <dig>  the rule-generation algorithm first generates two bracket forms for the unmatched noun phrase "two equally abundant mrnas for il8ra  <dig>  and  <dig>  kilobases in length", one for the "bioprop" column and the other for the "pasbio" column:
)  )")
)  )")
then, the first bracket form is merged with the second one as follows:
)  )")
as you can see in the merged bracket form, all the pasbio constituents annotated with semantic roles are represented by the variable ci.
for example arg <dig> becomes c <dig>
finally, the following three rules are automatically generated in the "rule candidates" field:
<dig>  bracketformpattern)  )") ?
c <dig> → arg <dig>  c <dig> → arg2
<dig>  bracketformpattern)  )") ?
c <dig> → arg <dig>  c <dig> → arg2
<dig>  bracketformpattern)  )") ?
c <dig> → arg <dig>  c <dig> → arg2
the first rule is the loosest, only considering the parse tree structure and srl tags.
the second also considers the head word, and the third adds pos information as well.
the user can check these rule candidates, and remove or modify the inappropriate ones.
although these rules are semi-automatically generated, we have found from our observations that with slight human modification, they can be quite accurate.
for the example in figure  <dig>  it is obvious that the first rule with no constraints on c <dig> and c <dig> is too loose.
likewise, the third rule, which limits c <dig> and c1's pos to nns, is too strict.
however, the second rule is surprisingly accurate.
if we look at the frameset definitions in bioprop and pasbio shown in table  <dig>  we can see that pasbio defines arg <dig> as a property of arg <dig> and limits arg <dig> to a gene or gene product name.
therefore, if we wish to annotate c <dig> as arg <dig> and c <dig> as arg <dig>  they must match these two conditions.
rule two stipulates that c1's head word should be "kilobase" and c0's should be "mrna", which matches pasbio's frameset definition for "express" because "kilobase" is a unit of mrna.
therefore, the annotator could choose the second rule with head word information.
RESULTS
datasets
the training data of our srl system, biosmile, is an extended version of bioprop  <cit> .
a total of  <dig>  pas's were annotated for  <dig> biomedical verbs.
to evaluate biosmile, the rule-based converter and the combined system, our in-lab biologists re-annotated the  <dig> annotated sentences available on pasbio's website according to the bioprop annotation format.
the dataset from pasbio's website is hereafter referred to as pasbiop and the pasbiop dataset annotated using the bioprop format is referred to as pasbiob.
evaluation metrics
performance was evaluated in terms of three metrics: precision , recall  and f-scores , which are defined as follows:
precision=the number of correctly recognized argumentsthe number of recognized argumentsrecall=the number of correctly recognized argumentsthe number of true argumentsf−scores=2×precision×recallprecision+recall
for srl and conversion evaluation, the official conll- <dig>  <cit>  srl evaluation script was used.
biosmile performance
we followed the same experimental procedure that we used in  <cit>  to evaluate biosmile performance on the extended bioprop dataset, details about which can be found in additional file  <dig>  the average results were an f-score of  <dig> %, a precision of  <dig> % and a recall of  <dig> %.
to evaluate the actual performance on arbitrary sentences and verbs, we used pasbiob as an extra test data.
biosmile achieved an overall f-score of  <dig> %, a precision of  <dig> % and a recall of  <dig> %.
the drop in biosmile's performance on pasbiob may be caused by the following factor: even though bioprop contains all pasbio verbs, it contains very few pas's for some verbs, which likely decreases the accuracy of ml-based srl on those verbs.
for example, there is only one pas for "splice" and two for "begin".
main system performance
we conducted two experiments – the first to test the bioprop-pasbio converter independently of biosmile srl performance, and the second to evaluate combined system performance.
for both, 3-fold cross validation  was applied, which involved partitioning the pasbiop dataset into three subsets.
a single subset is retained as the test data, and the remaining two subsets are used as training data for generating conversion rules.
the cv process is then repeated three times, with each of the test sets being used exactly once.
experiment 1: evaluating the rule-based converter
in this experiment, we examined conversion performance using the pasbiop dataset, first feeding the pasbiob  to the rule-based converter and then comparing the converted results with the pasbiop annotation to examine the precision, recall and f-scores.
by using the pasbiob, we can effectively eliminate the influence of biosmile srl performance from this test.
as shown in table  <dig>  we achieved an average f-score of  <dig> %.
the high f-score demonstrates the feasibility of our proposed semi-automatic conversion method.
experiment 2: evaluating the combined system
in this experiment, we examined the combined performance of our system, as shown in table  <dig>  compared with experiment  <dig>  the recall of the combined system drops 23%; however, the precision only drops 6%.
this may be due to the fact that biosmile has a high precision on pasbiob  but a low recall .
in addition, comparing the results in table  <dig> to the biosmile performance on pasbiob, we can see that the combined system's performance is higher.
this might seem counterintuitive; however, if we take into account that some argument types with low accuracy, such as argm-tmp and argm-dir, are not converted to pasbio since pasbio does not define those arguments, then we can explain this discrepancy.
discussion
after examining the pas's which were not labeled correctly in the experiments, we have concluded that the following two factors affected conversion performance most strongly:
absence of key terms for argument disambiguation
in cases where one bioprop argument can be divided into two or more pasbio arguments, our rules may be insufficient to disambiguate if nes or keywords are absent.
consider the following example annotated by our system with bioprop/pasbio annotations both given concatenated by a forward slash:
...   .
the last argument is incorrectly converted from bioprop arg <dig> to pasbio arg <dig> by our system.
to find out why, we must look at bioprop and pasbio's frameset definitions for "inhibit" shown in table  <dig>
we can see that pasbio defines both arg <dig> and arg <dig> as the objects being inhibited, but arg <dig> is further constrained to being the entity bound by the agent.
bioprop, which has no arg <dig> definition, does not make this distinction.
the automatically generated conversion rule for arg <dig>  therefore, will have the constraint containskeywords.
however, as the above example lacks any references to binding that would describe which entity "gets binding", the system converts to arg <dig> instead of arg <dig>  in this case, simple ne-/keyword-based rules cannot distinguish arg <dig> from arg <dig>
according to our analysis,  <dig> % of the pas's in the pasbiop suffered from this problem, especially pas's for verbs such as decrease, delete, inhibit, lost, mutate, transcribe and truncate.
coordination ambiguity
coordination ambiguity in the full parse information is another factor that affects conversion performance.
nk cells express cell-surface receptors of the immunoglobulin and c-type lectin superfamilies that recognize mhc class i peptides and inhibit nk-cell-mediated cytotoxicity.
the phrase "inhibit nk-cell-medidated cytotoxicity" can be coordinated with three different phrases, each with a different meaning.
this syntactic ambiguity is referred to as "coordination ambiguity"  <cit>  and is a major problem in parsing.
as you can see in figure  <dig>  our full parser coordinates the verb phrase "express cell-surface receptors of the ... class i peptides" with the verb phrase "inhibit nk-cell-mediated cytotoxicity."
therefore, biosmile tags the noun phrase "nk cells" as "arg0" for the verb "inhibit."
however, in the gold standard annotation, the pasbio developers annotate the "cell-surface receptors of ... superfamilies" as "arg0" for the verb "inhibit".
the parse tree for the pasbio's annotation is illustrated in figure  <dig>  it coordinates the verb phrase "recognize mhc class i peptides" with the verb phrase "inhibit nk-cell-mediated cytotoxicity."
although, both these parse trees were generated by our parser initially, in the end, it chose the incorrect one, figure  <dig>  because, based on the training data, that one appeared to have the highest probability.
in such cases it is impossible to distinguish the correct choice using syntactic parsing.
our results show that  <dig> % pas's in the pasbiop dataset suffered this problem.
correlation between biosmile and combined system performance
CONCLUSIONS
in this paper we have demonstrated the feasibility of converting between bioprop and pasbio annotation, which will hopefully facilitate and inspire further pasbio applications.
our approach has involved the use of our previous srl system, biosmile, as well as the development two new tools, a semi-automatic rule generator and a bioprop-pasbio converter.
our rule-generation tool can save considerable human effort by automatically generating conversion rules which only need fine tuning to be usable.
our bioprop-pasbio converter can achieve very high accuracy  using the gold-standard bioprop dataset.
our combined system  achieves an f-score of  <dig> % for pasbio's  <dig> verbs.
this performance is close to state-of-the-art ml-based srl systems in other specific domains  <cit> .
competing interests
the authors declare that they have no competing interests.
authors' contributions
rth tsai and hj dai designed the semi-automatic rule generation and rule-based conversion algorithms and wrote most of this paper.
hj dai implemented the conversion algorithm and wrote the rule generator and rule-based converter program.
ch huang, the biologist in our laboratory, verified the generated rules and conducted all experiments.
rth tsai and wl hsu guided the whole project.
supplementary material
additional file 1
supplementary materials.
click here for file
acknowledgements
this research was supported in part by the national science council under grant nsc 97-2218-e-155- <dig>  nsc96-2752-e-001-001-pae and the thematic program of academia sinica under grant as95asia <dig>
this article has been published as part of bmc bioinformatics volume  <dig> supplement  <dig>  2008: asia pacific bioinformatics network  seventh international conference on bioinformatics .
the full contents of the supplement are available online at .

 the amount of biomedical literature available online continues to grow rapidly today, creating a need for automatic processing using bioinformatics tools. many information extraction systems incorporating natural language processing techniques have been developed for use in the biomedical field. a key ie task in this field is the extraction of relations between named entities, such as protein-protein and gene-disease interactions. semantic role labeling, also called shallow semantic parsing, is a popular semantic analysis technique for extracting relations. in srl, sentences are represented by one or more predicate-argument structures, also known as propositions. each pas is composed of a predicate and several arguments that have different semantic roles, including main arguments such as an agent that deliberately performs an action and a patient that experiences an action, as well as adjunct arguments, such as time, manner, and location. here, the term argument refers to a syntactic constituent of the sentence related to the predicate; and the term semantic role refers to the semantic relationship between a predicate and an argument of a sentence. for example, in figure the sentence "il and il receptors activate stat stat and stat proteins in the human b cells" describes a molecular activation process. it can be represented by a pas in which "activate" is the predicate, "il and il receptors" comprises the agent, "stat stat and stat proteins" comprises the patient, and "in the human b cells" is the location. thus, the agent, patient, and location are the arguments of the predicate. an important preliminary task in srl is to define the set of possible semantic roles for each verb sense, referred to as a roleset. a roleset can be paired with a set of syntactic frames that shows all the acceptable syntactic expressions of those roles. this is called a frameset. in the proposition bank project published a guide, propbank i, which defined a format for pas annotation. alongside propbank i, the project also released a corpus of pas's for verbs in the newswire domain to facilitate ml-based srl system development. the semantic arguments of individual verbs in the propbank i annotation are numbered from for a specific verb, arg is usually the argument corresponding to the agent, while arg usually corresponds to the patient. however, higher-numbered arguments, which occupy about 10% of the total arguments, have no consistent role definitions. in addition to numbered arguments, there are also argms, which refer to annotation of modifiers. the semi-regular and flexible assignment of numbered arguments to semantic roles found in propbank i facilitates formulation of the srl task as a classification problem with machine-learning based systems. that is, given a phrase, the sentence containing it, and the predicate, a system must classify the phrase's semantic role corresponding to the predicate. for specific applications, however, the flexible argument assignment of propbank i annotation may be a disadvantage. in some cases, developers may wish to limit the semantic roles of each argument. take the frameset of "delete" for example. table shows the frameset definition. as you can see in table the agent is defined as "entity removing", and the patient is defined as "thing being removed" in propbank i. however, in certain biomedical events, a developer might want to limit the agent to being a certain causal mechanism such as a mutation or alternative splicing and the patient to being an "exon, gene, chromosomal region, cell". an alternative to propbank, the pasbio project provides more detailed and restrictive framesets for biomedical verbs. the well-known biomedical text mining researchers cohen and hunter have found the pasbio annotation viable for representing the pas's of biomedical verbs. several applications have been developed based on pasbio or following its spirit. for example, shah et al. used the frameset definitions of pasbio to construct semantic patterns which can extract information about tissue-specific gene expression from biomedical literature. later, shah and bork applied this approach to construct the lsat database system. kogan et al. followed the pasbio annotation to built a domain-specific set of pass for the medical domain, which successfully extended pasbio to clinical texts. all these systems mainly use handcrafted rules to identify and classify arguments into semantic roles. unfortunately, due to the lack of an annotated corpus and inconsistent definitions between specific numbered arguments, no publicly available ml-based srl systems based on the pasbio standard have been developed. to be able to apply ml to the biomedical srl problem, we constructed a biomedical domain specific proposition bank based on the more consistent propbank i annotation format. the project, bioprop, defined roles for common biomedical verbs and provided an annotated corpus on which we developed an ml-based srl system, biosmile. this work was expanded upon with the release of our web-based search application, biosmile web search, in february in this paper, we aim to build a bridge between bioprop and pasbio to facilitate pasbio-based srl system development. using our system, one will first be able to roughly classify arguments' semantic roles according to bioprop, and then translate the pas's into pasbio annotation using a rule-based converter. methods the approaches applied in this work include: named entity tagging, semantic role labeling following bioprop's annotation format, and rule-based conversion from bioprop to pasbio annotation. named entity tagging according to our observations, some bioprop arguments are equivalent to other pasbio arguments only under certain conditions, usually defined as the presence of a certain named entity in a certain argument. for example, arg of the verb "express" must be a gene or gene product in pasbio. therefore, it is necessary to first tag all nes in the sentences. to do this, we employ our previously developed ne recognition software, nerbio, to tag five ne types: protein, dna, rna, cell line, and cell type. we use a dictionary to find other ne types, such as extron and intron. semantic role labeling before conversion to the pasbio annotation format, a fundamental step is to identify the pas's of each sentence and annotate them using the bioprop format. here, we briefly introduce how we constructed the bioprop-based srl system, biosmile, used for this task. the first step was to construct a training corpus. in our previous work, chou et al., we annotated pas's in genia's corpus of full parse trees, the genia treebank, using propbank i framesets. we then defined and added framesets for biomedical verbs to fit specific usages in biomedical literature. however, all the new and modified framesets still conform strictly to the propbank annotation format. a total of pas's were annotated for biomedical verbs. the second step we took was to formulate the srl problem as an ml-based sentence tagging problem. the basic units of a sentence can be words, phrases, and constituents. punyakanok et al. has shown that constituent-by-constituent tagging is the best formulation for the srl problem; therefore, we adopted this formulation. finally, we constructed a biomedical full parser based on the charniak parser with gtb as its training data which could automatically generate parse trees for sentences. its performance is reported in additional file using bioprop as the training corpus, c-by-c formulation, and the parse trees generated by our biomedical full parser, we then constructed our srl system, biosmile, following the maximum entropy ml model. details of the features used in our srl system can be found in. development of conversion rules there are two main differences between bioprop and pasbio pas framesets annotations: pasbio developers usually define framesets to represent specific biological events. therefore, for each argument, it is necessary to include information in addition to its semantic role, such as whether the argument should be a specific ne or contain specific keywords. the order of arguments for a given verb sense in a bioprop frameset may not match that in a corresponding pasbio frameset. to deal with these two differences, we build conversion rules verb by verb using our semi-automatic rule-generation tool which describe under which conditions each mapping is valid. the algorithm used by the rule-generator compares corresponding framesets for a given verb sense, checks each argument in its pasbio frameset, and tries to find an argument in its bioprop frameset that has the same semantic role under a set of conditions. when a match is found, the algorithm maps a link between the two frameset arguments, which includes a description of required conditions. each conversion rule consists of two elements: predicates and transformations. the predicate is the target verb. the first part of each transformation is the condition, which specifies the criteria that the arguments should follow. these criteria are defined as the composition of one or more logical predicates, which are concatenated by logical operators, such as and, and or. two most common predicates are containsne and containskeywords. the former is true if the argument contains at least one instance of the ne type ne. the latter is true if the argument contains at least one specified keyword kw. if there are no conditions for a transformation, this part can be omitted. the second part is the mapping between a bioprop argument and a pasbio argument. the mapping consists of three elements: the source argument, an arrow "→", and the destination argument. for example, the transformation in figure defines a mapping from argm-loc to arg all the arguments that are not defined in the transformation source field are dropped. as shown in figure the condition of the transformation "arg → arg1" is containsne, which is interpreted as the mapping arg → arg holds if arg contains at least one protein. for a case in which arguments match, such as that in figure the conversion rules can be automatically generated as follows: for each argument pair,, if the argument phrase does not contain any recognized nes, a simple rule will be generated in the argument's "rule candidates" field: argumentb → argumentp if the argument contains recognized ne types, they will become the conditions imposed on the argument, and the following rule type will be generated: containsne?argumentb → argumentp users can modify the generated rules by editing the "rule candidates" field. in addition to defining simple conditions, such as containsne, we also describe complex conditions using a format called the bracket form pattern, which can represent syntactic and semantic information as criteria. the pattern can be applied when two or more pasbio arguments are covered by only one bioprop argument, and vice versa. a bracket form is a representation of a parse tree using brackets, to show the tree's structure. a simplified bracket form for the parse tree shown in figure with some internal bracket divisions omitted for clarity: ) )). each constituent and its daughters are enclosed with brackets. if we replace constituent words in the phrase with a wildcard symbol "", the above bracket form becomes: ) )) we can then use the bracket form as a pattern to match parse trees with the same structures. to make these patterns more precise, we can add restrictions on the phrase constituents, such as limiting their semantic roles, head words and head words' upenn pos. to restrict a constituent's semantic role, one would insert a hyphen followed by the semantic role after the constituent type. for example, might become. the head word can be defined as the most important word in a constituent, and we identify it using collins' rule-based method. head words of constituents are marked with an ampersand followed by the head word – e.g.. and the upenn pos of the head word is placed directly after, separated by a forward slash – e.g.. if we combine our above examples, we can make the pattern, ") ))", where the outside np must be arg and the inside nps' head word must be "mrna" and "kilobase" with pos's "nns." in our notation, a rule will appear as follows: bracketformpattern? c → argument c → argument..., ci → argumenti,..., ck → argumentk; "bracketformpattern" is a logical predicate which means the source argument, arguments, must match the bracket form pattern x for the transformations "ci → argumenti" to occur, where ci is any constituent of a source argument annotated by pasbio. in the example in figure for the verb "express", "arg1" in the bioprop column does not directly match any one pasbio argument, but instead overlaps two arguments, arg and arg the rule-generation algorithm first generates two bracket forms for the unmatched noun phrase "two equally abundant mrnas for il8ra and kilobases in length", one for the "bioprop" column and the other for the "pasbio" column: ) )") ) )") then, the first bracket form is merged with the second one as follows: ) )") as you can see in the merged bracket form, all the pasbio constituents annotated with semantic roles are represented by the variable ci. for example arg becomes c finally, the following three rules are automatically generated in the "rule candidates" field: bracketformpattern) )")? c → arg c → arg2 bracketformpattern) )")? c → arg c → arg2 bracketformpattern) )")? c → arg c → arg2 the first rule is the loosest, only considering the parse tree structure and srl tags. the second also considers the head word, and the third adds pos information as well. the user can check these rule candidates, and remove or modify the inappropriate ones. although these rules are semi-automatically generated, we have found from our observations that with slight human modification, they can be quite accurate. for the example in figure it is obvious that the first rule with no constraints on c and c is too loose. likewise, the third rule, which limits c and c1's pos to nns, is too strict. however, the second rule is surprisingly accurate. if we look at the frameset definitions in bioprop and pasbio shown in table we can see that pasbio defines arg as a property of arg and limits arg to a gene or gene product name. therefore, if we wish to annotate c as arg and c as arg they must match these two conditions. rule two stipulates that c1's head word should be "kilobase" and c0's should be "mrna", which matches pasbio's frameset definition for "express" because "kilobase" is a unit of mrna. therefore, the annotator could choose the second rule with head word information.  datasets the training data of our srl system, biosmile, is an extended version of bioprop. a total of pas's were annotated for biomedical verbs. to evaluate biosmile, the rule-based converter and the combined system, our in-lab biologists re-annotated the annotated sentences available on pasbio's website according to the bioprop annotation format. the dataset from pasbio's website is hereafter referred to as pasbiop and the pasbiop dataset annotated using the bioprop format is referred to as pasbiob. evaluation metrics performance was evaluated in terms of three metrics: precision, recall and f-scores, which are defined as follows: precision=the number of correctly recognized argumentsthe number of recognized argumentsrecall=the number of correctly recognized argumentsthe number of true argumentsf−scores=2×precision×recallprecision+recall for srl and conversion evaluation, the official conll- srl evaluation script was used. biosmile performance we followed the same experimental procedure that we used in to evaluate biosmile performance on the extended bioprop dataset, details about which can be found in additional file the average results were an f-score of %, a precision of % and a recall of %. to evaluate the actual performance on arbitrary sentences and verbs, we used pasbiob as an extra test data. biosmile achieved an overall f-score of %, a precision of % and a recall of %. the drop in biosmile's performance on pasbiob may be caused by the following factor: even though bioprop contains all pasbio verbs, it contains very few pas's for some verbs, which likely decreases the accuracy of ml-based srl on those verbs. for example, there is only one pas for "splice" and two for "begin". main system performance we conducted two experiments – the first to test the bioprop-pasbio converter independently of biosmile srl performance, and the second to evaluate combined system performance. for both, 3-fold cross validation was applied, which involved partitioning the pasbiop dataset into three subsets. a single subset is retained as the test data, and the remaining two subsets are used as training data for generating conversion rules. the cv process is then repeated three times, with each of the test sets being used exactly once. experiment 1: evaluating the rule-based converter in this experiment, we examined conversion performance using the pasbiop dataset, first feeding the pasbiob to the rule-based converter and then comparing the converted results with the pasbiop annotation to examine the precision, recall and f-scores. by using the pasbiob, we can effectively eliminate the influence of biosmile srl performance from this test. as shown in table we achieved an average f-score of %. the high f-score demonstrates the feasibility of our proposed semi-automatic conversion method. experiment 2: evaluating the combined system in this experiment, we examined the combined performance of our system, as shown in table compared with experiment the recall of the combined system drops 23%; however, the precision only drops 6%. this may be due to the fact that biosmile has a high precision on pasbiob but a low recall. in addition, comparing the results in table to the biosmile performance on pasbiob, we can see that the combined system's performance is higher. this might seem counterintuitive; however, if we take into account that some argument types with low accuracy, such as argm-tmp and argm-dir, are not converted to pasbio since pasbio does not define those arguments, then we can explain this discrepancy. discussion after examining the pas's which were not labeled correctly in the experiments, we have concluded that the following two factors affected conversion performance most strongly: absence of key terms for argument disambiguation in cases where one bioprop argument can be divided into two or more pasbio arguments, our rules may be insufficient to disambiguate if nes or keywords are absent. consider the following example annotated by our system with bioprop/pasbio annotations both given concatenated by a forward slash:.... the last argument is incorrectly converted from bioprop arg to pasbio arg by our system. to find out why, we must look at bioprop and pasbio's frameset definitions for "inhibit" shown in table we can see that pasbio defines both arg and arg as the objects being inhibited, but arg is further constrained to being the entity bound by the agent. bioprop, which has no arg definition, does not make this distinction. the automatically generated conversion rule for arg therefore, will have the constraint containskeywords. however, as the above example lacks any references to binding that would describe which entity "gets binding", the system converts to arg instead of arg in this case, simple ne-/keyword-based rules cannot distinguish arg from arg according to our analysis, % of the pas's in the pasbiop suffered from this problem, especially pas's for verbs such as decrease, delete, inhibit, lost, mutate, transcribe and truncate. coordination ambiguity coordination ambiguity in the full parse information is another factor that affects conversion performance. nk cells express cell-surface receptors of the immunoglobulin and c-type lectin superfamilies that recognize mhc class i peptides and inhibit nk-cell-mediated cytotoxicity. the phrase "inhibit nk-cell-medidated cytotoxicity" can be coordinated with three different phrases, each with a different meaning. this syntactic ambiguity is referred to as "coordination ambiguity" and is a major problem in parsing. as you can see in figure our full parser coordinates the verb phrase "express cell-surface receptors of the... class i peptides" with the verb phrase "inhibit nk-cell-mediated cytotoxicity." therefore, biosmile tags the noun phrase "nk cells" as "arg0" for the verb "inhibit." however, in the gold standard annotation, the pasbio developers annotate the "cell-surface receptors of... superfamilies" as "arg0" for the verb "inhibit". the parse tree for the pasbio's annotation is illustrated in figure it coordinates the verb phrase "recognize mhc class i peptides" with the verb phrase "inhibit nk-cell-mediated cytotoxicity." although, both these parse trees were generated by our parser initially, in the end, it chose the incorrect one, figure because, based on the training data, that one appeared to have the highest probability. in such cases it is impossible to distinguish the correct choice using syntactic parsing. our results show that % pas's in the pasbiop dataset suffered this problem. correlation between biosmile and combined system performance  in this paper we have demonstrated the feasibility of converting between bioprop and pasbio annotation, which will hopefully facilitate and inspire further pasbio applications. our approach has involved the use of our previous srl system, biosmile, as well as the development two new tools, a semi-automatic rule generator and a bioprop-pasbio converter. our rule-generation tool can save considerable human effort by automatically generating conversion rules which only need fine tuning to be usable. our bioprop-pasbio converter can achieve very high accuracy using the gold-standard bioprop dataset. our combined system achieves an f-score of % for pasbio's verbs. this performance is close to state-of-the-art ml-based srl systems in other specific domains. competing interests the authors declare that they have no competing interests. authors' contributions rth tsai and hj dai designed the semi-automatic rule generation and rule-based conversion algorithms and wrote most of this paper. hj dai implemented the conversion algorithm and wrote the rule generator and rule-based converter program. ch huang, the biologist in our laboratory, verified the generated rules and conducted all experiments. rth tsai and wl hsu guided the whole project. supplementary material additional file 1 supplementary materials. click here for file acknowledgements this research was supported in part by the national science council under grant nsc 97-2218-e-155- nsc96-2752-e-001-001-pae and the thematic program of academia sinica under grant as95asia this article has been published as part of bmc bioinformatics volume supplement 2008: asia pacific bioinformatics network seventh international conference on bioinformatics. the full contents of the supplement are available online at.
our first experiment evaluated our rule-based converter's performance independently from biosmile performance. in this paper, we aim to build a system to convert biosmile's bioprop annotation output to pasbio annotation. our approach allows pas conversion between bioprop and pasbio annotation using biosmile alongside our newly developed semi-automatic rule generator and rule-based converter. each pas is composed of a predicate and several arguments with different semantic roles, including main arguments as well as adjunct arguments. our system consists of biosmile in combination with a bioprop-pasbio rule-based converter, and an additional semi-automatic rule generator. semantic role labeling is an important text analysis technique. in previous work, we constructed a biomedical corpus based on the propbank standard called bioprop, on which we developed an ml-based srl system, biosmile. unfortunately, due to the lack of an annotated pasbio corpus, no publicly available machine-learning based srl systems based on pasbio have been developed. our system can match the performance of other state-of-the-art domain-specific ml-based srl systems and can be easily customized for pasbio application development. 20– october asia pacific bioinformatics network seventh international conference on bioinformatics taipei, taiwan propbank is the most widely used pas corpus and annotation format in the newswire domain. in srl, sentences are represented by one or more predicate-argument structures. in the biomedical field, however, more detailed and restrictive pas annotation formats such as pasbio are popular.
 semantic role labeling is an important text analysis technique. in srl, sentences are represented by one or more predicate-argument structures. each pas is composed of a predicate and several arguments with different semantic roles, including main arguments as well as adjunct arguments. propbank is the most widely used pas corpus and annotation format in the newswire domain. in the biomedical field, however, more detailed and restrictive pas annotation formats such as pasbio are popular. unfortunately, due to the lack of an annotated pasbio corpus, no publicly available machine-learning based srl systems based on pasbio have been developed. in previous work, we constructed a biomedical corpus based on the propbank standard called bioprop, on which we developed an ml-based srl system, biosmile. in this paper, we aim to build a system to convert biosmile's bioprop annotation output to pasbio annotation. our system consists of biosmile in combination with a bioprop-pasbio rule-based converter, and an additional semi-automatic rule generator.  our first experiment evaluated our rule-based converter's performance independently from biosmile performance. the converter achieved an f-score of %. the second experiment evaluated combined system. the system achieved an f-score of % for pasbio's verbs.  our approach allows pas conversion between bioprop and pasbio annotation using biosmile alongside our newly developed semi-automatic rule generator and rule-based converter. our system can match the performance of other state-of-the-art domain-specific ml-based srl systems and can be easily customized for pasbio application development. 20– october asia pacific bioinformatics network seventh international conference on bioinformatics taipei, taiwan 