Use of Claim Graphing and Argumentation Schemes in Biomedical Literature: A Manual Approach to Analysis (2020)

Abstract: Argumentation in an experimental life science paper consists of a main claim being supported with reasoned argumentative steps based on the data garnered from the experiments that were carried out. In this paper we report on an investigation of the large scale argumentation structure found when examining five biochemistry journal publications. One outcome of this investigation of biochemistry articles suggests that argumentation schemes originally designed for genetic research articles may transfer to experimental biomedical literature in general. Our use of these argumentation schemes shows that claims depend not only on experimental data but also on other claims. The tendency for claims to use other claims as their supporting evidence in addition to the experimental data led to two novel models that have provided a better understanding of the large scale argumentation structure of a complete biochemistry paper. First, the claim graph displays the claims within a paper, their interactions, and their evidence. Second, another aspect of this argumentation network is further illustrated by the Model of Informational Hierarchy (MIH) which visualizes at a meta-level the flow of reasoning provided by the authors of the paper and also connects the main claim to the paper’s title. Together, these models, which have been produced by a manual examination of the biochemistry articles, would be likely candidates for a computational method that analyzes the large scale argumentation structure.

## introduction the large and ever-growing quantity of biomedical literature is well known @xcite . included in biomedicine are the foundational experimental life sciences, such as genetics and biochemistry. despite the importance and abundance of this literature, few computational models have been proposed that address the argumentation that is used to support the claims made in the papers describing outcomes of experiments. such models would allow the mechanization of argumentation analysis which could enable scientific claim validation, a task made difficult because of the huge number of claims being made. a claim is any statement made within a paper which presents a novel finding based on the conducted experiment @xcite . a claim requires evidence to verify it. the structure of individual claims and their evidence is well illustrated by the toulmin model @xcite , and the logic underlying these relationships can be categorized with argumentation schemes and premise classes @xcite @xcite @xcite . the toulmin model of argumentation is adept at illustrating the components of an individual argument. when composing argumentation text, every claim that is made must be supported by evidence and a warrant connecting the claim and evidence @xcite . in the experimental sciences, the warrant is very often implicit, given that the intended audience can easily fill that slot in the argument structure. these arguments with implicit premises (and sometimes conclusions) are called enthymemes. although each individual argument can be modelled in this way, it fails to recognize the variations in how claims relate to their supporting evidence, and the means by which the warrant supports that relation. to account for this, the toulmin model can be supplemented with argumentation schemes. argumentation schemes vary by context, and none have been synthesized for biomedical literature specifically. in this research a set of 15 novel argumentation schemes developed for categorizing genetic research argumentation @xcite were used and were found to be fully transferable beyond genetics to biochemistry arguments. while categorizing all of the claims of a paper, it became evident that often the evidence for a claim was another claim, and that the majority of claims within the paper were interconnected in this way. as a result, it is possible to construct a visual representation of all of a paper's claims in a graph. this graph illustrates which claims are supported by what data and also which claims are the most representative of a paper's findings overall. the varying degrees of claim significance demonstrated by this graph can be organized into a hierarchical model, which accounts for all data within a paper while still allowing for nuance to be maintained. in addition, the analysis of individual claims is not lost, as each progression of information from one level of specificity to the next is facilitated by argumentation schemes and the toulmin model structure. the next step to understanding the argumentation set forth in a scientific paper is analyzing the complete argumentation structure of the full paper. we present here our preliminary work on the analysis of this larger scale argumentation structure of biochemistry papers. this study is done by examining the flow of the paper from its data used as premises to support certain claims, some of which are used as premises for other claims, ending finally with the main claim of the paper. this research follows a similar path to @xcite , who manually constructed large scale graphs, and adds to the argumentation structure research by examining complete biochemistry articles and linking the argument schemes. two models of analysis will demonstrate the interactions of claims. the first model is a highlevel argument diagram of the data and claims structure capturing the essence of @xcite and in the spirit of other high-level diagramming models @xcite @xcite and diagramming-assisting software @xcite . the network of data and claims allows one to investigate its properties. in particular, the data is found in figures, tables, and other comments by the paper's authors. this data initiates the argumentation flow which ends in the main claim of the paper. this investigation leads to the second model, the model of informational hierarchy (mih). it makes more precise the units of the argument structure and their position in that structure. this hierarchical description of data and claims differs from other argumentation structures such as online debates @xcite and student essays @xcite . in addition to the main focus, our research also demonstrates the applicability of @xcite argumentation schemes not only to genetics, but also to other biomedical research, biochemistry, in particular. also, as an application of these models, the mih can be viewed as a precursor to biomedical literature summarization. the paper is organized as follows: some related work to provide context for the current work is presented next. this is followed by a short description of the the five biochemistry papers that were used in the study. then, the two main contributions, the claim graph and the model of informational hierarchy, are explained. we conclude with a summary and some proposed future research directions. ## related work our interest in investigating the larger scale argumentation structure has a similar motivation to the works of @xcite and @xcite , who are interested in investigating various properties of large scale argument networks. this new dimension adds to the previous works they point to as examples that consider particular aspects of argument structure: distinguishing argumentative and non-argumentative sentences @xcite , classifying text spans as premises or conclusions @xcite , classifying relations between specific sets of premises and their conclusion @xcite , or classifying the different types of premise that can support a given conclusion @xcite . in addition to these examples, various manual and computational studies have been done to analyze different argumentation aspects, including: the structure of valid arguments in legal documents using feature-based machine learning @xcite , opinion and didatic texts using rhetorical structure theory (saint-dizier, 2012), debates using textual entailment @xcite , and deconstructing the argumentation into the premises (referred to as evidence in the evidence-based medical text genre) and conclusions (also referred to as claims) in scientific articles manually, with feature-based machine learning, or with neural end-to-end machine learning @xcite @xcite @xcite @xcite @xcite @xcite @xcite . @xcite provide an excellent survey of research done until 2015 which is further updated to 2018 by @xcite . corpus creation and analysis has also been another aspect of argumentation mining studies @xcite . more recently, neural net machine learning has provided a new machine learning paradigm for doing cross-domain claim analysis @xcite . @xcite and @xcite provide neural end-to-end models for computational argumentation mining. they label student essays and randomized control trials, respectively, with a bio encoding to indicate argumentative and non-argumentative text spans, component type, and the stance between the components. the rich history of work in argumentation mining has tended to focus on non-scientific text, however work in scientific text argumentation mining does have a following. the research done on various aspects of argumentation in scientific text begins with argumentation zoning (az) @xcite , also being some of the earliest work in argumentation mining. az is based on rhetorical moves, an important precursor for mapping out certain aspects of argument structure. rhetorical moves, captured as az or more generally, have been investigated in a few science genres: computational linguistics @xcite , biochemistry @xcite , molecular biology @xcite , and chemistry @xcite . argumentation schemes @xcite have been an important aspect of argumentation and argumentation mining. @xcite has provided an important addition to these argumentation schemes for experimental scientific writing, specifically for genetics articles. while green's argumentation schemes deal with aspects of the experiment, its outcomes, and the analysis of those outcomes, other work @xcite focusses on a different aspect of argumentation (via rhetorical moves): placing a research paper in its scientific context. al @xcite propose argumentation schemes for clinical discussions and use these schemes in an argument graph to analyze a discussion. argument diagramming is a technique that is commonly used to describe argumentation. while a manual operation, in the digital age, some computer-supported argument visualization tools have been developed. araucaria @xcite and ova+ @xcite are two examples. the first provides support for mapping argumentation schemes. the second was developed for assisting with diagramming the larger scale argumentation structures. @xcite provides an argument diagram for large scale online discussions. @xcite while mainly focussed on a neural endto-end model for computational argumentation mining also provides an almost tree-like argumentation structure of complete student essays. ## dataset although there has been significant research on the deconstruction of individual claims and the methods of identifying claims within texts of various sorts @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite , we are interested here in working with a new subset of scientific texts, biochemistry texts in particular, and linking claims into a larger argumentation structure. the dataset that we have used consists of five papers. although the number of papers in the dataset used in this research is small, what sets it apart is that the annotation of the claims has been done by our domain expert, dr. derek mclachlin, one of the co-authors of the five papers. our findings and the techniques and models presented are all based on the analysis of papers by our domain expert. the five papers all concern the dimerization interactions of the b-subunit of escherichia coli atp synthase. having a co-author of the paper source the claims directly increased their reliability as representative of the papers' findings. dr. mclachlin additionally provided detailed lists of claim interactions, indicating the direct source of evidence behind each claim. although the source material for the findings reported here is limited to one author, the literature used follows a structure ubiquitous to biomedical research, specifically, the progression of abstract, introduction, procedure/methods and materials, results, and discussion @xcite . this is standard for research of this nature and doesn't affect the applicability of the proposed models to other similar literature. once a paper's claims and their sources have been determined it is possible to employ graphing techniques and information modeling. ## argumentation structure as stated earlier, the focus of this paper is the investigation of large scale argumentation structure in the spirit of @xcite and @xcite . the notion of large scale for our purposes is one complete biochemistry article. as discussed in section 3, the claims and their spans have been provided by one of the co-authors of each paper. in addition, the interaction of the data and the claims has been provided by this co-author. our research uses the toulmin model @xcite . we have chosen the argumentation schemes provided by @xcite because they have an experimental science basis (cf. the clinical discussion schemes proposed by al @xcite and the randomized control trial evidence classes proposed by @xcite ). we refer to the model and scheme as the toulmin-green model. @xcite has noted that many of the arguments in scientific writing are enthymemes, arguments that are missing premises or conclusions, and in most cases it is the warrants that are missing. what is missing from the information provided by our domain expert are the warrants. these have been provided by the first author of the current paper. an example of an argument: premise (claim 19) @xcite : the highest level of disulfide formation was observed with the s60c + l65c and a61c + l65c combinations. premise [missing warrant]: proximity is necessary for disulfide binding between residues. claim 21a (green's argumentation scheme: effect to cause ( 5 )): the result suggests that residue 65 of one subunit is close to residues 60 and 61 of the other given these elements, we are now able to analyze the argument structure of a biochemistry paper. we provide a categorization of the claims, a graphing technique that demonstrates the large scale argumentation structure, and an organization of the data and claims that we call the informational hierarchy. ## conclusions and future work the purpose of analyzing the five papers with the toulmin-green model and subsequently generating the claim graph and the model of informational hierarchy has been to illustrate a manual approach for generating a large scale argumentation structure for a complete biochemistry paper. the methods presented are novel for this science genre. one can compare this argumentation structure with those that have been established for debates @xcite , persuasive essays @xcite , and student essays @xcite . as is shown with the graph, claims in biochemistry articles are intrinsically interlinked, with claims using both other claims and data for their evidence. the mih model lists argumentation schemes as intermediary steps between levels of information to illustrate the importance of claims relating to each other in a way that is categorizable and logically sound. beyond these results, there is potential for further research. the toulmin-green model, the claim graphing technique, and the mih all have aspects which can be further developed. green's @xcite innovative idea to use prolog rules to generate argument schemes can be improved. as proposed, the body of each rule matches with the prolog knowledge base extracted from the text. the body of each rule needs some discourse information to constrain what elements from these prolog facts are allowed to be combined. we can use the information derived from the claim graph to add to these rule bodies, i.e., only those facts that are extracted from elements that are connected by an edge would be allowed to imply the argument scheme (the head of the rule). however, it still remains to be seen how to do this in an automatic way since the graphs have been constructed manually. using discourse and rhetorical moves may be some directions to investigate. in addition to this, a method to produce implicit argument components for enthymemes is needed. biochemistry (and other experimental life sciences) papers are written following the imrad (introduction, methods, results, and discussion) structure. as indicated by dashed edges in the claim graph in figure 1 and mentioned in the example in section 4.3, the placement of a claim in this structure may have stylistic significance. this aspect of the writing style was not part of this initial study and needs to be further investigated and incorporated in the claim graph. indeed, that a claim is restated and where this restatement is placed may provide further justification for the split into the three claim categories proposed in the mih. as well, a characteristic of writing in the biochemistry genre is to explain the reasons for choosing the experiments discussed in the paper. this aspect is important to a full understanding of the argumentation structure. how to incorporate it in the claim graph needs to be investigated. discussion of claims in the paper highlighted above commented on claims in previously published papers. this important inter-paper argumentation structure will be investigated. an important next step is to produce the claim graph automatically. methods to identify claims in biomedical text (albeit not always directly applicable to this type of experimental biochemistry article) ranging from rule-based @xcite to neural end-to-end @xcite have been previously investigated and would comprise a first step toward this goal. a future application of the model of informational hierarchy could be summarization of the paper, a noted motivation for some of the earliest argumentation research @xcite . moving from the top of the hierarchy (the paper's title and main claim) downward would provide more and more detail which is not contained in an abstract. real-time summarization at a user-specified level of detail seems possible. and a summarization focussed on a particular aspect of a paper's research claims combining the mih and the claim graph could also be a possibility.