UniT: One Document, Many Revisions, Too Many Edit Intention Taxonomies
Authors: Fangping Lan, Abdullah Aljebreen, Eduard Dragut
Accepted at: acl25

UNIT: One Document, Many Revisions, Too Many Edit Intention Taxonomies

Fangping Lan1, Abdullah Aljebreen2, Eduard C. Dragut1,
1Temple University 2Shaqra University
fangping.lan@temple.edu, az.aljebreen@su.edu.sa, edragut@temple.edu

Abstract

Writing is inherently iterative, each revision enhancing information representation. One revision may contain many edits. Examination of the intentions behind edits provides valuable insights into an editor’s expertise, the dynamics of collaborative writing, and the evolution of a document. Current research on edit intentions lacks a comprehensive edit intention taxonomy (EIT) that spans multiple application domains. As a result, researchers often create new EITs tailored to specific needs, a process that is both time-consuming and costly. To address this gap, we propose UNIT, a UNified edit intention Taxonomy that integrates existing EITs encompassing a wide range of edit intentions. We examine the lineage relationship and the construction of 24 EITs. They together have 232 categories across various domains. During the literature survey and integration process, we identify challenges such as one-to-many category matches, incomplete definitions, and varying hierarchical structures. We propose solutions for resolving these issues. Finally, our evaluation shows that our UNIT achieves higher inter-annotator agreement scores compared to existing EITs and is applicable to a large set of application domains1.

1 Introduction

Text is an important medium for sharing information, whether it is reporting news(Chen et al., 2024; Zhijia Chen and Dragut, 2025), documenting scientific discoveries, or narrating stories. Writing is an iterative process that involves revising text to improve how information is conveyed. One revision is created when an editor saves changes to the current document, potentially including multiple local edits (Yang et al., 2017). A single document may undergo multiple revisions. Studying these revisions and the intentions behind them offers significant value in natural language processing tasks (Wang and Dragut, 2024; Zhang et al., 2024, 2025). For instance, analyzing stylistic edits in writing not only helps identify grammatical errors but also provides insight into the structure of arguments and the overall design of the text (Zhang and Litman, 2016; Shah et al., 2019; Spangher et al., 2022). Revisions may be grouped in patterns and the relationship between the patterns can inform about an editor’s level of writing experience (Jones, 2008; Yang et al., 2017). In addition, the analysis of intentions enables the creation of assistive tools, such as recommender systems for the collaborative writing (Yang et al., 2016). For lengthy documents such as laws, user agreements, and software manual, quickly identifying revisions and understanding their purpose can reduce reading time, enabling readers to focus on the most relevant sections without having to review the entire document.

Current studies tend to focus on single domains, such as collaborative authoring (Pfeil et al., 2006), informative writing (Spangher et al., 2022; Guo et al., 2022), journalistic writing (Zhang and Litman, 2015), and source code (Lee et al., 2021). Other works study certain aspects of editing revisions, such as style, syntax (Faigley and Witte, 1981), semantics analysis (Yang et al., 2017), and the edits objects analysis (Yang et al., 2016). A pattern observed in the literature is that when researchers need to analyze revision intentions in a new domain, they develop a new EIT either by building it from scratch or adapting an existing one, incurring time and potential financial cost.

We develop a comprehensive EIT that spans multiple application domains and encompasses all observed edit intentions, reducing the burden to create a new EIT for each application. This can be achieved either by constructing the EIT from scratch or by leveraging existing ones. We opt for the latter because jointly existing EITs cover

1Our UNIT is publicly released at https://github.com/lanfangping/Unit-EditIntentionTaxonomy
<table>
  <tr>
    <th colspan="6">One-to-many matches*</th>
  </tr>
  <tr>
    <td colspan="6">Copy-Editing[Yan17]<br><b>Definition:</b> Rephrase, improve grammar, spelling, tone or punctuation.</td>
  </tr>
  <tr>
    <td>Rephrase[Yan16]<br><b>Definition:</b> An edit attempts to paraphrase words or sentences</td>
    <td>Grammar[Pfe06]<br><b>Definition:</b> Alterations of the grammar (e.g., change of punctuation).</td>
    <td>Spelling[Pfe06]<br><b>Definition:</b> Correction of spelling mistakes (e.g., reversed letters or capital letter).</td>
    <td>Point of View Change [Raj22]<br><b>Definition:</b> Removing bias, rewriting using neutral tone.</td>
    <td>Punctuation [Fai81]<br><b>Definition:</b>/<br>No definition and example</td>
  </tr>
</table>

* omitting category examples for clarity.

Figure 1: A category in an EIT is overlapping with more categories in other EITs (one-to-many category matches).

![Workflow of EIT Integration](page_349_563_1047_180.png)

Figure 2: Workflow of EIT Integration

multiple application domains, have a large number of edit (sub)categories, and include many edit intentions. In this paper, we report our work on creating UNiT, a unified and comprehensive EIT, by painstakingly studying and integrating all\(^{2}\) existing EITs. Thus, users aiming to develop a new EIT tailored to their specific task can simply select a subset of edit intentions from our UNiT without the need to search through multiple existing EITs.

1.1 Challenges

There are a number of challenges toward the construction of UNiT. First is the literature coverage to find all proposed EITs. Second is the integration procedure. A naive approach is to study EITs pairwise and iteratively merge them (Do and Rahm, 2002; Sundaresan and Hu, 2005; Chiticariu et al., 2007). An alternative approach is to identify groups of related EITs and perform n-way merge within each group (He et al., 2004; Wu et al., 2004; Dragut et al., 2009a,c,b, 2006a, 2022). We apply the n-way merge iteratively for the integrated EITs in each group. Third is edit intention matching. This is challenging because of (1) one-to-many category matches, i.e., a category in an EIT overlapping with two or more categories in another EIT. Figure 1 gives a one-to-many match example, Copy-Editing (Yang et al., 2017) matches Rephrase (Yang et al., 2016), Grammar and Spelling (Pfeil et al., 2006), Point of View Change (Rajagopal et al., 2022) and Punctuation (Faigley and Witte, 1981), if one studies their definitions and revision examples. (2) Incomplete definitions of categories in some EITs, e.g., missing or incomplete definitions and/or revision examples. The category Punctuation (Faigley and Witte, 1981) does not have a definition nor revision examples. We match it to other categories assuming its intended meaning from the name. 58.6% of categories have definitions and at least one revision example, but not all of them have enough revision examples to exemplify their intended edit use cases. (3) EIT hierarchical organization, some EITs are flat while others have multiple levels. Fourth is coping with conflict resolution and missing information (i.e., definition or revision examples) in the integrated EIT. Finally, is the experimental setup. We empirically study if UNiT can be used in multiple domains and if the curation process in constructing UNiT (e.g., improved category definitions and revision examples) leads to better annotations.

1.2 Integration Process Overview

Figure 2 depicts our process of building UNiT. We start by searching for publications related to edit intentions. For each relevant paper (i.e., a paper that defines an EIT), we look into its references and the papers that cite it for additional relevant pa-
pers. We exclude papers that study edit intentions but do not provide an EIT. We iterate until we can not find any relevant papers. We gather 24 papers, summarized in Table 1. Then, we try to group the EITs from the 24 papers in smaller groups. We study the citation network between the papers to find relationships between EITs. That is a useful exercise because EITs in the same group tend to have many one-to-one matches and fewer requirements on solving one-to-many matches. For example, we find EITs that merely adapt existing ones, or authors that refine their EITs over time. This leads to a lineage relationship between subsets of EITs, where an EIT is obtained from another EIT via a number of edits, such as adding new nodes, collapsing categories, and renaming categories to denote broader edit intentions. We then integrate the EITs in each group, then integrate the resulting EITs into the final EIT, UNIT. Thus, the integration of EITs is an inherently iterative process. We evaluate our UNIT by evaluating its value to annotators and its versatility across different domains. This is done through the design of a manual annotation task and the measurement of inter-annotator agreement (IAA) among the participants. The contributions of this paper are as follows:
• We study 24 EITs and analyze their properties, such as structure, definitions, and revision examples (Section 2). We also collect external resources, i.e., codes and datasets, and report on their reproducibility.
• We study the lineage relationship between EITs. (Section 3.1).
• We build an integrated EIT that spans multiple application domains and includes a large set of edit intentions. (Section 4).
• We compare our UNIT with existing EITs showing that it achieves higher IAA scores and it is applicable to a larger set of application domains (Section 5).

2 Literature Review

This section presents, to our knowledge, all the papers that propose EITs. A criteria for a good EIT is defined as follows: an edit intention taxonomy provides well-structured categories with unambiguous category names, concise definitions, and representative examples. Thus, we organize it along four dimensions: type of the EIT structure, the presence of definitions and revision examples for EIT categories, application domains (covered in Appendix A.1), and availability of EITs (covered in Appendix A.2). Table 1 summarizes our findings.

2.1 Structure

EITs are tree structures, ranging from flat (i.e., list of categories) to 3-level deep trees. EITs with multiple levels include (sub)categories for edit actions (e.g., add, merge, delete) in addition to the revision purposes (e.g., Update Content).

We present first the 3-level EITs. Their top layer typically indicates whether a revision affects the meaning of text. Faigley and Witte (1981) and Zhang and Litman (2015) use the terms Surface Changes for meaning preserving edits and Text-Base Changes for meaning changing edits. Yang et al. (2016) adopt simpler labels, Meaning-Preserving and Meaning-Changing. Faigley and Witte (1981) subdivide Text-Base Changes based on whether the text’s summary is affected, distinguishing Microstructure Changes (summary unchanged) from Macrostructure Changes (summary altered). Yang et al. (2016) refine their Meaning-Changing category by considering the specific objects being edited (e.g., templates, files). The leaf categories represent edit actions (e.g., addition and permutation of text pieces). The top layer of most 2-level EITs (Jones, 2008; Faltings et al., 2021; Du et al., 2022; Daxenberger and Gurevych, 2012, 2013; Zhang et al., 2016, 2017; Kashefi et al., 2022; Laban et al., 2023; Ruan et al., 2024) is similar to that of 3-level EITs. Some 2-level EITs introduce application-specific categories, such as Wikipedia Specific Intention (Jones, 2008; Daxenberger and Gurevych, 2012, 2013). Jiang et al. (2022) mainly focus on meaning-preserving changes. Ruan et al. (2024) categorize edit actions into basic methods(i.e., addition, deletion, modification) and complex operations(i.e., merges, splits and fusions).

Regarding the flat EITs, we note that some works flatten the hierarchical EITs proposed in previous works, e.g., Zhang and Litman (2016) and Rajagopal et al. (2022) flatten those in Zhang and Litman (2015) and Yang et al. (2017), respectively. Spangher et al. (2022) propose the EIT only on edit actions. Rathjens (1985) give fine-grained categories for tracking edits for the clarity of technical writing. Guo et al. (2022) include categories specific to news headline changes. Faruqui et al. (2018) have categories to characterize the types of insertions. Pfeil et al. (2006), Liu and Ram (2011), and Anthonio et al. (2020) define categories for surface and content changes, though they do not
<table>
  <tr>
    <th>alias</th>
    <th>paper</th>
    <th>domain</th>
    <th>#le.</th>
    <th colspan="3">has_definition</th>
    <th colspan="2">has_examples</th>
    <th>link?</th>
  </tr>
  <tr>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th>11</th>
    <th>12</th>
    <th>13</th>
    <th>extents</th>
    <th>from</th>
    <th></th>
  </tr>
  <tr>
    <td>Fai81</td>
    <td>Faigley and Witte, 1981</td>
    <td>Writings</td>
    <td>3</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>Partial</td>
    <td>Partial</td>
    <td>Inside</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Zha15</td>
    <td>Zhang and Litman, 2015</td>
    <td>Writings</td>
    <td>3</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>No</td>
    <td>Full</td>
    <td>Inside</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Zha16(A)</td>
    <td>Zhang et al., 2016</td>
    <td>Writings</td>
    <td>2</td>
    <td>Yes</td>
    <td>Yes</td>
    <td></td>
    <td>Full</td>
    <td>Outside</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Zha17</td>
    <td>Zhang et al., 2017</td>
    <td>Writings</td>
    <td>2</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>/</td>
    <td>Full</td>
    <td>Both</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Fal21</td>
    <td>Faltings et al., 2021</td>
    <td>Writings, Wikipedia</td>
    <td>2</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>/</td>
    <td>Partial</td>
    <td>Inside</td>
    <td>Broken</td>
  </tr>
  <tr>
    <td>Du22*</td>
    <td>Du et al., 2022</td>
    <td>Writings, Wikipedia, News</td>
    <td>2</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>/</td>
    <td>Full</td>
    <td>Both</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Jia22</td>
    <td>Jiang et al., 2022</td>
    <td>Writings</td>
    <td>2</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>/</td>
    <td>Full</td>
    <td>Both</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Kas22</td>
    <td>Kashfi et al., 2022</td>
    <td>Writings</td>
    <td>2</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>/</td>
    <td>Full</td>
    <td>Both</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Rua24*</td>
    <td>Ruan et al., 2024</td>
    <td>Writings</td>
    <td>2</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>/</td>
    <td>Full</td>
    <td>Both</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Rat85</td>
    <td>Rathjens, 1985</td>
    <td>Writings</td>
    <td>1</td>
    <td>Yes</td>
    <td>/</td>
    <td>/</td>
    <td>Full</td>
    <td>Inside</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Zha16(B)</td>
    <td>Zhang and Litman, 2016</td>
    <td>Writings</td>
    <td>1</td>
    <td>Yes</td>
    <td>/</td>
    <td>/</td>
    <td>Partial</td>
    <td>Inside</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Yan16</td>
    <td>Yang et al., 2016</td>
    <td>Wikipedia</td>
    <td>3</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>No</td>
    <td>No</td>
    <td>/</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Jon08</td>
    <td>Jones, 2008</td>
    <td>Wikipedia</td>
    <td>2</td>
    <td>Yes</td>
    <td>No</td>
    <td>/</td>
    <td>No</td>
    <td>/</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Dax12</td>
    <td>Daxenberger and Gurevych, 2012</td>
    <td>Wikipedia</td>
    <td>2</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>/</td>
    <td>Full</td>
    <td>Inside</td>
    <td>Moved</td>
  </tr>
  <tr>
    <td>Dax13</td>
    <td>Daxenberger and Gurevych, 2013</td>
    <td>Wikipedia</td>
    <td>2</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>/</td>
    <td>No</td>
    <td>/</td>
    <td>Broken</td>
  </tr>
  <tr>
    <td>Yan17</td>
    <td>Yang et al., 2017</td>
    <td>Wikipedia</td>
    <td>2</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>/</td>
    <td>Full</td>
    <td>Outside</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Lab23</td>
    <td>Laban et al., 2023</td>
    <td>Wikipedia</td>
    <td>2</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>/</td>
    <td>Full</td>
    <td>Both</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Pfe06</td>
    <td>Pfeil et al., 2006</td>
    <td>Wikipedia</td>
    <td>1</td>
    <td>Yes</td>
    <td>/</td>
    <td>/</td>
    <td>Full</td>
    <td>Inside</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Liu11</td>
    <td>Liu and Ram, 2011</td>
    <td>Wikipedia</td>
    <td>1</td>
    <td>Yes</td>
    <td>/</td>
    <td>/</td>
    <td>No</td>
    <td>/</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Far18</td>
    <td>Faruqui et al., 2018</td>
    <td>Wikipedia</td>
    <td>1</td>
    <td>Yes</td>
    <td>/</td>
    <td>/</td>
    <td>Full</td>
    <td>Both</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Raj22</td>
    <td>Rajagopal et al., 2022</td>
    <td>Wikipedia</td>
    <td>1</td>
    <td>Yes</td>
    <td>/</td>
    <td>/</td>
    <td>Full</td>
    <td>Both</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Ant20</td>
    <td>Anthonio et al., 2020</td>
    <td>WikiHow</td>
    <td>1</td>
    <td>No</td>
    <td>/</td>
    <td>/</td>
    <td>Full</td>
    <td>Both</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Spa22</td>
    <td>Spagnher et al., 2022</td>
    <td>News</td>
    <td>1</td>
    <td>Yes</td>
    <td>/</td>
    <td>/</td>
    <td>Full</td>
    <td>Both</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Guo22</td>
    <td>Guo et al., 2022</td>
    <td>News</td>
    <td>1</td>
    <td>Yes</td>
    <td>/</td>
    <td>/</td>
    <td>Full</td>
    <td>Inside</td>
    <td>Active</td>
  </tr>
</table>

Table 1: It shows the application domain of the taxonomy in a paper; #le. is the number of levels and whether it is flat (#le.=1) or hierarchical (#le.>1); whether the categories in level # (1#) has definitions (has_definition); whether ‘Full’, ‘Partial’ or ‘No’ leaf categories (extents) have concrete revision examples (has_examples) and they are from ‘Inside’ of the paper, ‘Outside’ of the paper through provided link (link?) or ‘Both’. ‘/’ denotes not applicable. ‘*’ denotes EIT in this work is compared against UNIT in Section 5.

emphasize the meaning changes within revisions.

2.2 Definitions and Revision Examples
The ease of use and adoption of a taxonomy depends largely on the clarity of its category definitions and the inclusion of illustrative examples (Dilnutt, 2004). We survey those properties here.
Definitions We observe several patterns about category definitions in EITs. While most include definitions for their categories, some may omit them, particularly when a category has a self-explanatory name, such as Spelling/Grammar, or when suggestive examples are provided (Anthonio et al., 2020). There are cases where categories miss both, leading to ambiguity. For example, Style and Readability (Jones, 2008) may refer to the edits that adjust writing style (making the text more technical or narrative) or to changes in the text appearance (e.g., bold and italic formatting). The quality of the definition is also important. For instance, some definitions are overly broad (e.g., Copy-Editing in Figure 1), while others are misaligned with the category name. For example, Adding Support Evidence implies adding information, but its definition includes edits that involve removing or replacing information, creating confusion.

Revision Examples Most EITs provide revision examples for leaf categories but not for the internal node ones. Among those that lack revision examples, we notice two cases 1) offer examples only for a subset of the categories (Faigley and Witte, 1981; Zhang and Litman, 2016; Faltings et al., 2021) and 2) no category has examples (Jones, 2008; Liu and Ram, 2011; Yang et al., 2016). In some instances, the absence of examples is mitigated by offering detailed explanations for the categories (Pfeil et al., 2006; Laban et al., 2023).

Definition and revision examples support category matching, while existing EIT structures and their application domains inform the design of top-layer categories, helping annotators identify edit intentions efficiently. Their released datasets are a useful resource to evaluate EITs, including UNIT, and advancing research in this field.
Figure 3: The lineage of the prior EITs. Each block is an EIT in a paper and the dashed arrow lines show the references to other EITs. The link labels indicate how the “child” EIT inherits from the “parent” EITs. The blocks with the same color are grouped based on the lineage analysis, e.g., Pfe06 and Liu11 with yellow color are grouped.

3 Grouping EITs based on Lineage

The previous section outlines the contributions of past work on EITs. Here, we examine the relationships between EITs by analyzing the citation network among the reviewed papers, which reveals a lineage network among EITs, e.g., one EIT builds upon another by performing changes, such as category merging or splitting. We describe this network in this section and use it in the next section to group EITs. Then we proceed with integration of EITs by groups. Figure 3 outlines the lineage.

3.1 Observations on EIT Lineage

Fai81\(^3\) is the first EIT. Jon08 expanded on it by simplifying the subcategories under Macro/Micro-structure Changes and adding new categories to account for Wikipedia-specific revision behaviors. Dax12 and Zha15 adopted the high-level categories Surface Changes and Text-Base Changes from Fai81 but modified the subcategories to align with their respective domains and targets, focusing on Wikipedia objects (e.g., file, template) and the structural elements of academic writings (e.g., claims/ideas, introductory materials), respectively.

Du22 categorized the edits into Meaning-changed and Non-meaning-changed, focusing on the impact of edits on text meaning. Building on the work of Raj85 and Harris (2017), Du22 defined subcategories for Non-meaning-changed. Liu11, in turn, streamlined the detailed categories in Pfe06, arguing that these categories were not feasible for automatic identification by classifiers. In addition to the “link”-related categories in Pfe06, Liu11 added “references”-related categories, arguing that they are important quality indicators.

Zha16(B) built on Zha15, simplifying it by consolidating all subcategories under Surface into a single Surface category, as the focus was on augmentative changes. Zha16(B) also merged Rebuttal and Warrant categories into Warrant based on the observation in Zha15 that rebuttal edits were rare. Yan16 extended Dax12 by further fine-graining Reference. Yan16 used edit intentions to identify editor roles, while Dax12 studied the collaborative behavior among authors.

Yan17 was an EIT for wiki edits by distilling and extending categories in Fai81, Dax12 and Zha16(B). Yan17 organized the top-level categories based on whether the revisions were general or specific to Wikipedia, with most subcategories being newly introduced. Fal21 reorganized the 13 leaf categories from Yan17 into three groups Fluency, Content, and Other, as they distinguished fluency edits, which affect the grammar or structure of a sentence, from content edits, which change the meaning of a sentence. Raj22 further refined Yan17 by collapsing similar categories and merging all Wikipedia-specific categories into Other category.

Guo22 was built using Yan17 and Dax13 as references but tailored to news headline changes, whereas the source EITs focused on edits in Wikipedia articles. Since many categories from the referenced EITs were unsuitable for news headlines, Guo22 added new categories based on their journalism expertise (Blom and Hansen, 2015; Brown and Sinta, 2016). Ant20 was built on their

\footnotetext{3For compactness, we use aliases for the papers to reference their EITs. See Table 1 for full reference.}
experience of manually inspecting revisions in a sample of 100 sentence pairs.

3.2 Grouping

We leverage our observations about the lineage relationship between EITs to organize them into groups, such that EITs in the same group have higher overlap between their categories. This approach simplifies category matching between EITs and the integration process. Appendix A.3, Table 6-10 give the resulted groups.

Reuse-based Grouping Dax12 and Dax13 go into the same group since Dax13 is builds on the EIT from Dax12 (Table 7). Fal21 and Yan17 are grouped because they share the same leaf categories despite different top layer structures (Table 9). Zha15, Zha16(A), Zha17, and Kas22 are grouped together because the subsequent works reused the EIT in Zha15, making minor adjustments to category names to fit their specific system requirements (Table 8).

Merge-based Grouping We group Yan17 and Raj22 as Raj22 consolidates all Wikipedia-related categories into one category and combines similar categories from Yan17 (Table 9). We group Zha15 and Zha16(B) as Zha16(B) mainly merges some of the categories from Zha15 (Table 8). We group Rat85 and Du22 since Du22 consolidates all categories in Rat85 under one category (Table 6).

Category Refinement Grouping We group Pfe06 and Liu11 as well as Dax12 and Yan16 because Liu11 simply refines Reference changes from Pfe06, while Yan16 refines Reference categories from Dax12 (Table 7 and Table 10, respectively).

After grouping, we have 5 groups and 9 individual EITs, reducing the total number of EITs to be integrated from 24 to 14. This reduction of nearly 50% significantly reduces the integration workload.

4 Taxonomy Integration

We define the EIT integration task as follows: given a set of EITs, generate a single integrated EIT that includes all the categories from the individual EITs, subject to certain modifications. This section discusses the process of EIT integration, including matching (Appendix A.7 gives partial matching results), merging, as well as adding revision examples (naming and adding definitions are covered in Appendix A.4 and A.5).

4.1 Matching

Given a group of EITs, the first step is category matching, which identifies semantically equivalent categories across the EITs. We employ the three sources of information: name, definition, and revision examples. The matching process is challenging because of incomplete definitions/revision examples and one-to-many category matches.

We first conduct matching using the category names, e.g., Copy Editing [Yan17] and copy-editing [Fal21]. Next we employ category definitions, which can help in cases such as categories with different names, but equivalent meaning, and categories with broad edit meanings. For example, in Figure 4, Edit [Spa22] and Sentence Split [Lab23] can only be matched because of their definitions. Finally, when neither name nor definition can help match two categories, we study their accompanying revision examples. For example, we are able to match Information Modification/Insertion [Ant20], for which a definition is missing, with Anaphora Resolution [Lab23] (Figure 4). This is achieved by analyzing an example of Information Modification/Insertion where “them” is edited to “your parents”. By combining this example with Anaphora Resolution’s definition, which states that changing implicit entity mention with a resolved entity mention, we can reliably match them. Note that the above process needs to account for instances when a category has a definition but no examples (e.g., Disambiguation [Fal21]), or has examples but no definition (e.g., Information Modification/Insertion [Ant20]).

Exceptions (i) There are categories that lack definitions and revision examples. During the construction process, if we could not reliably match them, we set them aside to revisit after completing the initial draft of the integrated EIT. However, they did not give new categories. (ii) Certain categories share similar names, but different meanings. For example, Clarification [Yan17] and Clarity [Du22] have similar category names but their definitions have different actions, the former defines “Specify or explain fact or meaning by example or discussion without adding new information” and the latter defines “Make the text more formal, concise, readable and understandable.” (iii) Lastly, there are one-to-many category matches, as illustrated in Figure 1. Establishing such matches requires the presence of definitions and revision examples. These matches often indicate categories with a
broad range of edit revisions. We often find the one-side confusing and opt to keep the many-side in the integrated EIT as children of the one-side.

4.2 Merging

After the matching step, semantically equivalent categories from the EITs are organized in semantic clusters. There are 31 clusters organizing the 232 categories from the 24 EITs. A cluster can have up to 24 categories, but the average is 8. We organize the clusters, into two groups, Fine-grained Intentions and Edit Intentions (Figure 5 in Appendix A.8). In general, a cluster becomes a node/category in the UNiT. For example, the cluster of Edit [Spa22] and Sentence Split [Lab23] becomes a node Splitting Sentences in UNiT (we discuss category naming in Appendix A.4).

One-to-Many Integration of one-to-many cases requires careful consideration. Consider the example of Refactoring [Yan17] in Figure 4. It matches one-to-one with Refactoring [Fal21] and one-to-many with Paraphrase [Dax12], Refactors [Spa22], and Word-smithing [Raj22] (refer to their definitions in the figure). We handle it by creating a subtree with the root Refactoring and Paraphrase, Refactors, and Word-smithing as its children in UNiT. (More precisely the clusters of these categories form the nodes in UNiT.) Note that Refactoring (renaming to Word-Smithing discussed in Appendix A.4) belongs to Edit Intentions, while those of Paraphrase (renaming to Substituting Words/Phrases), Refactors, and Word-smithing (renaming to Permuting Blocks) belong to Fine-grained Intentions (Figure 5). At the end of the merge process, we have 24 categories in Edit Intentions and 13 in Fine-grained Intentions. Next, we discuss the organization of the edit intentions.

Top Layer Integration The top-level categories of UNiT are based on two main criteria: (1) whether an edit changes the meaning of text, and (2) whether the edit is domain-specific. We introduce a Top-layer defined as follows. Edit Intention clusters become children of Non-Meaning-Changed and Meaning-Changed if they denote edits that affect the summary of the text; children of Collaborative Authoring if the application domain is an online collaborative writing system, e.g., Wikipedia or Github; children of Informative/Explanatory Writing if the application domain is academic writing, e.g., ArXiv papers, and software manuals; children of Journalistic/Literary Writing if the application domain is News articles or novels; and finally children of Source Code if the application domain is programming languages, like markup languages that are used for text representation, e.g. HTML and XML.

A challenge in top-layer integration is that certain sub-categories can logically fit under multiple parent categories. For instance, Claim/Ideas and Warrant/Reason/Backing may be inserted as children of either Meaning-Changed or Informative/Explanatory Writing. To ensure clarity, we assign each category to a single parent, even though this may not align with a universal consensus. Users can adjust these assignments as needed.

4.3 Adding Revision Examples

Our experience during constructing UNiT and then using it in the user studies (Section 5.1) indicate that illustrative examples of edit revisions are critical for understanding the meaning of a category. Thus, we seek to add revision examples for each category in UNiT. We follow two directions. First, we gather all relevant revision examples from the existing EITs. Second, we propose new revision examples. The latter is needed when the provided revision examples do not fully encompass the entire definition of a category (see Table 11, Appendix A.6) or a category does not have an accompanying revision example. The common practice in related work includes one revision example per category, with some cases featuring two or more. We use LLMs to generate revision examples. Specifically, we prompt the LLM by providing the category name and its definition, thereby facilitating the creation of representative revision examples. Due to the limited number of categories and samples that needed to be investigated, we employed consensus discussion to select representative revision examples.

5 Evaluation

To evaluate our UNiT, we ask two questions:
RQ1 Does the use of UNiT result in better annotator agreement compared to other EITs?
RQ2 Does UNiT perform consistently across datasets from varied domains?

5.1 Evaluation Setups

Evaluation Task We evaluate UNiT via an edit intention classification task, where a given text pair (original and revised) and an EIT serve as input to
<table>
  <tr>
    <th>Edit [Spa22]</th>
    <td><i>Definition:</i> Substantially similar except for syntactic changes, rephrased and minimally changed or updated information; merged or split sentences without substantial changes</td>
    <th>Sentence Split [Lab23]</th>
    <td><i>Definition:</i> An edit that leads to a single sentence being divided into two or more shorter sentences. In order for the split to be fluent,…</td>
  </tr>
  <tr>
    <th>Information Modification/Insertion [Ant20]</th>
    <td><i>Example:</i> …it makes [them]→[your parents] happy to know you’re getting educated.</td>
    <th>Anaphora Resolution [Lab23]</th>
    <td><i>Definition:</i> An edit that replaces the repeated or implicit mention of an entity – typically a pronoun – with a resolved mention of the entity (i.e., that doesn’t require prior context)</td>
  </tr>
  <tr>
    <th>Refactoring [Yan17]</th>
    <td><i>Definition:</i> Restructure the article; move and rewrite content, without changing the meaning of it</td>
    <th>Refactoring [Fal21]</th>
    <td><i>Definition:</i> Restructure the article; move and rewrite content, without changing the meaning of it</td>
  </tr>
  <tr>
    <th></th>
    <th></th>
    <th>Paraphrase [Dax12]</th>
    <td><i>Definition:</i> Textual edits paraphrasing words or sentences</td>
  </tr>
  <tr>
    <th></th>
    <th></th>
    <th>Refactors [Spa22]</th>
    <td><i>Definition:</i> the sentence is moved within a document</td>
  </tr>
  <tr>
    <th></th>
    <th></th>
    <th>Word-smithing [Raj22]</th>
    <td><i>Definition:</i> Rephrasing or rearranging text, improving grammar, spelling, and punctuation</td>
  </tr>
</table>

Figure 4: This figure depicts examples of one-to-one, one-to-many cases. We move category examples in Appendix A.6 for compactness. The bold parts of definitions share the same concept between matched categories.

determine the corresponding edit intention. The input granularity varies, and we focus on sentence- and paragraph-level text pairs in this study.

Datasets We use two datasets, ITERATEr-HUMAN from Du22 and ARXIVEDITS from Jia22. ITERATEr-HUMAN covers Wikipedia articles, ArXiv papers, and Wikinews and contains paragraph-level as well as sentence-level edits. ARXIVEDITS is an annotated corpus of papers from ArXiv with different granularities. We randomly sample 10 edits from ITERATEr-HUMAN and ARXIVEDITS per category per domain with mixed levels as a training set. This training set is divided into two subsets for two rounds of the training session, 155 and 100 text pairs, respectively. Then, we use the same strategy to sample 1000 text pairs from two datasets, denoted by \( D_{Du22} \) and \( D_{Jia22} \), for the evaluations.

Inter-Annotator Agreement We follow the method of inter-annotator agreement (IAA) in Du22, using the Fleiss’ \( \kappa \) (Fleiss, 1971). A higher IAA score corresponds to better agreement among annotators, suggesting that the taxonomy is easily understood and used by human annotators.

Annotators We follow the qualification criteria from Du22 to select three annotators, who then undergo training for each EIT used in our study (Du22, Rua24, and UnitT). The training covers annotation guidelines, revision examples, live demonstrations, and practice annotations. Appendix A.10 provides the instructions and examples. To ensure a thorough understanding, we conduct two rounds of practice annotations. Table 2 reports the IAA scores on the training set. After the first round, we hold a discussion to clarify annotator confusions, leading to a substantial improvement in IAA in the second round.

Comparison Groups We choose EITs in Du22 and Rua24 as our baselines. The two EITs are representative of the set of EITs. Du22 gives a general purpose EIT seeking to cover multiple domains and different edit intentions and provides detailed instruction, revision examples, and labeling interface design. Rua24 is a recent work proposing a holistic framework that includes edit intentions. Appendix A.9 introduces details of these two EITs. We conduct 3 sets of experiments:
CG1 Compare UnitT to Du22 on \( D_{Du22} \) (RQ1).
CG2 Compare UnitT to Rua22 on \( D_{Jia22} \) (RQ1).
CG3 Use UnitT on multiple datasets (RQ2).

<table>
  <tr>
    <th></th>
    <th>Dataset</th>
    <th>\( \kappa_{UNIT} \)</th>
    <th>\( \kappa_{Du22} \)</th>
    <th>\( \kappa_{Rua24} \)</th>
  </tr>
  <tr>
    <th rowspan="2">Training 1</th>
    <td>\( D_{Du22} \)</td>
    <td>0.4744</td>
    <td>0.4553</td>
    <td>/</td>
  </tr>
  <tr>
    <td>\( D_{Jia22} \)</td>
    <td>0.5483</td>
    <td>/</td>
    <td>0.3844</td>
  </tr>
  <tr>
    <th rowspan="2">Training 2</th>
    <td>\( D_{Du22} \)</td>
    <td>0.5422</td>
    <td>0.4801</td>
    <td>/</td>
  </tr>
  <tr>
    <td>\( D_{Jia22} \)</td>
    <td>0.5590</td>
    <td>/</td>
    <td>0.5107</td>
  </tr>
</table>

Table 2: Inter-Annotator agreement (Fleiss’ \( \kappa \)) across two rounds of training annotations using the EIT in Du22(\( \kappa_{Du22} \)), Rua24(\( \kappa_{Rua24} \)) and UnitT(\( \kappa_{UNIT} \)).

5.2 Evaluation Results

Table 3 shows the IAA of manual annotations.

RQ1 We use the IAA scores for each of the three taxonomies (\( \kappa_{UNIT} \), \( \kappa_{Du22} \) and \( \kappa_{Rua24} \)) to assess UnitT’s improvement gain compared to Du22 and Rua24. For CG1, UnitT achieves a higher agreement (\( \kappa_{UNIT} = 0.4007 \)) compared to Du22’s EIT
<table>
  <tr>
    <th>Dataset</th>
    <th>\( \kappa_{\text{UNIT}} \)</th>
    <th>\( \kappa_{D_{Du22}} \)</th>
    <th>\( \kappa_{D_{Jia22}} \)</th>
  </tr>
  <tr>
    <td>\( D_{Du22} \)</td>
    <td>0.4007</td>
    <td>0.3695</td>
    <td>/</td>
  </tr>
  <tr>
    <td>\( D_{Jia22} \)</td>
    <td>0.4139</td>
    <td>/</td>
    <td>0.2885</td>
  </tr>
</table>

Table 3: The IAA scores are denoted as follows: \( \kappa_{D_{Du22}} \) for Du22’s EIT, \( \kappa_{Rua24} \) for Rua24’s EIT, and \( \kappa_{\text{UNIT}} \) for UNIT. A ‘/’ symbol indicates not applicable.

(\( \kappa_{Du22} = 0.3695 \)). This indicates that UNIT provides clearer definitions or categories that better align with annotators’ interpretations compared to the EIT specifically designed for this dataset. For CG2, UNIT outperforms Rua24’s EIT, with \( \kappa_{\text{UNIT}} = 0.4139 \) versus \( \kappa_{Rua24} = 0.2885 \). The performance difference suggests that UNIT better generalizes to human annotation tasks, even on datasets from different sources, as discussed in RQ2.

RQ2 For CG3, the performance consistency of UNIT is evaluated by analyzing its performance on datasets from diverse and third-party sources \( D_{Du22} \) and \( D_{Jia22} \), which our work and Rua24 do not release. Consistency is assessed by examining the stability and competitiveness of \( \kappa_{\text{UNIT}} \) in comparison to other EITs. UNIT achieves \( \kappa_{\text{UNIT}} = 0.4007 \) on \( D_{Du22} \) and \( \kappa_{\text{UNIT}} = 0.4139 \) on \( D_{Jia22} \). On both two datasets, UNIT continues to outperform competing.

6 Conclusion

This work addresses the need for a comprehensive and unified EIT across multiple domains by integrating existing EITs into a large, cohesive framework. Our research resolves the inconsistencies and ambiguities inherent in previous works. The proposed EIT, UNIT, not only spans diverse application domains but also considers various analytical aspects, making it highly applicable for human annotators. Evaluation results demonstrate that the integrated taxonomy is more applicable to human annotators by comparing IAA scores among annotators with different EITs. Future research should focus on applying the UNIT to new application domains. We believe its extensibility and adaptability make it a robust tool for future research.

7 Limitations

Comprehensiveness of Evaluation The evaluation is not comprehensive, as the dataset is limited to already explored domains. We lack edits from new domains, such as software documentation or legal documents, which are necessary to test the generalizability of our UNIT. Future work will involve applying UNIT to new application domains, including the exploration of edit revisions in software documentation, legal documents, and more.

Agreement scores are subject to sampling, data, and user background. In this study, our training and evaluation datasets are sampled from ITERATER-HUMAN and ARXIVEDITS. The slight variation in agreement across training sessions and comparison groups may stem from sampling randomness and dataset size differences.

Further Refinement and Expansion We believe UNIT mitigates issues of overlapping definitions by complementing category names and definitions with examples to improve clarity. While we work diligently to construct UNIT, we encourage the community to refine and expand it rather than treating it as a definitive version. UNIT is not a finalized product—taxonomies rarely are.

Future Work To ensure consistency, we evaluate UNIT using the same practices reported in the surveyed studies on existing EITs. The study its hierarchical organization is challenging, as it requires masking the intent of evaluating user experiences across taxonomies of different depths. We leave this for future work.

Ethical Considerations The data used to evaluate the taxonomy is from publicly available datasets released by Du22 and Jia22. We respect the copyrights of the original document authors and the dataset creators. During the data annotation process, our human annotators are volunteers and are anonymized to protect their privacy rights. We ensure a reasonable annotation time to avoid excessive workload. Our work poses no potential harm to marginalized or vulnerable populations.

8 Acknowledgements

This work was supported by the National Science Foundation awards 2026513 and 2230692 (subaward to Temple University). We also thank our reviewers for their feedback and comments.

References

Talita Anthonio, Irshad Bhat, and Michael Roth. 2020. wikiHowToImprove: A resource and analyses on edits in instructional texts. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 5721–5729, Marseille, France. European Language Resources Association.
Jonas Nygaard Blom and Kenneth Reinecke Hansen. 2015. Click bait: Forward-reference as lure in online news headlines. Journal of Pragmatics, 76:87–100.

Danielle Brown and Vinicio Sinta. 2016. Six things you didn’t know about headline writing: Sensational form in viral news of traditional and digitally native news organizations. Research Journal of ISOJ, 6.

Zhijia Chen, Lihong He, Arjun Mukherjee, and Eduard Dragut. 2024. Comquest: Large scale user comment crawling and integration. In Companion of the 2024 International Conference on Management of Data, SIGMOD ’24, page 432–435, New York, NY, USA. Association for Computing Machinery.

Laura Chiticariu, Mauricio Hernández, Phokion Kolaitis, and Lucian Popa. 2007. Semi-automatic schema integration in clio. pages 1326–1329.

Johannes Daxenberger and Iryna Gurevych. 2012. A corpus-based study of edit categories in featured and non-featured Wikipedia articles. In Proceedings of COLING 2012, pages 711–726, Mumbai, India. The COLING 2012 Organizing Committee.

Johannes Daxenberger and Iryna Gurevych. 2013. Automatically classifying edit categories in Wikipedia revisions. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 578–589, Seattle, Washington, USA. Association for Computational Linguistics.

Rod Dillnutt. 2004. The role of taxonomy in knowledge management. The International Journal of Knowledge, Culture, and Change Management: Annual Review, 3.

Hong-Hai Do and Erhard Rahm. 2002. Coma: a system for flexible combination of schema matching approaches. In Proceedings of the 28th International Conference on Very Large Data Bases, VLDB ’02, page 610–621. VLDB Endowment.

E. Dragut, Wensheng Wu, P. Sistla, C. Yu, and Weiyi Meng. 2006a. Merging source query interfaces on-web databases. In 22nd International Conference on Data Engineering (ICDE’06), pages 46–46.

E.C. Dragut, W. Meng, and C. Yu. 2022. Deep Web Query Interface Understanding and Integration. Synthesis Lectures on Data Management. Springer International Publishing.

Eduard Dragut, Fang Fang, Prasad Sistla, Clement Yu, and Weiyi Meng. 2009a. Stop word and related problems in web interface integration. Proc. VLDB Endow., 2(1):349–360.

Eduard C. Dragut, Fang Fang, Clement Yu, and Weiyi Meng. 2009b. Deriving customized integrated web query interfaces. In 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, volume 1, pages 685–688.

Eduard C. Dragut, Thomas Kabisch, Clement Yu, and Ulf Leser. 2009c. A hierarchical approach to model web query interfaces for web source integration. Proc. VLDB Endow., 2(1):325–336.

Eduard C. Dragut, Clement Yu, and Weiyi Meng. 2006b. Meaningful labeling of integrated query interfaces. In Proceedings of the 32nd International Conference on Very Large Data Bases, VLDB ’06, page 679–690. VLDB Endowment.

Wanyu Du, Vipul Raheja, Dhruv Kumar, Jae Myung Kim, Melissa Lopez, and Dongyeop Kang. 2022. Understanding iterative revision from human-written text. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3573–3590, Dublin, Ireland. Association for Computational Linguistics.

Lester Faigley and Stephen Witte. 1981. Analyzing revision. College Composition and Communication, 32(4):400–414.

Felix Faltings, Michel Galley, Gerold Hintz, Chris Brockett, Chris Quirk, Jianfeng Gao, and Bill Dolan. 2021. Text editing by command. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5259–5274, Online. Association for Computational Linguistics.

Manaal Faruqui, Ellie Pavlick, Ian Tenney, and Dipanjan Das. 2018. WikiAtomicEdits: A multilingual corpus of Wikipedia edits for modeling language and discourse. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 305–315, Brussels, Belgium. Association for Computational Linguistics.

Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychol. Bull., 76(5):378–382.

Xingzhi Guo, Brian Kondracki, Nick Nikiforakis, and Steven Skiena. 2022. Verba volant, scripta volant: Understanding post-publication title changes in news outlets. In Proceedings of the ACM Web Conference 2022, WWW ’22, page 588–598, New York, NY, USA. Association for Computing Machinery.

Robert A Harris. 2017. Writing with clarity and style, 2 edition. Routledge, Second edition. | New York, NY : Routledge, [2018] | Previous edition published in 2003.

Bin He, Kevin Chen-Chuan Chang, and Jiawei Han. 2004. Discovering complex matchings across web query interfaces: a correlation mining approach. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’04, page 148–157, New York, NY, USA. Association for Computing Machinery.

Chao Jiang, Wei Xu, and Samuel Stevens. 2022. arXivEdits: Understanding the human revision process
in scientific writing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9420–9435, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

John Jones. 2008. Patterns of revision in online writing: A study of wikipedia’s featured articles. Written Communication, 25(2):262–289.

Omid Kashefi, Tazin Afrin, Meghan Dale, Christopher Olshefski, Amanda Godley, Diane Litman, and Rebecca Hwa. 2022. ArgRewrite v2: an annotated argumentative revisions corpus. Lang. Resour. Eval., 56(3):881–915.

Philippe Laban, Jesse Vig, Wojciech Kryscinski, Shafiq Joty, Caiming Xiong, and Chien-Sheng Wu. 2023. SWiPE: A dataset for document-level simplification of Wikipedia pages. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10674–10695, Toronto, Canada. Association for Computational Linguistics.

Seonah Lee, Rongxin Wu, Shing-Chi Cheung, and Sung-won Kang. 2021. Automatic detection and update suggestion for outdated api names in documentation. IEEE Transactions on Software Engineering, 47(4):653–675.

Xin Li, Jia Zhou, Honglian Xiang, and Jingjing Cao and. 2024. Attention grabbing through forward reference: An erp study on clickbait and top news stories. International Journal of Human–Computer Interaction, 40(11):3014–3029.

Jun Liu and Sudha Ram. 2011. Who does what: Collaboration patterns in the wikipedia and their impact on article quality. ACM Trans. Manage. Inf. Syst., 2(2).

Ulrike Pfeil, Panayiotis Zaphiris, and Chee Siang Ang. 2006. Cultural Differences in Collaborative Authoring of Wikipedia. Journal of Computer-Mediated Communication, 12(1):88–113.

Dheeraj Rajagopal, Xuchao Zhang, Michael Gamon, Sujay Kumar Jauhar, Diyi Yang, and Eduard Hovy. 2022. One document, many revisions: A dataset for classification and description of edit intents. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5517–5524, Marseille, France. European Language Resources Association.

Dietrich Rathjens. 1985. The seven components of clarity in technical writing. IEEE Transactions on Professional Communication, PC-28(4):42–46.

Qian Ruan, Ilia Kuznetsov, and Iryna Gurevych. 2024. Re3: A holistic framework and dataset for modeling collaborative document revision. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4635–4655, Bangkok, Thailand. Association for Computational Linguistics.

Darsh J. Shah, Tal Schuster, and Regina Barzilay. 2019. Automatic fact-guided sentence modification. In AAAI Conference on Artificial Intelligence.

Alexander Spangher, Xiang Ren, Jonathan May, and Nanyun Peng. 2022. NewsEdits: A news article revision dataset and a novel document-level reasoning challenge. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 127–157, Seattle, United States. Association for Computational Linguistics.

Srikrishnan Sundaresan and G. Hu. 2005. Schema integration of distributed databases using hyper-graph data model. In IRI -2005 IEEE International Conference on Information Reuse and Integration, Conf. 2005., pages 548–553.

Lei Wang and Eduard Dragut. 2024. The overlooked repetitive lengthening form in sentiment analysis. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16225–16238, Miami, Florida, USA. Association for Computational Linguistics.

Wensheng Wu, Clement Yu, AnHai Doan, and Weiyi Meng. 2004. An interactive clustering-based approach to integrating source query interfaces on the deep web. In Proceedings of the 2004 ACM SIGMOD International Conference on Management of Data, SIGMOD ’04, page 95–106, New York, NY, USA. Association for Computing Machinery.

Diyi Yang, Aaron Halfaker, Robert Kraut, and Eduard Hovy. 2016. Who did what: Editor role identification in wikipedia. Proceedings of the International AAAI Conference on Web and Social Media, 10(1):446–455.

Diyi Yang, Aaron Halfaker, Robert Kraut, and Eduard Hovy. 2017. Identifying semantic edit intentions from revisions in Wikipedia. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2000–2010, Copenhagen, Denmark. Association for Computational Linguistics.

Fan Zhang, Homa B. Hashemi, Rebecca Hwa, and Diane Litman. 2017. A corpus of annotated revisions for studying argumentative writing. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1568–1578, Vancouver, Canada. Association for Computational Linguistics.

Fan Zhang, Rebecca Hwa, Diane Litman, and Homa B. Hashemi. 2016. ArgRewrite: A web-based revision assistant for argumentative writings. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 37–41, San Diego, California. Association for Computational Linguistics.

Fan Zhang and Diane Litman. 2015. Annotation and classification of argumentative writing revisions. In
Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 133–143, Denver, Colorado. Association for Computational Linguistics.

Fan Zhang and Diane Litman. 2016. Using context to predict the purpose of argumentative writing revisions. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1424–1430, San Diego, California. Association for Computational Linguistics.

Qi Zhang, Zhijia Chen, Huitong Pan, Cornelia Caragea, Longin Jan Latecki, and Eduard Dragut. 2024. Scier: An entity and relation extraction dataset for datasets, methods, and tasks in scientific documents. Preprint arXiv:2410.21155.

Qi Zhang, Huitong Pan, Zhijia Chen, Longin Jan Latecki, Cornelia Caragea, and Eduard Dragut. 2025. Dynclean: Training dynamics-based label cleaning for distantly-supervised named entity recognition. Preprint, arXiv:2504.04616.

Weiyi Meng Zhijia Chen and Eduard Dragut. 2025. Comcrawler: General crawling solution for article comments. https://openproceedings.org/2025/conf/edbt/paper-297.pdf. [Accessed 27-05-2025].

A Appendix

A.1 Application Domains

Existing research investigates edit intentions across various application domains, such as article writing, news reporting, Wikipedia, and scientific papers. Wikipedia Pfeil et al. (2006) studies the impact of national cultures on Wikipedia by exploring the patterns in editors’ contributions. Jones (2008) examines revision patterns in the collaborative writing process of Wikipedia to explore the editors’ experience. Liu and Ram (2011) studies the relationship between collaboration and article quality in Wikipedia, identifying the roles of editors such starters, copy editors, and all-round contributors. Daxenberger and Gurevych (2012, 2013) compile a corpus of English Wikipedia edits, train a classifier to identify the edit intentions and study the collaborative behaviors among editors. Yang et al. (2016) explores the roles of editors in Wikipedia article revisions and examines how these roles influence article quality. Yang et al. (2017) focuses on capturing semantic edit intentions underlying syntactic edit operations in Wikipedia articles. (Faltings et al., 2021) analyzes the revisions with comments to develop an interactive text editing system that assists users in editing text according to their goals. (Rajagopal et al., 2022) models the relationship between edits and comments, offering insights into document evolution through an analysis of revision intentions. Similarly, Anthonio et al. (2020) investigates edits in instructional text on wikiHow to determine if the changes improve instructions in terms of style or clarity.

Writings Faigley and Witte (1981) examines the intentions behind revisions and the impact of edits on meaning by analyzing both student and expert writings. Rathjens (1985) provides an exploration of the concept of ‘clarity’ in technical writing. Zhang and Litman (2015, 2016) identify the revision purposes in student writings to predict potential grade improvements after revisions. Zhang et al. (2017); Kashefi et al. (2022) provide revision corpora to support natural language applications. Jiang et al. (2022) unveils the common strategies practiced by researchers for revising their papers.

News Spangher et al. (2022) analyzes the revision histories to explore the narrative and factual evolution of News articles, as well as the predictability of edit actions. Guo et al. (2022) investigates the motivation and frequency of post-publication changes to news headlines, particularly those made after the
initial release. Du et al. (2022) focuses on defining broader edit intentions across multiple domains, including news.

Others Anthonio et al. (2020) explored the edits of instructional text in wikiHow to identify whether the edits improve instructions in terms of style and correctness or necessary information for clarity.

A.2 Data Sources and Links

We survey the availability of publicly released data such EITs, annotated text, and guidelines in this section. We gather the sources of data that researchers analyze to explore edit revisions, as detailed in Table 4. Additionally, we collect the links of source code for re-generating data, re-training classifiers, and datasets with annotated labels, contributing to the analysis of edit intentions, as presented in Table 5.

Daxenberger and Gurevych (2012) released their annotated data with a complete annotation guideline, but the link in the follow-up work Daxenberger and Gurevych (2013) is no longer valid. Zhang et al. (2016) deployed a rewrite assistant website and released a revision component in Java. Yang et al. (2017) released the code for re-extracting the data from Wikipedia article revisions, along with a multi-label classifier to predict edit intentions. Zhang et al. (2017) released a small revision corpus with sentence-level edits in student papers and manually developed annotations. Faruqui et al. (2018) released revision corpus focusing on insertions and deletions. Kashefi et al. (2022) provides a larger revision corpus of student papers with sentential and subsentential-level edits. Anthonio et al. (2020) provided the pre-trained BiLSTM model to identify the edit intentions for text pairs, the code for re-training the classifier from scratch, and the crawler to fetch the instructional text revisions from the wikiHow website. Rajagopal et al. (2022) released data annotations along with editor comments. Spangher et al. (2022) collected news edits from 20 news agencies, processing revision from 17 of them and releasing the data publicly. Jiang et al. (2022) released datasets with different granularities of edits and provided a pipeline to do sentence-level alignment and extract fine-grained edits by inputting aligned paragraph pairs. Du et al. (2022) released both human-annotated and automatic-annotated datasets, as well as the code for crawling and preprocessing data for model training and testing. Guo et al. (2022) released their news headline dataset, though the data lacks the annotation with their taxonomy. Laban et al. (2023) released document-level revisions for the natural language simplification task and provided fine-trained language models to automatically label edits. Ruan et al. (2024) released corpus including review, revision and response data.

A.3 Grouping Results

Tables 6 to 10 illustrate the results of grouped EITs after lineage analysis. Each column, except for the leftmost one, represents an existing EIT. Each row presents matched categories, conveying a similar edit intention. Entries in a row may contain one or more categories. A single category may appear in different entries due to one-to-many category matches. The leftmost column of each table is a node proposed to tentatively represent the grouped categories for the following integration process.

A.4 Category Name Assignment

Another task in the EIT integration is to assign names to the categories/nodes of UNiT. Good category name helps user to understand the edit intention and also for their applications (Dragut et al., 2006b). We follow three strategies, selecting an existing category name, slightly modifying an existing category name, or proposing a new name. For example, in the cluster Refactoring, Word-Smithing, Paraphrase, and Word-usage/Clarity, we designate Word-Smithing as the cluster name. In general, we prioritize longer and more descriptive names. However, when a name is consistently used across EITs and conveys a precise, narrow meaning, we opt for shorter, single-word names. In some cases, we make slight edits to candidate names, such as modifying Syntactic Generic to Syntactic Changes. Finally, we introduce new names, particularly for top-layer nodes, which lack corresponding categories in the original EITs. Examples of new names include Source Code and Collaborative Authoring. Figure 5, which illustrates UNiT, uses a coding scheme to indicate the procedures followed for naming.

A.5 Category Definition

We follow a similar approach for assigning definitions to categories. Our first attempt is find a representative definition from the existing ones for a category. In some clusters, we create a definition by modifying an existing one or combining text from multiple definitions. For example, the
<table>
  <tr>
    <th>paper</th>
    <th>code?</th>
    <th>data?</th>
    <th>sources</th>
    <th>link?</th>
  </tr>
  <tr>
    <td>Fai81</td>
    <td>No</td>
    <td>No</td>
    <td>(In)experienced student and expert revisions</td>
    <td>No</td>
  </tr>
  <tr>
    <td>Zha15</td>
    <td>No</td>
    <td>No</td>
    <td>Student written papers</td>
    <td>No</td>
  </tr>
  <tr>
    <td>Zha16(A)</td>
    <td>No</td>
    <td>No</td>
    <td>Student writings</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Zha17</td>
    <td>No</td>
    <td>Ye</td>
    <td>Student writings</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Fal21</td>
    <td>Yes</td>
    <td>Ye</td>
    <td>Wikipedia revision histories</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Du22</td>
    <td>Yes</td>
    <td>Ye</td>
    <td>Formally human-written text(Wikipedia, ArXiv, Wikinews)</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Jia22</td>
    <td>Yes</td>
    <td>Ye</td>
    <td>ArXiv Papers Revisions</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Kas22</td>
    <td>/</td>
    <td>Ye</td>
    <td>argumentative writing essays</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Rua24</td>
    <td>Yes</td>
    <td>Ye</td>
    <td>F1000RD dataset in Kuznetsov et al., 2022, ARR-22 subset of the NLPeer corpus in Dycce et al., 2023</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Rat85</td>
    <td>/</td>
    <td>/</td>
    <td>/</td>
    <td>No</td>
  </tr>
  <tr>
    <td>Zha16(B)</td>
    <td>No</td>
    <td>No</td>
    <td>High school student written papers</td>
    <td>No</td>
  </tr>
  <tr>
    <td>Yan16</td>
    <td>No</td>
    <td>No</td>
    <td>Three datasets from English edition of Wikipedian</td>
    <td>No</td>
  </tr>
  <tr>
    <td>Jon08</td>
    <td>No</td>
    <td>No</td>
    <td>(Non-)featured Wikipedia articles revision histories</td>
    <td>No</td>
  </tr>
  <tr>
    <td>Dax12</td>
    <td>No</td>
    <td>Ye</td>
    <td>(Non-)featured Wikipedia articles revision histories</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Dax13</td>
    <td>No</td>
    <td>Ye</td>
    <td>Wikipedia revision histories</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Yan17</td>
    <td>Yes</td>
    <td>Ye</td>
    <td>Wikipedia revision histories</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Lab23</td>
    <td>Yes</td>
    <td>Ye</td>
    <td>Wikipedia’s revision history</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Pfe06</td>
    <td>No</td>
    <td>No</td>
    <td>Wikipedia page revision histories</td>
    <td>No</td>
  </tr>
  <tr>
    <td>Liu11</td>
    <td>No</td>
    <td>No</td>
    <td>Different quality-level Wikipedia articles based on quality</td>
    <td>No</td>
  </tr>
  <tr>
    <td>Far18</td>
    <td>No</td>
    <td>Yes</td>
    <td>revision histories</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Raj22</td>
    <td>No</td>
    <td>Yes</td>
    <td>Wikipedia revision histories</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Ant20</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>wikiHow (non-)featured articles revision histories</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Spa22</td>
    <td>Yes</td>
    <td>Yes</td>
    <td>News revision histories</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Guo22</td>
    <td>No</td>
    <td>Yes</td>
    <td>News headlines from major US news agencies</td>
    <td>Yes</td>
  </tr>
</table>

Table 4: This table summarizes whether the literature provided their code and data. code? indicates if the code for preprocessing or regenerating data was released. data? shows if the data was provided along with their taxonomy. sources specifies the sources of the revisions their taxonomy and analysis are based on. link? indicates if the links to their code or data were provided. '/' denotes not applicable.

definition of Forward Reference (Figure 5) is modified from “a reference to upcoming discourse or to a word or a phrase later in the text” (Li et al., 2024). When the existing definitions are specific to the methodology/application domain of a particular paper, we generalize them to ensure broader applicability. We create new definitions to a few categories, such as “Edits for all online collaborative writing: Wikipedia, Github, online forums, etc.” for the top-layer category Collaborative Authoring. We use the same coloring scheme as in Figure 5 to indicate the definition assignment procedure.

A.6 Revision Examples

Table 11 illustrates the revision examples of categories in Figure 4.

A.7 Existing EITs Matching and Comparison

Table 12 presents a structured comparison of edit intention categories across different studies, including their definitions and representative revision examples. Due to space constraints, we illustrate only a subset of this comparison—specifically, the categories related to Information Modification and Reference Changes. The full comparison table is available at the provided link.

Information Modification. The edit intentions grouped under Information Modification share a common purpose: to alter or update informational content. This includes changes to facts, theorems, or supporting evidence (Zha15/Zha16/Kas22), as well as factual data updates (Fal21/Raj22). Dax12 introduces a broader category for content-affecting edits, while Du22 emphasizes the addition of new information, bridging factual revision and meaning shifts. The bolded phrases in their definitions highlight a shared goal of modifying factual or evidential content.

Reference Changes. The intentions categorized under Reference Changes center around modifying references or citations—whether by adding, removing, or verifying them. While Zha15/Zha16/Kas22 emphasize citation as a form of evidential support, Yan17 and Fal21 focus more specifically on source verifiability. Despite overlapping definitions, their underlying purposes differ: one supports claims, the other ensures reliability.

It is worth noting that the Evidence category (Zha15/Zha16/Kas22) spans both Information Modification and Reference Changes due to its broad definition. However, the revision examples associated with it in the Reference Changes context do not directly align with the bolded definition. This indicates the need for more representative revi-
<table>
  <tr>
    <th>alias</th>
    <th>Link</th>
    <th>Work?</th>
  </tr>
  <tr>
    <td>Zha16(A)</td>
    <td>https://people.cs.pitt.edu/~zhangfan/argrewrite/</td>
    <td>Active</td>
  </tr>
  <tr>
    <td></td>
    <td>https://www.cs.pitt.edu/~zhangfan/revisionCorrection.jar</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Zha17</td>
    <td>https://argrewrite.cs.pitt.edu/</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Fal21</td>
    <td>http://microcosm.com/research/project/interactive-document-generation</td>
    <td>broken</td>
  </tr>
  <tr>
    <td>Du22</td>
    <td>https://github.com/vipulraheja/IteraTeR</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Jia22</td>
    <td>https://github.com/chaojiang06/arXivEdits</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Kas22</td>
    <td>https://argrewrite.cs.pitt.edu/</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Rua24</td>
    <td>Code: https://github.com/UKPLab/rca<br>Dataset: https://tudatalib.tub.tu-darmstadt.de/handle/tudatalib/4300</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Dax12</td>
    <td>https://tudatalib.tub.tu-darmstadt.de/handle/tudatalib/2354</td>
    <td>Moved</td>
  </tr>
  <tr>
    <td>Dax13</td>
    <td>http://www.ukp.tu-darmstadt.de/data/edit-classification</td>
    <td>Broken</td>
  </tr>
  <tr>
    <td>Yan17</td>
    <td>https://github.com/diyi/Wiki_Semantic_Intention</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Lab23</td>
    <td>https://github.com/Salesforce/simplification</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Far18</td>
    <td>https://github.com/google-research-datasets/wiki-atomic-edits?tab=readme-ov-file</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Raj22</td>
    <td>https://tinyurl.com/editsumm</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Ant20</td>
    <td>https://github.com/irshadibhai/wikiHowToImprove</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Spa22</td>
    <td>https://github.com/si-nlp/NewsEdits.git</td>
    <td>Active</td>
  </tr>
  <tr>
    <td>Guo22</td>
    <td>https://scripta-volant.github.io/</td>
    <td>Active</td>
  </tr>
</table>

Table 5: It collects the links from the existing works which release their code or data. work? shows whether the provided link is still working or not; ‘Moved’ means the source link has changed but is still accessible. ‘Broken’ indicates the provided link is not accessible and the source cannot be found via searches.

<table>
  <tr>
    <th>Combined(DuRa)</th>
    <th>Rat85</th>
    <th>Du22</th>
  </tr>
  <tr>
    <td>Fluency</td>
    <td></td>
    <td>Fluency</td>
  </tr>
  <tr>
    <td>Coherence</td>
    <td></td>
    <td>Coherence</td>
  </tr>
  <tr>
    <td rowspan="2">Clarity</td>
    <td>Brevity, Order, Accuracy, Completeness, Emphasis, Consistency, Objectivity</td>
    <td rowspan="2">Clarity</td>
  </tr>
  <tr></tr>
  <tr>
    <td>Style</td>
    <td>Style</td>
    <td></td>
  </tr>
  <tr>
    <td>Meaning-Changed</td>
    <td></td>
    <td>Meaning-Changed</td>
  </tr>
</table>

Table 6: The result of grouping Rat85 and Du22.

<table>
  <tr>
    <th>Combined (DaYang)</th>
    <th>Dax12/Dax13</th>
    <th>Yan16</th>
  </tr>
  <tr>
    <td>Vandalism</td>
    <td>Vandalism</td>
    <td></td>
  </tr>
  <tr>
    <td>Revert</td>
    <td>Revert</td>
    <td></td>
  </tr>
  <tr>
    <td>Paragraph</td>
    <td>Paragraph</td>
    <td>Rephrase</td>
  </tr>
  <tr>
    <td>Spelling /Grammar</td>
    <td>Spelling /Grammar</td>
    <td>Grammar</td>
  </tr>
  <tr>
    <td>Relocation</td>
    <td>Relocation</td>
    <td>Relocation</td>
  </tr>
  <tr>
    <td>Markup-IDM</td>
    <td>Markup-IDM</td>
    <td>Markp-IDM</td>
  </tr>
  <tr>
    <td>Info.-IDM</td>
    <td>Info.-IDM</td>
    <td>Info.-IDM</td>
  </tr>
  <tr>
    <td>File-IDM</td>
    <td>File-IDM</td>
    <td>File-IDM</td>
  </tr>
  <tr>
    <td>Ref.-IDM</td>
    <td>Ref.-IDM</td>
    <td>External Link-IDM, Wikilink-IDM</td>
  </tr>
  <tr>
    <td>Template-IDM</td>
    <td>Template-IDM</td>
    <td>Template-IDM</td>
  </tr>
</table>

Table 7: The result of grouping Dax12, Dax13, and Yan16. Dax13 reused EIT in Dax12, so we combine two EITs into one column. Inf.-IDM and Ref.-IDM are abbreviated forms of Information-IDM and Reference-IDM, respectively.

sion examples—particularly ones that align better with the reference-focused intentions seen in Verification (Yan17) and Verifiability (Fal21).

A.8 Taxonomy Integration Result

Figure 5 presents our UNiT: the unified and comprehensive EIT. UNiT has three layers including Top-layer (helping annotators quickly locate edit intentions), Edit Intentions (the categories used in the evaluation), Fine-grained Intentions (using as needed). Categories in each layer have clear definitions and illustrative revision examples (omit in Figure 5 due to space limitations). Blocks surrounded by a solid line are categories and by a dashed line are definitions of categories. The blocks in grey blocks mean that categories/definitions are taken from the literature. Those in blue mean that categories are proposed by the authors based on matched categories, addressing cases where definitions may be missing or not general for all edit intentions. Those in purple mean that categories/definitions are modified from the literature.

Application Configuration Our UNiT can be applied across various domains. For instance, if our research focuses on analyzing the edit intentions behind PostgreSQL documentation, which falls under informative/explanatory writings, we need to filter relevant categories from our UNiT. In this scenario, we select categories under Informative/Explanatory Writing, Source Code, Non-Meaning-Changed, and Meaning-Changed. We include categories under Source Code categories because PostgreSQL documentation also incorporates SQL code to demonstrate SQL usage.

Our UNiT does not explicitly specify the unit of edit within categories, aligning with conventions in related work. Instead, we allow users to determine the input granularity.
<table>
  <tr>
    <th>Combined(zhaka)</th>
    <th>Zha15</th>
    <th>Zha16(B)</th>
    <th>Zha16(A)</th>
    <th>Zha17</th>
    <th>Kas22</th>
  </tr>
  <tr>
    <th>Organization</th>
    <td>Organization</td>
    <td>Organization</td>
    <td>Reordering (Organization)</td>
    <td>Organization (Reordering)</td>
    <td>Organization (ORG)</td>
  </tr>
  <tr>
    <th>Conventions /Grammar /Spelling</th>
    <td>Conventions /Grammar /Spelling</td>
    <td>Conventions</td>
    <td>Errors (Conventions /Grammar /Spelling)</td>
    <td>Conventions /Grammar /Spelling (Errors)</td>
    <td>Spelling and Gramma (SPL)</td>
  </tr>
  <tr>
    <th>Word-usage /Clarity</th>
    <td>Word-usage /Clarity</td>
    <td>Clarity</td>
    <td>Fluency (Word-usage /Clarity)</td>
    <td>Word Usage /Clarity (Fluency)</td>
    <td>Word Usage (WRD)</td>
  </tr>
  <tr>
    <th>Claim/Ideas</th>
    <td>Claim/Ideas</td>
    <td>Claim/Ideas</td>
    <td>Thesis/Ideas (Claim)</td>
    <td>Claims/Ideas (Ideas)</td>
    <td>Claim(CLIM)</td>
  </tr>
  <tr>
    <th>Warrant /Reasoning /Backing</th>
    <td>Warrant /Reasoning /Backing</td>
    <td>Warrant /Reasoning /Backing</td>
    <td>Reasoning (Warrant)</td>
    <td>Warrant /Reasoning /Backing (Reasoning)</td>
    <td>Reasoning (RSN)</td>
  </tr>
  <tr>
    <th>Rebuttal /Reservation</th>
    <td>Rebuttal /Reservation</td>
    <td>Rebuttal</td>
    <td>Rebuttal</td>
    <td>Rebuttal /Reservation (Rebuttal)</td>
    <td>Rebuttal (RBL)</td>
  </tr>
  <tr>
    <th>General Content</th>
    <td>General Content</td>
    <td>General Content</td>
    <td>Other content changes</td>
    <td>General Content (Other)</td>
    <td>General Content Development(GCD)</td>
  </tr>
  <tr>
    <th>Evidence</th>
    <td>Evidence</td>
    <td>Evidence</td>
    <td>Evidence</td>
    <td>Evidence</td>
    <td>Evidence (EVD)</td>
  </tr>
  <tr>
    <th>Precision</th>
    <td></td>
    <td></td>
    <td>Precision</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <th>Unknown</th>
    <td></td>
    <td></td>
    <td>Unknown</td>
    <td></td>
    <td></td>
  </tr>
</table>

Table 8: The result of grouping Zha15, Zha16(B), Zha16(A), Zha17 and Kas22

<table>
  <tr>
    <th>Combined(YFR)</th>
    <th>Yan17</th>
    <th>Fal21</th>
    <th>Raj22</th>
  </tr>
  <tr>
    <th>Clarification</th>
    <td>Clarification</td>
    <td>Clarification</td>
    <td>Adding Support Evidence</td>
  </tr>
  <tr>
    <th>Copy-editing</th>
    <td>Copy Editing</td>
    <td>Copy-editing</td>
    <td>Word-smithing</td>
  </tr>
  <tr>
    <th>Counter-vandalism</th>
    <td>Counter Vandalism</td>
    <td>Counter-vandalism</td>
    <td>Wikipedia-specific edits</td>
  </tr>
  <tr>
    <th>Disambiguation</th>
    <td>Disambiguation</td>
    <td>Disambiguation</td>
    <td>Wikipedia-specific edits</td>
  </tr>
  <tr>
    <th>Elaboration</th>
    <td>Elaboration</td>
    <td>Elaboration</td>
    <td>Adding New Information</td>
  </tr>
  <tr>
    <th>Fact Update</th>
    <td>Fact Update</td>
    <td>Fact Update</td>
    <td>Fact Update</td>
  </tr>
  <tr>
    <th>Point of View</th>
    <td>Point of View</td>
    <td>Point of View</td>
    <td>Point of View Change</td>
  </tr>
  <tr>
    <th>Process</th>
    <td>Process</td>
    <td>Process</td>
    <td>Wikipedia-specific edits</td>
  </tr>
  <tr>
    <th>Refactoring</th>
    <td>Refactoring</td>
    <td>Refactoring</td>
    <td>Word-smithing</td>
  </tr>
  <tr>
    <th>Simplification</th>
    <td>Simplification</td>
    <td>Simplification</td>
    <td>Remove existing information</td>
  </tr>
  <tr>
    <th>Vandalism</th>
    <td>Vandalism</td>
    <td>Vandalism</td>
    <td>Wikipedia-specific edits</td>
  </tr>
  <tr>
    <th>Verification</th>
    <td>Verification</td>
    <td>Verifiability</td>
    <td>Adding Support Evidence</td>
  </tr>
  <tr>
    <th>Wikification</th>
    <td>Wikification</td>
    <td>Wikification</td>
    <td>Wikipedia-specific edits</td>
  </tr>
  <tr>
    <th>Unlabeled</th>
    <td></td>
    <td>Unlabeled</td>
    <td></td>
  </tr>
</table>

Table 9: The result of grouping Yan17, Fal21 and Raj22.

<table>
  <tr>
    <th>Combined(PL)</th>
    <th>Pfe06</th>
    <th>Liu11</th>
  </tr>
  <tr>
    <th>Add Information</th>
    <td>Add Information</td>
    <td>Sentence Insertion</td>
  </tr>
  <tr>
    <th>Reference Insertion</th>
    <td>Add link</td>
    <td>Link Insertion, Reference Insertion</td>
  </tr>
  <tr>
    <th>Modify Information</th>
    <td>Clarify Information</td>
    <td>Sentence Modification</td>
  </tr>
  <tr>
    <th>Delete Information</th>
    <td>Delete Information</td>
    <td>Sentence Deletion</td>
  </tr>
  <tr>
    <th>Reference Deletion</th>
    <td>Delete Link</td>
    <td>Link Deletion, Reference Deletioin</td>
  </tr>
  <tr>
    <th>Reference Modification</th>
    <td>Fix Link</td>
    <td>Link Modification, Reference Modification</td>
  </tr>
  <tr>
    <th>Format</th>
    <td>Format</td>
    <td></td>
  </tr>
  <tr>
    <th>Grammar</th>
    <td>Grammar</td>
    <td>Sentence Modification</td>
  </tr>
  <tr>
    <th>Mark-up Language</th>
    <td>Mark-up Language</td>
    <td></td>
  </tr>
  <tr>
    <th>Reversion</th>
    <td>Reversion</td>
    <td>Revert</td>
  </tr>
  <tr>
    <th>Spelling</th>
    <td>Spelling</td>
    <td>Sentence Modification</td>
  </tr>
  <tr>
    <th>Style/Typography</th>
    <td>Style/Typography</td>
    <td></td>
  </tr>
  <tr>
    <th>Vandalism</th>
    <td>Vandalism</td>
    <td></td>
  </tr>
</table>

Table 10: The result of grouping Pfe06 and Liu11.
<table>
  <tr>
    <th>Category</th>
    <th>Definition</th>
    <th>Example</th>
  </tr>
  <tr>
    <td>Edit[Spa22]</td>
    <td>Substantially similar except for syntactic changes, rephrased and minimally changed or updated information; merged or <b>split sentences without substantial changes</b></td>
    <td>...he regrets exposing her to the deadly virus [and had] → [ Had] he known he was carrying Ebola, ...</td>
  </tr>
  <tr>
    <td>Sentence Split[Lab23]</td>
    <td>An edit that leads to a single sentence being divided into two or more shorter sentences. In order for the split to be fluent, words are typically removed and inserted at the sentence boundary. If no non-connector content is added, then it is not only a sentence split.</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Anaphora Resolution[Lab23]</td>
    <td>An edit that replaces the repeated or implicit mention of an entity – typically a pronoun – with a resolved mention of the entity (i.e., that doesn’t require prior context).</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Refactoring[Yan17]</td>
    <td>Restructure the article; move and <b>rewrite content, without changing the meaning of it</b></td>
    <td>/</td>
  </tr>
  <tr>
    <td>Refactoring[Fal21]</td>
    <td>Restructure the article; move and <b>rewrite content, without changing the meaning of it</b></td>
    <td>/</td>
  </tr>
  <tr>
    <td>Paraphrase[Dax12]</td>
    <td><b>Textual edits paraphrasing words or sentences</b></td>
    <td>denominations [like] → [such as] the</td>
  </tr>
  <tr>
    <td>Refactors[Spa22]</td>
    <td><b>the sentence is moved within a document</b></td>
    <td>"[The mother, this was the first time seeing her son since he got to the States.]" ... → "[The mother, this was the first time seeing her son since he got to the States.]"</td>
  </tr>
  <tr>
    <td>Word-smithing[Raj22]</td>
    <td>Rephrasing or rearranging text, <b>improving grammar, spelling, and punctuation</b></td>
    <td>...as well as [accommodation] → [accommodation] for visitors...</td>
  </tr>
</table>

Table 11: The categories in this table are the actual category names, definitions, and revision examples in existing Edit Intention Taxonomies (EITs). The revision examples of categories only cover the bold part of the definitions.

A.9 Compared EITs in Evaluation

Du22 introduces a comprehensive, multi-domain taxonomy to model iterative text revision processes in formal writing. Its edit intention taxonomy is divided into two broad categories:

Non-meaning-changing edits:
• Fluency: Fixing grammar and syntax issues.
• Clarity: Improving formality, conciseness, or readability.
• Coherence: Enhancing logical flow and consistency.
• Style: Reflecting personal writing preferences (tone, emotion).

Meaning-changing edits:
• Meaning-Changed: Adding or modifying factual content or evidence.

Other: Edits not fitting into the above categories.

This taxonomy supports annotations at sentence and paragraph levels across domains like scientific abstracts, Wikipedia, and news. The goal is to better understand how edits improve writing and guide models in generating high-quality revisions.

Rua24 proposes the Re3 framework, which focuses on the collaborative revision process, particularly in academic writing, integrating reviews, revisions, and author responses. It offers a full-scope annotation taxonomy along three dimensions:

Granularity: Edits are labeled at the section, paragraph, sentence, and subsentence levels.
Action: Includes Add, Delete, Modify, Merge, Split, and Fusion.

Intent:
• Surface-level: Grammar and Clarity.
• Semantic-level: Claims and Factual/Evidence changes.
• Other: Edits that don’t fall into the above.

This taxonomy enables deep analysis of scholarly editing behaviors and supports novel tasks like edit intent classification and document edit summarization using large language models.

A.10 Human Annotation Instructions and Interface

To guide human annotators in making accurate edit-intention annotations, we provide a brief task instruction in Figure A.10, followed by concrete revision examples. Due to space limitations, these revision examples are shown in the shared link. We highlight the edits (i.e., the differences between text pairs) within sentence-level revisions and pose questions to the annotators to accurately determine the edit intention, as illustrated in Figure 8. Figure 8 displays only partial questions for obtaining the edit intention; the complete set of questions is available in the shared link. Additionally, we provide
<table>
  <tr>
    <th>UNIT Label</th>
    <th>Category</th>
    <th>Definition</th>
    <th>Revision Examples</th>
  </tr>
  <tr>
    <td rowspan="5">Information Modification</td>
    <td>Information-Modify[Dax12]</td>
    <td>Textual edits affecting information content</td>
    <td>open steppes of Kashmir and [Siberia → Manchuria].</td>
  </tr>
  <tr>
    <td>Evidence[Zha15]</td>
    <td>change of facts, theorems or citations for supporting claims/ideas</td>
    <td>In the circle I would place Fidel. He was annoyed with the existence of the United States and used his army to force them out of his country. Although Fidel claimed that this is for his peoples’ interest, it could not change the fact that he is a wrathful person.</td>
  </tr>
  <tr>
    <td>Evidence[Zha16(B)]</td>
    <td>change of facts, theorems or citations for supporting claims/ideas</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Evidence[Kas22]</td>
    <td>change of facts, theorems or citations for supporting claims/ideas</td>
    <td>An example for the case where the electronic communication is limited would be [China → North Korea].</td>
  </tr>
  <tr>
    <td>Fact Update[Fal21]</td>
    <td>Update numbers, dates, scores, episodes, status, etc. based on newly available information</td>
    <td>He married Margaret Frances Prowse Shaw in Sydney in [1874 → 1871].</td>
  </tr>
  <tr>
    <td>Fact Update[Raj22]</td>
    <td>Updating facts in document</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Meaning-Changed[Du22]</td>
    <td>Update or add new information to the text</td>
    <td>This method improves the model accuracy from 64% to {78→83}%.</td>
  </tr>
  <tr>
    <td rowspan="5">Reference Changes</td>
    <td>Evidence[Zha15]</td>
    <td>change of facts, theorems or citations for supporting claims/ideas</td>
    <td>In the circle I would place Fidel. He was annoyed with the existence of the United States and used his army to force them out of his country. Although Fidel claimed that this is for his peoples’ interest, it could not change the fact that he is a wrathful person.</td>
  </tr>
  <tr>
    <td>Evidence[Zha16(B)]</td>
    <td>change of facts, theorems or citations for supporting claims/ideas</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Evidence[Kas22]</td>
    <td>change of facts, theorems or citations for supporting claims/ideas</td>
    <td>An example for the case where the electronic communication is limited would be [China → North Korea].</td>
  </tr>
  <tr>
    <td>Verification[Yan17]</td>
    <td>Add/modify references/citations: remove unverified text</td>
    <td>/</td>
  </tr>
  <tr>
    <td>Verifiability[Fal21]</td>
    <td>Add/modify references/citations: remove unverified text</td>
    <td>/</td>
  </tr>
</table>

Table 12: Partial comparison of labels related to Information Modification and Reference Changes across different EITs. Bolded phrases in the definitions highlight semantically aligned concepts across studies. The table also includes example revisions and comments on subtle differences in scope or intent.

the context of the edit (the document revision) to better ascertain the edit intention, as some edits require contextual understanding.
Figure 5: UNIT, a unified, comprehensive, large, extensible edit-intention taxonomy by integrating all EITs.
Instructions
The goal of this project is to identify editors’ intentions when revising texts. Your task is to review a single revision within a paragraph and label the intention behind it.

Pay attention to the following tips when annotating the revision:

Review the examples (below) and label definitions (click the “Instructions” button).

• Read the whole text and consider the context. Revision intentions will be clearer if you know the full context. This includes the full paragraph and the other revisions.
• See the domain of the revision coming from, then locate from 4 domains: Collaborative Authoring, Information/Explanatory Writing, Journalistic/Literary Writing and Source code. The table Application Domain below shows the definition, examples.
• If the revision matches multiple domains, such as the revisions from Wikihow can match to Collaborative Authoring and Informative/Explanatory Writing, you need to select a label from categories under Collaborative Authoring and Informative/Explanatory Writing. The table below shows the definition, explanation and examples. You can also click under “domain”.
• If no category under that four domain matches to the revision, then you need to determine whether the revision alters the summary of text. If yes, selecting a category under meaning-changed. We only pick the next-level category of non-meaning-changed category.
• If not, selecting a category under non-meaning-changed.
• If you still cannot find a category matching to the revision, check the Other.
• If you think multiple intentions exist in one revision, select the additional labels in the following checkbox.

Figure 6: A screenshot of the annotation instruction for human annotators.

<table>
  <tr>
    <th>Domain</th>
    <th>Definition</th>
    <th>Examples</th>
  </tr>
  <tr>
    <td>Collaborative Authoring</td>
    <td>Edits for all online collaborative writing: Wikipedia, Github, online forums, etc.</td>
    <td>Wikipedia, Wikihow, etc.</td>
  </tr>
  <tr>
    <td>Informative/Explanatory Writing</td>
    <td>Edits for academic articles, research papers, how-to guides, manuals, reports, etc.</td>
    <td>Wikihow, ArXiv papers, Wikipedia articles, etc.</td>
  </tr>
  <tr>
    <td>Journalistic/Literary Writing</td>
    <td>Edits for news, events, novels, short stories, etc.</td>
    <td>News, WikiNews, etc.</td>
  </tr>
  <tr>
    <td>Source Code</td>
    <td>Edits for source code: HTML, XML, etc.</td>
    <td>HTML, Wikipedia markup languages, etc.</td>
  </tr>
</table>

Figure 7: A screenshot of the Application Domain table in the instruction.

![Screenshot of the annotation interface with partial questions for human annotators.](page_120_1092_1247_384.png)

Figure 8: A screenshot of the annotation interface with partial questions for human annotators.

Reviewer paper summaries: [Partly taken from my previous review] This paper is concerned with the task of classifying edits in textual revisions by intent, such as “syntactic changes” or “information insertion”. The authors survey existing edit intention taxonomies and propose a new comprehensive taxonomy that tries to integrate all existing taxonomies. To validate the new taxonomy, the authors conduct an annotation study on edits from two datasets, and compare the agreement obtained with their taxonomy to the agreement obtained with a taxonomy from related work. The new taxonomy results in better inter annotator agreement.

The paper introduces UNIT, a comprehensive and unified taxonomy designed to address the fragmentation in edit intention taxonomies across multiple application domains. The authors analyze and synthesize 24 existing EITs, which total 232 categories, to create a taxonomy that aims to simplify the process of analyzing revisions and understanding their intentions, thereby enhancing the usability and efficiency of text revision processes in various domains.

The paper introduces a comprehensive and unified taxonomy for revision intentions in text documents as a solution to existing taxonomies. The authors conduct an extensive literature review and identify several key challenges and sparsity in existing taxonomies. UniT is proposed after an extensive lineage relationship and merging process.
Summary of strengths: [Partly taken from my previous review]

- Text revision is an important research area. Most texts are the result of repeated revisions, but this is hidden in the final text. Making progress in text revision analysis can help to improve text generation as well as automated  tutoring.
- The attempt to unify existing edit intent taxonomies into a single taxonomy is laudable, as this can increase the comparability of future studies using the same taxonomy.
- The inheritance analysis of existing taxonomies (Figure 1) is interesting.
- The authors validate their taxonomy in a human annotation study (Section 5). They show that inter-annotator agreement is improved compared to a taxonomy from previous work, even though the newly proposed taxonomy has more categories. This empirical part strongly improved compared to the previous version. Now, the authors first show the results of training annotators on the compared edit intention taxonomies. Then, they evaluate the compared taxonomies on two datasets (previously 1) and 1000 edits (previously 100).

1. The paper successfully integrates a diverse range of existing EITs, covering 232 categories from 24 different sources. This comprehensive approach ensures that UNIT is robust and versatile, capable of addressing various needs across multiple domains without the need to develop new EITs for specific cases.
2. The authors demonstrate that UNIT achieves higher inter-annotator agreement scores compared to existing EITs, suggesting that it provides clearer and more consistent guidelines for annotators, which can significantly improve the reliability of text annotation and revision tasks.
3. The evaluation of UNIT includes empirical studies that compare its performance with other EITs across different datasets. This not only underscores its practical applicability but also highlights its effectiveness in real-world scenarios, making it a valuable tool for both researchers and practitioners.

- The paper addresses an interesting and important problem, often overlooked in empirical NLP works. A unified taxonomy has the potential to set a standard for future works in this area.
- The paper provides valuable insights in studying and merging several, sparse pieces of edit intention taxonomies. The motivation behind the contribution is very well justified through extensive literature survey
- The inherently iterative nature of UniT can be helpful in maintaining the taxonomy in the future
- The proposed taxonomy is validated through an annotation study achieving better scores with 3 annotators.
- The authors have addressed major concerns raised in the previous cycle.
Summary of weaknesses: A proper discussion of the task is still missing. This would also inform the merging of different taxonomies.

1. The authors have agreed to include some of the points below into the next version / camera ready. 

- What is input and output? Give a task definition.
- How can the input change in terms of granularity (e.g. span, sentence, paragraph)? How does this affect the labels?
- Edit intent labels can describe *what* changes (e.g. “Claim/Ideas”) or *how* it changes (”Information insertion”), and maybe also other things. How does this affect the labels?

2. After the author discussion, I have a better understanding of the reasons for the different categories existing next to each other in the integrated taxonomy. Several labels can't be subsumed under meaning-changed / non-meaning changed. A good use of the taxonomy would therefore probably allow applying multiple labels to one intent. 

Still, I think more efforts could have been made to integrate more of the categories into a single branch (see my original review for this version of the paper).

3. The sharp drop between agreement on train and test is still somewhat worrying.

1. While the paper aims to standardize definitions, some categories may still suffer from vague or overlapping definitions, which could affect the accuracy and consistency of their application.
2. The validation of the multi-level hierarchical structure of UNIT is not extensively discussed, leaving questions about the optimal levels of granularity and their practical benefits.
3. The data sources used for developing and validating UNIT, primarily academic and Wikipedia edits, might introduce a bias that affects the taxonomy's effectiveness in other types of text or edits not represented in the training data.

- The writing still needs a lot of work. I would suggest the authors to revise the paper and focus on more clarity and less grammatical mistakes in the text.
- The paper overall lacks significant technical novelty.
- The contribution is specific and niche, which might not be very useful for a wider audience.
- Some revision examples are generated using LLMs as mentioned in L493. These generations should be evaluated first before adding them to the taxonomy.
Comments, suggestions and typos: General

- The text definitely improved in terms of clarity. Similarly, the empirical evaluation improved. After the author discussion, I have raised my score to 3.5.
- “Revision” vs “edit”:
    - The abstract says that “one revision may contain many edits”. The first introduction paragraph uses both the words “revision” and “edit” and does not distinguish clearly between the terms.
    - L175ff only talks about “revisions”.
    - L251ff only talks about “revision”.
- L091: What is “EIT matching”? It becomes clear later but is vague here.
- L579: Why is the agreement on DJia22 based on two annotators? I did not understand this. 
- Tables 2 and 3: Why did the agreement numbers decrease so much from training (Table 2) to evaluation (Table 3)? I understand that the training dataset is a mixture of DJia22 and Du22? 

Typos

- L112: “Forth” → Fourth
- L167-168: “all surveys all the papers” - seems like either the papers or the surveys should be deleted?

Nil.

L029: “a important” -> “an important”

L112: “Forth is coping…” -> i think it should be “Fourth”?

L138: “...leads to a lineage relationships” -> “leads to a lineage relationship”

L168: grammatical error

L190: “subdivides” -> “subdivide”

L370: grammatical error
Overall assessment scores: 3.5

3

3.5