Introducing Rhetorical Parallelism Detection: A New Task with Datasets, Metrics, and Baselines (2023)

Abstract: Rhetoric, both spoken and written, involves not only content but also style. One common stylistic tool is \textit{parallelism} : the juxtaposition of phrases which have the same sequence of linguistic ( \textit{e.g.} , phonological, syntactic, semantic) features. Despite the ubiquity of parallelism, the field of natural language processing has seldom investigated it, missing a chance to better understand the nature of the structure, meaning, and intent that humans convey. To address this, we introduce the task of \textit{rhetorical parallelism detection} . We construct a formal definition of it; we provide one new Latin dataset and one adapted Chinese dataset for it; we establish a family of metrics to evaluate performance on it; and, lastly, we create baseline systems and novel sequence labeling schemes to capture it. On our strictest metric, we attain F _1 scores of 0.40 and 0.43 on our Latin and Chinese datasets, respectively.

## introduction ueni, uidi, uici, 1 or, "i came, i saw, i conquered" -why is this saying so memorable? one reason is the high degree of parallelism it exhibits in its morphology (each verb is first-person, singular, perfect tense), phonology (each verb starts with /w/ and ends with /i:/), and prosody (each verb has two long syllables with accent on the first). in similar sayings (as in fig. 1 ), syntax and semantics also contribute. the related elements generally occur in either the same order or in an inverted order. parallelism is organically employed in many argumentative structures (e.g., "on the one hand . . . on the other hand . . ."), making parallelism a routine rhetorical figure. the rhetorical impact of 1 popularly attributed to julius caesar @xcite . in classical latin, u and v were graphical variants of one letter; as a consonant, it was pronounced /w/. following common practice, we write it as u. we also write i and j as i. these structures is apparent even for audiences not schooled in classical rhetoric. recognition of parallelism, then, is important for grasping the structure, meaning, and intent that humans wish to convey; thus, the computational modeling of parallelism is both an interesting and practical problem. however, because parallelism can occur at so many levels linguistically-often with no lexical overlap-modeling parallelism computationally is difficult. as a first foray into studying the problem of rhetorical parallelism detection (rpd) and enabling others to study it, we present in this paper multiple public datasets, metrics, and models for it. for one dataset, we turn to the latin sermons of augustine of hippo (354-430). augustine had been trained as a rhetorician and had taught the craft of secular rhetoric before his conversion to christianity. however, upon becoming a preacher, he consciously replaced the adorned style of late ancient rhetoric with a style streamlined toward speaking well with diverse north african congregations. parallelism frequently marked this style. augustine did not just employ parallelism stylistically, however; he also theorized about it. in his work de doctrina christiana, augustine described three rhetorical styles, or genera dicendi ("ways of speaking") (augustine of @xcite ; from these, he characterized the genus temperatum ("moderate style") by highly-parallel passages. 2 thus, his sermons are ideal for studying parallelism. in addition, his theory implies that parallelism detection may be useful for automatic stylistic analysis. already, it has been used to analyze discourse structure @xcite , summarize documents @xcite , identify idioms @xcite , evaluate student writing @xcite , study political speech @xcite and detect fake news @xcite . parallelism detection may also have broader nlp applications. in syntactic parsing, in the sentence i saw a man with a mustache and a woman with a telescope, the reading where telescope modifies saw is all but ruled out because it would violate parallelism. thus, modeling parallelism could assist in syntactic disambiguation. parallelism is also vital in disfluency detection, as speakers tend to maintain prior syntactic context when amending verbal errors @xcite . toward such ends, we establish the task of rhetorical parallelism detection (rpd, section 2). we create one dataset from augustine's sermons and adapt another consisting of chinese student essays (section 4), establish evaluation metrics (section 5), and investigate baseline models (section 6). by automatically learning and exploiting relationships among linguistic features, we achieve roughly a 40% f 1 score on both datasets' test sets (section 7). ## task definition moving away from the sentence-level conceptions of @xcite and @xcite , we formalize the task of rhetorical parallelism detection (rpd). broadly, we deem sequences to be parallel if they meet two conditions: suppose that we have a document w, which is a sequence of tokens 1 . . . . a span of w is a pair ( , ), where 1 ≤ ≤ ≤ , whose contents are the tokens . . . . we say that two spans ( 1 , 1 ) and ( 2 , 2 ) are overlapping if they have at least one token in common-that is, 1 ≤ 2 ≤ 1 or 2 ≤ 1 ≤ 2 . meanwhile, they are nested if a parallelism is a set of two or more nonoverlapping spans whose contents are parallel. we call these spans the branches of a parallelism. an example of a complete parallelism with three branches is given in fig. 1 . rpd, then, is the task of identifying all of the parallelisms in a text. let be a set of parallelisms. then falls into one of three categories: our augustinian sermon parallelism (asp) dataset (see section 4) is nested, and the paibi student essay (pse) dataset @xcite is flat. 3 although our baseline models only predict flat sets of parallelisms, our evaluations include nested ones. ## related work in this section, we connect two subareas of nlp to rpd in order to display their relation to and influence on its development. ## datasets this paper presents two datasets for rpd: the augustinian sermon parallelism (asp) and paibi student essay (pse) datasets. we first describe steps taken for the individual datasets in sections 4.1 and 4.2 before discussing shared preprocessing steps and data analyses in sections 4.3 and 4.4. ## evaluation metrics next, we describe a general family of evaluation metrics for rpd and specify one instance of it. suppose that we have a document w, as above, and a reference set g and a hypothesis set h of parallelisms in w. an evaluation metric for rpd should check both that the branches in h have the same spans as those in g and that the branches are grouped into parallelisms in h as they are in g. given these criteria, we follow prior work in coreference resolution (cr) to establish our metrics. coreference resolution involves detecting an entity of interest and linking all mentions related to that entity together; in a similar manner, our task links a set of spans together. as in the constrained entity-alignment f-measure metrics @xcite , we express our metrics as a bipartite matching between g and h. we use the kuhn-munkres (also known as "hungarian") algorithm @xcite to maximally align parallelisms between g and h. to do this, we must define two functions: score and size. the function score( 1 , 2 ), where 1 and 2 are parallelisms, determines how well 1 matches 2 , and the function size( ), where is a parallelism, bounds the score that can merit with another parallelism. these functions must satisfy size( ) > 0 and 0 ≤ score( 1 , 2 ) ≤ min{size( 1 ), size( 2 )}. as an example of this metric, consider the passage given in fig. 4 . suppose that the depicted parallelism-call it 1 -is the only parallelism in g. furthermore, suppose that the model proposes two hypotheses, @xmath0, 2 }, where 2 is a parallelism that does not overlap with 1 . then from this, we derive the maximally-weighted matching, = {( 1 , 1 )}, @xmath1. appendix b introduces some alternative choices of score and size that give partial credit for imperfectly-matching hypothesized parallelisms. 7 6 models in this section, we propose some models for automatic rpd as a starting point for future research. our models here treat rpd as a sequence labeling problem, using several novel variants of bio tagging (ramshaw and marcus, 1995). 8 6.1 tagging schemes branches are substrings, like named entities in ner, and parallelisms are sets of related substrings, like disfluencies in dd. variants of bio tagging have been successful for these sequence labeling tasks @xcite , so they are a natural choice for rpd as well. in this scheme, the first word of a branch is tagged b; the other words of a branch are tagged i; words outside of branches are tagged o. however, bio tagging does not indicate which branches are parallel with which. in each parallelism, for each branch except the first, we augment the b tags with a link, which is a number that indicates the previous branch with which is parallel. we propose two linking schemes. suppose we have consecutive parallel branches 1 = ( 1 , 1 ) and 2 = ( 2 , 2 ) with 1 < 2 . a token distance link, akin to links used in dd @xcite , is 1 -2 ; it is the (negative) 7 we accompany this paper with the pyrallelism library. it implements each of our metrics and provides sample formats for evaluating rpd. it is available on pypi and at https: //github.com/mythologos/pyrallelism . it is important to form tag sequences that help the model to learn better, diverging decisions from the dominant majority class o. therefore, in addition to adding links, we include three additional tag types, each of which is exhibited in fig. 4 . first, the m tag replaces o tags that occur for tokens that occur in the middle of consecutive branches in a parallelism. lines 2 and 3 of fig. 4 exhibit this: four non-branch tags become m instead of o because they are between related branches. this shift obstructs a model from predicting singlebranch parallelisms by providing a separate pathway to seek linked branches, as @xmath2. second, the e tag replaces branch-ending i tags to indicate the end of a branch. adding this tag removes the @xmath3. this change may encourage a model to be more sensitive to a branch's endpoint; because most parallelisms are more than two tokens long, branches largely must transition from b to either i or e, and e must eventually be selected before returning to o. third, the j tag replaces i tags in non-initial branches, where an initial branch is the first branch of a parallelism that occurs in a document. when paired with m tags, j tags help to promote a sequence of transitions which do not include o as a likely candidate until two branches have concluded. while the second example of figure 4 displays this, figure 5 shows this behavior across asp's entire training set. treating a sequence of tags as a linear chain, we tally all node transition pairs. when compared to the bio chain of tags (left), we can see that @xmath4. only non-initial branch tokens (i.e., b or j) can transition to o. through its supported transitions, the altered tagset actively enforces the constraint that a parallelism must have at least two branches. ## experiments in this section, we describe our experiments (section 7.1 and present our results (section 7.2). ## conclusions and future work in this paper, we introduced the task of rhetorical parallelism detection. we described two datasets, an evaluation framework, and several baseline models for the task. our baselines achieved fairly good performance, and we found that bert embeddings, a bilstm-based encoder, and an m-inclusive tagging scheme were valuable modeling components. we identify a few directions for future work. the models described here have a closed tagset, so they cannot predict links for distances not seen in the training data; modeling links in other ways might be more effective. our models only predict flat parallelisms; approaches for nested ner @xcite ) may be a viable direction for extending the encoder-crf paradigm toward this end. finally, applying this work's methods to detect grammatical parallelism might enhance tasks like syntactic parsing, automated essay evaluation, or disfluency detection.