RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Data
We use as our primary source the Translator’s Amalgamated Hebrew Old Testament (TAHOT) 2 2 2 www.STEPBible.org , which is based on the Leningrad Codex – the oldest complete extant version of the Hebrew Old Testament. Note that modern English translations follow a versification system that is at times slightly different to the Hebrew text due to a difference in textual traditions. We use the Hebrew versification system to better uncover chiasmi as they may be in the original text. N.B. We carry out all detection experiments using the Hebrew text, but for clarity and accessibility, report English translations 3 3 3 We release a formatted version of STEP Bible’s data, including translations, on the Huggingface Hub. DOI: 10.57967/hf/4174 . in tables and figures.

We segment the text into two levels: verses and half-verses. In the Hebrew text, half-verses are naturally marked by the cantillation symbol, atnach , which typically separates the two halves of a verse. We consider up to and including the word with the atnach to be the first half, while the remainder is the second half. We then remove all vocalizations and cantillation symbols before embedding.
Formalizing Chiasmus
The first step in our method involves constructing a cosine similarity matrix, denoted as S , based on feature vectors extracted from the text via E5, a multilingual embedding model Wang et al. ( 2024 ) 4 4 4 We use the ‘small’ variant of this model, with 118M parameters. . Our method is similar to that of Burns et al. ( 2021 ) , which uses pairwise cosine similarity of embedded representations to identify intertextual phrases in Latin.

Each element S_{ij} represents the cosine similarity between the feature vectors of textual units i and j . Next, we identify potential chiastic structures by focusing on matching groups of text pairs, such as A and A^{\prime} , B and B^{\prime} , and so forth. For each potential chiastic structure, we compute the chiasmus score \mu_{\text{chiasmus}} , which is the average cosine similarity of these matching pairs:

\mu_{chiasmus}=\frac{1}{k}\sum_{i=1}^{k}S_{pair(i)} (1)

where \text{pair}(i) refers to the indices of the matching pairs (e.g., A and A^{\prime} , B and B^{\prime} ). To assess the distinctiveness of this chiastic structure, we compute the average of all non-pair similarities, denoted \mu_{\text{non-pair}} , which includes comparisons such as S_{A,B},S_{B,C^{\prime}} , and others:

\mu_{non-pair}=\frac{1}{n}\sum_{i,j\innon-pair}S_{ij} (2)

Our final score for each window is computed as the difference between these two averages:

Final Score=\mu_{chiasmus}-\mu_{non-pair} (3)

To detect chiasmi across the text, we apply this method in a sliding window fashion, where each starting position in the text serves as a potential beginning of a chiastic structure. The length of the sliding window, N , is fixed for each experiment, and we test several different N values, analyzing the aggregated results. We ensure that chiasmi do not cross book boundaries by disallowing matches across these divisions.

Finally, we standardize the chiasmus scores across the text by calculating their z-scores. The z-score z_{i} for each window i is determined by:

z_{i}=\frac{\mu_{chiasmus,i}-\mu_{chiasmus,mean}}{\sigma_{chiasmus}} (4)

where \mu_{\text{chiasmus},\text{mean}} and \sigma_{\text{chiasmus}} are the mean and standard deviation of all chiasmus scores, respectively. We classify chiastic structures as significant if their z-scores exceed a threshold of three (3) standard deviations above the mean, thereby identifying statistically salient chiasmi within the text.
Why not use an LLM?
While large language models (LLMs) have enabled remarkable advances in a wide variety of NLP tasks, data contamination concerns and a lack of explainability limit their scope of usefulness for chiasmus detection in Biblical text.

Preliminary exploration revealed that some LLMs have a propensity to generate verbatim copyrighted English translations (e.g., the ESV) from Ancient Hebrew source passages. This behavior suggests that the extensive availability of online Biblical commentaries, which may reference chiastic structure, is likely included in web-based training corpora. Consequently, the outputs of LLMs risk being skewed by prior exposure to human annotation Balloccu et al. .

Furthermore, the lack of transparency in LLM-generated responses poses a significant barrier for adoption in scholarly contexts. Biblical scholars, who are our primary target audience, require interpretable and verifiable methods rather than opaque, black-box solutions. Additionally, our aim extends beyond merely detecting chiasmi; we seek to formalize the concept mathematically, thereby offering a rigorous and standardized framework for discussing what remains a somewhat ambiguous topic. Such a formalism could serve as a valuable tool for facilitating scholarly discourse and advancing the study of chiasmus.
Experiments
We run our model over the Hebrew Old Testament, considering every line or half-line as a potential starting position and length (N) of chiasmus to be in the range of four to eight ( N\in[4,8] ). We take the top-50 highest-scoring outputs for both half-verse and verse grouping and evaluate them via human annotation. Annotation guidelines and results are found in § ˜ 3.1 . We use top- k precision as our evaluation metric as we are primarily interested in creating a tool for scholars to find the most-promising candidates for chiasmus to further examine.

Table 2 presents an overview of the system’s output for chiastic structures at the half-verse and verse levels. A total of 1,896 chiastic structures were identified at the half-verse level, with an average length of 5.93 textual units ( \pm 1.34) and an average score of 0.32 ( \pm 0.1). For verse-level groupings, 879 chiastic structures were found, with an average length of 6.01 lines ( \pm 1.38) and an average score of 0.29 ( \pm 0.08). The book of Genesis contains the highest number of half-verse chiasmi, while Numbers contains the most verse-level chiasmi.

Full Output

Annotated

As shown in Figure 1 , the number of detected chiastic structures varies across books of the Bible, with more instances found at the half-verse level than at the verse level for all books. Notably, certain books exhibit disproportionately higher numbers of half-verse chiasmi, particularly Genesis, 1 Samuel, Judges, 1 Chronicles, Psalms, Jeremiah, and Ezekiel. This trend is consistent with the literary nature of these texts: Psalms, Jeremiah, and Ezekiel include significant poetic sections, where half-verse chiastic structures are more prominent, while Genesis features dense narrative and highly literary passages as well as many formulaic genealogies. The high counts in 1 Samuel, Judges, and 1 Chronicles, which are historical books, likely reflect the system’s identification of formulaic narrative patterns, such as the recurring descriptions of the kingly line of Israel (e.g., “X became king, reigned for Y years, and did evil in the sight of the Lord”).
Human Annotation
The top-50 scoring half-verse and verse chiasmi were manually reviewed by the first two authors, who both have graduate-level training in ancient languages and literature 5 5 5 While the chiasmus identification is done entirely in Hebrew, the annotators use a literal English translation following Hebrew word order alongside the Hebrew text for easier inspection. . Given a three-class rubric, they were asked to determine whether the set of verses of half-verses identified by the model exhibited (1) chiastic repetition : a chiastic structure of repetition formed either through lexical or semantic textual units, (2) non-chiastic repetition : lexical or semantic repetition of textual units, but not in a discernibly chiastic way, or (3) no repetition : no discernible parallel or repeating content. Cohen’s Kappa ( \kappa ), used to quantify inter-annotator agreement, is 0.76 and 0.89 for half-verses and verses, respectively, indicating strong agreement between the annotators.

Two verse-level passages and four half-verse level passages were putative between chiastic repetition and non-chiastic repetition, while there were only two (both half-verse) passages that were disputed between no repetition and chiastic repetition. In other words, annotators were nearly always in agreement over which passages had elements of structural repetition, but discerning between chiastic and non-chiastic repetition poses a slightly more difficult challenge.

Considering “true” chiasmi to be those marked as chiastic by both annotators, we achieve a system precision@ k of 0.60 for half-verses and 0.80 for verses. In both experiments, the majority of top-scoring chiasmi are found in narrative sections of text.

Interestingly, passages classified as non-chiastic repetition often involved formulaic or ritualistic language, which could be of interest to scholars seeking computational methods for identifying such patterns in texts. We find 29 examples of this across the top 100 collectively. Only 3 of the top 100, or 3%, of the top-scoring passages belonged to the no repetition class.
Discussion
Several qualitatively interesting examples of chiasmus were identified by our method, highlighting the richness of the Biblical texts and the alignment with existing literary scholarship. One notable example is Genesis 1:19-23, as shown in Table 3 . This five-line chiasmus, positively identified by both annotators, exhibits clear lexical parallels between its paired sections. The chiastic structure here emphasizes the order and the rhetorical intentionality in the Creation narrative, underscoring God’s repeated affirmation that His creation is “good”. This example aligns with scholarly interpretations that highlight the poetic nature of the Creation account.

Other significant examples include the story of Jacob stealing Esau’s birthright, where the chiastic structure reflects the tension and reversal of fortune between the brothers. Similarly, the account of Isaac and Abraham and the sacrificial lamb contains a chiasmus that heightens the dramatic and theological impact of the narrative, as God intervenes at the critical moment.

The method also uncovered a clear chiasmus in God’s covenant with Noah after the flood, where the repetitive structure emphasizes God’s promise of restoration and the symbolic importance of the ‘bow’ in the clouds. Additionally, in Ezekiel’s poetic description of the image of the glory of the LORD, chiastic elements serve to enhance the vividness and majesty of the vision, a hallmark of Ezekiel’s prophetic style. Illustrations of these chiasmi may be seen in appendix ˜ A .

Notably, many instances of God’s reported speech are presented in chiastic or poetic form, which may suggest an intentional literary quality meant to convey authority and solemnity. These findings further support the hypothesis that chiasmus is often employed for rhetorical and theological purposes in Biblical texts.
Conclusion
Our approach demonstrates the ability to uncover intricate literary patterns that might otherwise be overlooked, providing valuable insights for scholars of Biblical texts, political oratory, and literary studies. This example, along with our overall findings, underscores the importance of advanced computational techniques in literary analysis and supports the broader application of our method for discovering chiasmi across various texts and translations. One future step is using a chiasmus detection method to create a labeled corpus of chiasmi within the Bible, particularly the Psalms, for scholarly exploration.
Limitations
In this study, we only investigate chiastic structures at the verse-level and half-verse-level. However, chiasmi can also be identified at the narrative level, where narrative segments topically form a chiastic plot structure, such as the narrative of the flood in Genesis. We exclude this type since it exhibits many fewer lexical features and is overall less precisely defined in the scholarship.
Acknowledgments
We would like to thank scholars at Tyndale House, especially Ellie Wiener and Caleb J. Howard, for useful discussions about Biblical Hebrew. Thanks to Andrew Caines for his helpful comments on previous drafts.
Hope McGovern’s work is supported by the Woolf Institute for Interfaith Relations and the Cambridge Trust
Figure 1: Total number of chiasmi per Book at verse and half-verse level. While some books tend to have more chiasmi overall, this figure shows whether verse-level or half-verse-level chiasmi are more prevalent in each book.; Total number of chiasmi per Book at verse and half-verse level. While some books tend to have more chiasmi overall, this figure shows whether verse-level or half-verse-level chiasmi are more prevalent in each book.
The ‘X’ pattern of chiasm in Psalm 35:4-7 (ESV). Pairs (A, A’) and (B, B’) exhibit repeated phrases and conceptual links.
Summary of detected chiasmi. 2700+ chiasmi were detected at the verse and half-verse level. The highest number of chiasmi was found in the Book of Genesis and Book of Numbers. Both the precision and the inter-annotator agreement increase for the verse-level chiasmi.
English translation of a positive example of a chiasmus automatically detected by our method. Gen 1:19-23 (ESV)
The story of Jacob stealing Esau’s birthright. English translation of a chiasmus automatically detected by our method. Gen 25:30-34 (ESV)
God’s covenant with Noah after the flood. English translation of a chiasmus automatically detected by our method. Gen 9:12-15 (ESV)