A Statistical Exploration of Text Partition Into Constituents: The Case of the Priestly Source in the Books of Genesis and Exodus
Year: 2023

Abstract:
We present a pipeline for a statistical textual exploration, offering a stylometry-based explanation and statistical validation of a hypothesized partition of a text. Given a parameterization of the text, our pipeline: (1) detects literary features yielding the optimal overlap between the hypothesized and unsupervised partitions, (2) performs a hypothesis-testing analysis to quantify the statistical significance of the optimal overlap, while conserving implicit correlations between units of text that are more likely to be grouped, and (3) extracts and quantifies the importance of features most responsible for the classification, estimates their statistical stability and cluster-wise abundance. We apply our pipeline to the first two books in the Bible, where one stylistic component stands out in the eyes of biblical scholars, namely, the Priestly component. We identify and explore statistically significant stylistic differences between the Priestly and non-Priestly components.

heading: 1 Introduction
text: It is agreed among scholars that the extant version(s) of the Hebrew Bible is a result of various editorial actions (additions, redaction, and more). As such, it may be viewed as a literary patchwork quilt – whose patches differ, for example, by genre, date, and origin – and the distinction between biblical texts as well as their relation to other ancient texts from the Near East is at the heart of biblical scholarship, theology, ancient Israel studies, and philology. Many debates in this field have been ongoing for decades, if not centuries, with no verdict (e.g., the division of the biblical text into its “original” constituents; (Gunkel, 2006; von Rad, 2001; Wellhausen et al., 2009)). While several paradigms gained prevalence throughout the evolution of this discipline (Wellhausen et al., 2009; von Rad, 1972; Zakovitch, 1980), the jury is still out on many related hypotheses, such that these paradigms are
prone to drastic (and occasionally abrupt) changes over time (e.g., Nicholson, 2002).
When scrutinized, the interrelation between various literary features within the text may shed light on its historical context. From it, one may infer the number of authors, time(s) and place(s) of composition, and even the geopolitical, social, and theological setting(s) (e.g., Wellhausen et al., 2009; von Rad, 1972; Knohl, 2010; Römer, 2015).
Such scrutiny thus serves a double purpose: (1) to disambiguate and identify features in the text that are insightful of the lexical sources of the partition (e.g., Givón, 1991; Yosef, 2018; van Peursen, 2019); (2) to attempt to trace these features to the context of the text’s composition (e.g., Koppel et al., 2011; Pat-El, 2021).
Works employing computational methods in text stylometry – a statistical analysis of differences in literary, lexical, grammatical or orthographic style between genres or authors (Holmes, 1998) – have been introduced several decades ago (Tweedie et al., 1996; Koppel et al., 2002; Juola et al., 2006; Koppel et al., 2009; Stamatatos, 2009), with biblical exegesis spurring very early attempts at computerized authorship-identification tasks (Radday, 1970; Radday and Shore, 1985). Since then, these methods have proved useful also in investigating ancient (e.g., Kestemont et al., 2016; Verma, 2017; Kabala, 2020), and biblical texts as well (Koppel et al., 2011; Dershowitz et al., 2014; Roorda, 2015; van Peursen, 2019), albeit to a humbler extent. Finally, statistical-learning-based research, which makes headway in an impressively diverse span of disciplines, is taking its first steps in the context of ancient scripts (Murai, 2013; FaigenbaumGolovin et al., 2016; Dhali et al., 2017; Popović et al., 2021; Faigenbaum-Golovin et al., 2020). In the biblical context, Dershowitz et al. (2014, 2015) addressed reproducing hypothesized partitions of various biblical corpora with a computerized approach as well, using features such as orthographic
1918
differences and synonyms. In the first work – the Cochran-Mantel-Haenszel (CMH) test was applied as a means of hypothesis testing, with the null hypothesis that the synonym features are drawn from the same distribution. While descriptive statistics were successfully applied to various classification and attribution tasks of ancient texts (see above), uncertainty quantification has been insufficiently explored in NLP-related context (Dror et al., 2018), and in particular – in that of text stylometry.
In this work, we introduce a novel exploratory text stylometry pipeline, with which we: (1) find a combination of textual parameters that optimizes the agreement between the hypothesized and unsupervised partitions, (2) test the statistical significance of the overlap, (3) extract features that are important to the classification, the proportion of their importance, and their relative importance in each cluster. Each stage of this analysis was crossvalidated (i.e., applied on many randomly-chosen sub-samples of the corpus rather than applied once on the entire corpus) and tested for statistical stability.
To perform (2) in a meaningful way for textual analysis, we had to overcome the fact that labelpermutation tests do not consider correlations between units of text, which affect their likelihood of being clustered together. This results in unrealistically optimistic p-values, as, in fact, the hypothesized partition does implicitly consider such correlations (by, e.g., grouping texts of a similar genre, subject, etc.). We overcome this by introducing a cyclic label-shift test which preserves the structure of the hypothesized partition, thus conserving the implicit correlations therein. Furthermore, we identify literary features that are responsible for the clustering, as opposed to intra-cluster feature selection techniques (e.g., Hruschka and Covoes, 2005; Cai et al., 2010; Zhu et al., 2015), which seek to detect significant features within each cluster. This is also a novel approach to text stylometry.
With this pipeline, we examined the hypothetical distinction between texts of Priestly (P) and non-Priestly (nonP) origin in the books of Genesis and Exodus. The Priestly source is the most agreed-upon constituent underlying the Pentateuch (i.e., Torah). The consensus over which texts are associated with P (mainly through semantic analysis) stems from the stylistic and theological distinction from other texts in the Pentateuch, streamlined across texts associated therewith (e.g., Holzinger,
1893; Knohl, 2007; Römer, 2014; Faust, 2019). Therefore, the distinction between P and nonP texts is considered a benchmark in biblical exegesis.

heading: 2 Methodology

heading: 2.1 Data – Digital Biblical Corpora
text: We use two digital corpora of the Masoretic variant of the Hebrew Bible in (biblical) Hebrew: (1) a version of the Leningrad codex, made freely available by STEPBible.1 This dataset comes parsed with full morphological and semantic tags for all words, prefixes, and suffixes. From this dataset, we utilize the grammatical representation of the text through phrase-dependent part-of-speech tags (POS). (2) A digitally parsed version of the Biblia Hebraica Stuttgartensia (Roorda, 2018) (hereafter BHSA).2 In the BHSA database, we consider the lexematic (i.e., words reduced to lexemes) and grammatical representation of the text through POS. The difference between the two POS-wise representations of the text is that (1) encodes additional morphological information within tags, resulting in several hundreds of unique tags, whereas (2) assigns one out of 14 more “basic” grammatical tags3 to each word. We refer to the POS-wise representation of (1) and (2) as “high-res POS” and “low-res POS”, respectively.

heading: 2.2 Manual Annotation of Partition
text: From biblical scholars, we receive verse-wise labeling, assigning each verse in the books of Genesis (1533 verses) and Exodus (1213 verses) to one of two categories: P or nonP, made available online4. Hereafter, we refer to this labeling as “scholarly labeling”.
While the dating of P texts in the Pentateuch remains an open, heavily-debated question (e.g., Haran, 1981; Hurvitz, 1988; Giuntoli and Schmid, 2015), there exists a surprising agreement amongst biblical scholars regarding what verses are affiliated to P (e.g., Knohl, 2007; Römer, 2014; Faust, 2019), amounting to an agreement of 96.5% and 97.3% between various biblical scholars for the books of Genesis and Exodus, respectively. We describe the computation of these estimates in Appendix A.
1https://github.com/STEPBible/STEPBible-Data 2https://etcbc.github.io/bhsa 3https://etcbc.github.io/bhsa/features/pdp/ 4https://github.com/YoffeG/PnonP

heading: 2.3 Text Parameterization
text: The underlying assumption in this work is that a significant stylistic difference between two texts of a roughly-similar genre (or, indeed, any number of distinct texts) should manifest in simple observables in NLP, such as the utilization of vocabulary (i.e., distribution of words) and grammatical structure.
We consider three parameters whose different combinations emphasize different properties-, and therefore yield different classifications- of the text:
• Lexemes, low-, and high-res POS: we consider three representations of the text: words in lexematic form and low- and high-res POS (see §2.1). This parameter tests the ability to classify the text based on vocabulary or grammatical structure. • n-gram size: we consider sequences of consecutive lexemes/POS of different lengths (i.e., ngrams). Different sizes of n-grams may be reminiscent of different qualities of the text (e.g., Suen, 1979; Cavnar and Trenkle, 1994; Ahmed et al., 2004; Stamatatos, 2013). For example, a distinction based on a larger n-gram may indicate a semantic difference between texts or the use of longer grammatical modules in the case of POS (e.g., parallelisms in the books of Psalms and Proverbs (Berlin, 1979)). In contrast, a distinction made based on shorter n-grams is indicative of more embedded differences in the use of language (e.g., Wright and Chin, 2014; Litvinova et al., 2015). That said, both these examples indicate that a “false positive” distinction can be made where there is a difference in genre (e.g., Feldman et al., 2009; Tang and Cao, 2015). This degeneracy requires careful analysis of the resulting clustering or inclusion of genre-specific texts only for the clustering phase. • Verse-wise running-window width: biblical verses have an average length of 25 words. Hence, a single verse may not contain sufficient context that can be robustly classified. This is especially important since our classification is based on statistical properties of features in the text (see §2.4). Therefore, we define a running window parameter, which concatenates consecutive verses into a single super-verse (i.e., a running window of k would turn the ith verse to the sequence of the i − k : i + k verses) to provide additional context.

heading: 2.4 Text Embedding
text: We consider individual verses, or sequences of verses, as the atomic constituents of the text (see
§2.3). We use tf-idf (term frequency divided by document frequency) to encode each verse, assigning a relevance score to each feature therein (Aizawa, 2003). Works such as Fabien et al. (2020); Marcińczuk et al. (2021) demonstrate that in the absence of a pre-trained neural language model, tf-idf provides an appropriate and often optimal embedding method in tasks of unsupervised classification of texts. For a single combination of an n-gram size and running-window width (using either lexemes, low- or high-res POS), the tf-idf embedding yields a single-feature matrix X ∈ Rn×d, where n is the number of verses and d is the number of unique n-grams of rank n.
It is important to note that this work aims to set a benchmark for future endeavors using strictly traditional machinery throughout our analysis. To ensure collaboration with biblical scholars, our methodology allows for full interpretability of the exploration process (see §D.4). This has threefold importance: (1) the field of text stylometry, especially that of ancient Hebrew texts, has hitherto been explored statistically and computationally to a limited extent, such that even when utilizing conservative text-embeddings, such as in this work, considerable insight can be gained concerning both the quality of the analysis and the philological question. (2) Obtaining benchmark results using traditional embeddings is a pre-condition for implementing more sophisticated yet convoluted embeddings, such as pre-trained language models (e.g. Shmidman et al., 2022) or self-trained/calibrated language models (Wald et al., 2021), which we intend to apply in future works. (3) Finally, the interdisciplinary nature of this work and our desire to contribute to the field of biblical exegesis (and traditional philology in general) requires our results to be predominantly interpretable, such that they can be subjected to complementary analysis by scholars from the opposite side of the interdisciplinary divide (Piotrowski, 2012).

heading: 2.5 Clustering
text: We choose the k-means algorithm as our clustering tool of choice (Hastie et al., 2009, Ch. 13.2.1) and hardwire the number of clusters to two, according to the hypothesized P/nonP partition. The justification for our choice of this algorithm is its simple loss-optimization procedure, which is vital to our feature importance analysis and is discussed in detail in §2.8.
We use the balanced accuracy (BA) score (Sokolova et al., 2006) for our overlap statistic, a standard measure designed to address asymmetries between cluster sizes.
Due to the stochastic nature of k-means (Bottou, 2004), every time it is used in this work – it is run 50 times – (with different random initialization) – and the result yielding the smallest k-means loss is chosen.

heading: 2.6 Optimizing for Overlap
text: We perform a 2D grid search over a pre-determined range of n-gram sizes and running-window widths to find the parameters combination yielding the optimal overlap for low- or high-res POS lexemes. We test the statistical stability of each combination of these parameters (i.e., to ensure that the overlap reached by each combination is statistically significant) by cross-validating the 2D grid search over some number of randomly-chosen sub-sets of verses, from which we derive the average overlap value for each combination of parameters and the standard deviation thereof. We describe the optimization process in detail in Appendix B.

heading: 2.7 Hypothesis Testing and Validating Results
text: Through hypothesis testing, we establish the statistical significance of the achieved optimal overlap value between the unsupervised and hypothesized partitions. To derive a p-value from some empirical null distribution, we consider the assumption that the hypothesized partition, manifested in the scholarly labeling, exhibits a specific formulaic partition of chunks of the text. A formulaic partition, in turn, suggests that verses within each of the P (nonP) blocks are correlated – a fact that the standard label-permutation test is intrinsically agnostic of, as it permutes labels without considering potential correlations between verses (Fig. 1 left). Thus, the null distribution synthesized through a series of permutations would represent an overlyoptimistic scenario that does not correspond to any conceivable scenario in text stylometry.
To remedy this, we devise a more prohibitive statistical test. Instead of permuting the labels to have an arbitrary order, we perform a cyclic shift test of the scholarly labeling (Fig. 1 right). This procedure retains the scholarly labels’ hypothesized structure but shifts them across different verses. We perform as many cyclic shifts as there are labels (i.e., number of verses) in each book by skips of twice the largest running-window width considered in the
optimization procedure. For each shift, we perform the parameter optimization procedure (see §2.6), where we now have the shifted scholarly labels instead of the original (“un-shifted”) ones. Thus, we generate a distribution of our statistic under the null hypothesis, from which we derive a p-value.
In Fig. 1, we present an intuitive scheme where we demonstrate our rationale concerning the hypothesis-testing procedure in text stylometry.
2.8 Feature Importance and Interpretability of Classification
Given a k-means labeling produced for the text, which was embedded according to some combination of parameters (§2.3), we wish to quantify the importance of individual n-grams to the classification, the proportion of their importance, and associate to which cluster they are characteristic of.
Consider the loss function of the k-means algorithm:
argmin S
k∑
i=1
|Si| · var(Si),
where k = 2 is the number of desired clusters, S is the group of all potential sets of verses, split into k clusters, |Si| is the size of the ith cluster (i.e., number of verses therein) and var(Si) is the variance of the ith cluster. That is, the k-means aims to minimize intra-class variance. Equivalently, we could optimize for the maximization of the inter-cluster variance (i.e., the variance between clusters), given by
argmax S
∑
i∈S1
∑ j∈S2 ∥xi − xj∥2. (1)
Let D ∈ R|S1|·|S2|×d denote the matrix of intercluster differences whose columns are Dℓ, which for i ∈ {1, . . . , |S1| − 1}, j ∈ {1, . . . |S2|} and ℓ = i · |S2|+ j are defined by
(D)ℓ = xi − xj .
Then, applying PCA to D would yield the axis across which Eq. (2.8) is optimized as the first principal component. This component represents the axis of maximized variance, and each feature’s contribution is given by its corresponding loading. Finally, it can be shown that this principal axis could also be computed by subtracting the centroids of the two clusters. Therefore, to leverage this observation and extract the features’ importance, we perform the following procedure:
• Compute the principal separating axis of the two clusters by subtracting the centroid of S1 from that of S2. • The contribution of each n-gram feature to the cluster separation is given by its respective loading in the principal separating axis. • Since tf-idf assigns strictly non-negative scores to each feature, the signs of the principal separating axis’ loadings allow us to associate the importance of each n-gram to a specific cluster (see Appendix C). • To determine the stability of the importance of n-grams across multiple sub-samples of each book, we perform a cross-validation routine (similar to the one performed in §2.6). Explicitly, we perform all the steps listed above for some number of simulations (i.e., randomly sample a sub-set of verses and extract the importance vector) and compute the mean and standard deviation of all simulations. • Finally, we compute the relative uniqueness of
each important feature w.r.t. both clusters – assigning it a score between −1 : 1. A score of 1 (−1) indicates that the feature is solely abundant in its associated (opposite) cluster. Thus, a feature’s abundance nearer zero indicates that its contribution to the clustering is rather in its combination with other features than a standalone indicator.

heading: 3 Results
text: We apply a cross-validated optimization analysis to the three representations of the book (§2.6) by performing 2D grid searches for 20 randomly chosen sub-sets of 250 verses for each representation. We perform a cross-validated cyclic hypothesistesting analysis for the optimal overlap value (§2.7) using five randomly-chosen sub-sets of verses, similarly to the above – and derive a p-value. Finally, we perform a cross-validated feature importance analysis for every representation (§2.8), over 100 randomly-chosen sub-sets of verses, similar to the
above. In Table 1, we list the cross-validated results of each representation for both books and the derived p-value. In Fig. 2, we visualize results for all stages of our analysis applied to the lexematic representation of the book of Genesis. In Appendix D, we plot the results for all stages of our analyses for both books and discuss them in detail. Appendix E presents a detailed biblical-exegetical analysis of our results and an expert’s evaluation of our approach.

heading: 3.1 Understanding the Discrepancy in Results Between the Books of Genesis and Exodus
text: In Table 1, we list our optimal overlap values for all textual representations of both books. Notice that there exists a statistically-significant discrepancy between the optimal overlap values between both books that is as follows:
For Exodus, all three representations reach the same optimal overlap of roughly 88%. In contrast, in Genesis – there is a difference between the achieved optimal overlap values for the lexematic and low-res POS representations on the one hand (73%) and high-res POS representation on the other (65%).
When examining the verses belonging to each cluster when classified with the optimal parameter combination, it is evident that most of the overlapping P-associated verses in Exodus are grouped in two blocks of P-associated of text, spanning 243 and 214 verses. These make considerable outliers in the size distribution of P-associated blocks in both books (Fig. 3), which may be related to the observed discrepancy.
This discrepancy begs two questions:
1. Are the linguistic differences between P/nonP – which may be captured by our analysis – that are attenuated for shorter sequences of P texts?
2. Does the high overlap in the book of Exodus arise due to an implicit sensitivity to the generic/semantic uniqueness of the two big Passociated blocks rather than a global stylistic difference between P/nonP?
To examine this, we perform the following experiment: each time, we remove a different Passociated block (1st largest, 2nd largest, 3rd largest, and the 1st + 2nd largest) from the text
and perform a cross-validated optimization analysis with low- and high-res POS (see §2.6). We then compare the resulting optimal overlap values of each time. We plot the results of this experiment in Fig. 4.
We find that, as expected, the optimal overlap drops as a function of the size of the removed Passociated text. Interestingly, the optimal overlap increases when a smaller block is removed. Additionally, unlike in the case of Genesis, the lowres POS representation doesn’t lead to increased optimal overlap relative to the high-res POS representation. This suggests that: (1) the fluctuation of the optimal overlap indicates that our pipeline is sensitive to some semantic field associated with the two large P blocks rather than to a global stylistic difference between P/nonP. (2) In cases of more sporadically-distributed texts that are stylistically different from the text in which they are embedded – one representation of the text is not systematically preferable to others.

heading: 4 Conclusions
text: We examined the hypothetical distinction between texts of priestly (P) and non-priestly (nonP) origin in the books of Genesis and Exodus, which we explored with a novel unsupervised pipeline for text stylometry. We sought a combination of a running-window width (i.e., the number of consecutive verses to consider as a single unit of text) and n-gram size of lexemes, low- or high-resolution phrase-dependent parts-of-speech that optimized the overlap between the unsupervised and hypothesized partitions. We established the statistical significance of our results using a cyclic-shift test, which we show to be more adequate for text stylometry problems than a naive permutation test. Finally, we extracted n-grams that contribute the most to the classification, their respective proportions, statistical robustness, and correlation to other features. We achieve optimal, statistically significant overlap values of 73% and 90% for the books of Genesis and Exodus, respectively.
We find the discrepancy in optimal overlap values between the two books to stem from two factors: (1) A more sporadic distribution of P texts in Genesis, as opposed to a more formulaic one in Exodus. (2) The sensitivity of our pipeline to a distinct semantic field manifested in two large P blocks in Exodus, comprising the majority of the P-associated text therein.
Through complementary exegetical and statistical analyses, we show that our methodology differentiates the unique generic style of the Priestly source, characterized by lawgiving, cult instructions, and streamlining a continuous chronological sequence of the story through third-person narration. This observation corroborates and hones the stance of most biblical scholars.

heading: 5 Limitations
text: • The interdisciplinary element – at the heart of this work – mandates that our results be interpretable and relevant to scholars from the opposite side of the methodological divide (i.e., biblical scholars). This, in turn, introduces constraints to our framework – the foremost is choosing appropriate text-embedding techniques. As discussed in §2.4 and §2.8, the ability to extract specific lexical features (i.e., unique n-grams) that are important to the
classification, to quantify them, and subject them to complementary philological analysis (see Appendix E) – requires that they be interpretable. This constraint limits the ability to implement state-ofthe-art language-model-based embeddings without devising the required framework for their interpretation. Consequently, using traditional embeddings – which encode mostly explicit lexical features (e.g., see §2.4) – limits the complexity of the analyzed textual phenomena and is therefore agnostic of potential signal that is manifested in more complex features.
• In text stylometry questions, especially those related to ancient texts, it is often problematic (and even impossible) to rely on a benchmark training set with which supervised statistical learning can take place. This, in turn, means that supervised learning in such tasks must be implemented with extreme caution so as not to introduce a bias into a supposedly-unbiased analysis. Therefore, implementing supervised learning techniques for such tasks requires a complementary framework that could overcome such potential biases. In light of this, our analysis involves predominantly unsupervised exploration of the text, given different parameterizations.
• Our ability to draw insight from exploring the stylistic differences between the hypothesized distinct texts relies heavily on observing significant overlap between the hypothesized and unsupervised partitions. Without it, the ability to discern the similarity between the results of our pipeline is greatly obscured, as the pipeline remains essentially agnostic of the hypothesized partition. Such a scenario either deems the parameterization irrelevant to the hypothesized partition or disproves the hypothesized partition. Breaking the degeneracy between these two possibilities may entail considerable additional analysis.

heading: 6 Acknowledgements
text: We thank Dr. Rotem Dror for her kind assistance and contribution to the editing process and Ziv Ben-Aharon for providing technical guidance in operating the HUJI cluster.

heading: Appendices

heading: A Estimating Scholarly Consensus of P-associated Texts in Genesis and Exodus
text: To quantify the consensus amongst biblical scholars concerning the distinction between P/nonP texts in Genesis and Exodus, we consider two sets of labelings of P/nonP: the first is that provided by biblical scholars, and the other is similar labeling used in the work of Dershowitz et al. (2015), who also apply computational methods in an attempt to detect a meaningful dichotomy between P and nonP texts as well, albeit under a different paradigm. In their work, Dershowitz et al. consider P/nonP distinctions of three independent biblical scholars and compile a single “consensus” labeling of verses over the affiliation of which all three scholars agree (1291 verses in Genesis and 1057 verses in Exodus), except for explicitly incriminating P texts (such as genealogical trees that are strongly affiliated with P (e.g., Jonker, 2012)). We find a 96.5% and 97.3% agreement between the two labelings for the books of Genesis and Exodus, respectively.

heading: B Cross-validated Overlap Optimization
text: The cross-validated overlap optimization is performed as follows:
• For lexemes/POS, we consider a range of ngram sizes and a range of running-window widths, through combinations of which we optimize the classification overlap. These ranges are determined empirically by observing the overlap values decrease monotonically as the n-gram sizes/running-window widths become too large (see Figs. 6, 5). This produces a 2D matrix, where the (i, j)th entry is the resulting overlap value achieved by the k-means given the combination of the ith running-window width and the jth n-gram size.
• We cross-validate the grid-search process by performing a series of simulations, where in each simulation, we generate a random subset of 250 verses from the given book, containing at least 50 verses of each class (according to the scholarly labeling). Each such simulation produces a 2D overlap matrix, as mentioned above, for the given subset of verses.
• Finally, we average the 2D overlap matrices
of all simulations, and for each entry, we calculate its standard deviation across all simulations, producing a 2D standard deviation matrix. After normalizing the average overlap matrix by the standard deviation matrix, we choose the optimal combination that produces the classification with the optimal overlap.
• We perform this analysis on both lexemes, low- and high-res POS, yielding three averaged overlap matrices, from which optimal parameter combinations and their respective standard deviations (calculated across the cross-validation simulations) can be identified for each feature.

heading: C Feature Importance Analysis
text: We consider the two optimal overlap clusters of verses as the algorithm assigned them. We calculate a difference matrix D, where from every verse in cluster 1, we subtract every verse in cluster 2, receiving a matrix D ∈ R|S1|·|S2|×d, where |S1|, |S2| are the number of verses assigned to cluster 1 and 2, respectively, and d is the dimension of the embedding (i.e., number of unique n-grams in the text). Note that tf-idf embedded texts are non-negative, such that the difference matrix D has positive values for features with a high tf-idf score in cluster 1 and negative values for features with a high tf-idf score in cluster 2. As mentioned in §2.8, the first principal axis of D is equivalent to the difference between the two centroids produced by the k-means. Similarly, this difference vector is a linear combination of all the features (i.e., n-grams), where a numerical value gives the importance of each feature called “loading”, ranging from negative to positive values and their importance to the given principal axes is given by their absolute value. Due to the nature of the difference matrix D (see §2.8) – the sign of the loading indicates in which cluster the feature is important. Thus we can assign distinguishing features to the specific cluster of which they, or a combination thereof with other features, are characteristic.
Finally, we seek to determine the stability of the importance of features across multiple sub-samples of each book. We perform this computation as follows: given a parameters combination, we perform all the steps listed above for 100 simulations, where in each simulation, we randomly sample a sub-sample of 250 verses and extract the importance loadings of the features. We then average
all simulation-wise loadings and derive the variance thereof to receive a cross-validated vector of feature importance loadings and their respective uncertainties. These are plotted as the error bars in Figs. 2, and similarly in the figures in Appendix C

heading: D Results

heading: D.1 Optimization Results: Genesis
text: For each representation, we achieve the following optimal overlap values (see Fig. 6): 72.95±6.45% for lexemes, 65.03±5.64% for high-res POS, and 73.96±2.91% for low-res POS. We observe the following:
• Optimal overlap values achieved for lexemes and high-res POS are consistent to within a 1σ, whereas the optimal overlap achieved for low-res POS is higher by ∼ 2σ.
• We find a less consistent and considerably larger spread of optimal parameter combinations for the low- and high-POS-wise embeddings, as opposed to Exodus. While no clear pattern of optimal parameter combinations is observed in any feature, small n-gram sizes are also preferred here. For lexemes, a welldefined range of running-window widths of unigrams is observed to yield optimal overlaps.
• Unlike in the case of Exodus, parameter combinations yielding optimal overlap values of each feature (marked with red cells in Fig. 5) do not exhibit higher consistency across the cross-validation simulations than combinations that yield smaller overlap (i.e., small cross-validation variance, see the right panels in Fig. 5), except for the low-res POS feature.

heading: D.2 Optimization Results: Exodus
text: For each feature, we achieve the following optimal overlaps (see Fig. 6): 89.23±2.53% for lexemes, 88.63±1.96% for high-res POS, and 86.53±2.91% for low-res POS. We observe the following:
• For all three representations, optimal overlap values are consistent to within 1σ.
• For all three representations, parameter combinations yielding optimal overlap values (marked with red cells in Fig. 6) exhibit high consistency across the cross-validation simulations (i.e., slight cross-validation variance, see the right panels in Fig. 6).
• For lexemes, we find that the range of optimal parameter combinations is concentrated within 1- and 2-grams and is relatively independent of running-window width (i.e., when some optimal running-window width is reached–the overlap values do not change dramatically as it increases).
• For the high-res POS, we find that the range of optimal combinations is restricted to 2-grams, but is also insensitive to the running-window width.
• For low-res POS, we find that larger n-gram sizes, and a wider range thereof, produce the optimal overlap values. Additionally, we observe a dependence between given n-gram sizes and running-window widths to reach a large overlap.

heading: D.3 Hypothesis Testing Results
text: We present our results of the hypothesis testing through cyclic-shift, described in §2.7. Here, too, we perform a cross-validated test by performing five simulations – each containing a randomly chosen sub-sample of 250 verses (with a mandatory minimum of 50 verses of each class), to which the cyclic-shift analysis is applied. We compute the optimal overlap for every shift and generate a shiftseries of optimal overlap values (i.e., the null distribution). We then average across simulations. We then derive the p-value from the synthesized null distribution. The chosen “real optimal overlap”, which we use to derive the p-value, is the average optimal overlap at a shift of 0 (i.e., original labeling) minus its standard deviation. For each book, we perform this analysis for the features yielding the optimal overlap; low-res POS for Genesis and high-res POS for Exodus. We plot our results for both books in Fig. 7.
The resulting p-values are 0.08 and 0.06 for the books of Genesis and Exodus, respectively.

heading: D.4 Feature Importance Analysis Results

heading: D.4.1 Feature Importance: Genesis
text: In Figs. 8-10 we plot feature importance analysis results for the three representations of the book of Genesis.

heading: D.4.2 Feature Importance: Exodus
text: In Figs. 11-13 we plot feature importance analysis results for the three representations of the book of Exodus.

heading: E Biblical-Exegetical Discussion
text: Here we perform an exegetical analysis of our results for each book. All data to which this analysis was applied is available online5.

heading: E.1 Genesis

heading: E.1.1 Semantics
text: The extraction of the features of P for Genesis overlaps with the work of characterizing the priestly stratum (Holzinger, 1893). Thus, the characteristic use of numbers in P (here, in descending order of importance, the algorithm considered the terms 100, 9, 8, 5, 3, 6, 4, 2, 7 as characteristic of P) appears mainly in the genealogies, e.g., Gen 5 and 11, but also in the use of ordinal numbers to give the months and in the definition of the dimensions of the tabernacle. The term “year” (!הנשׁ) is used in both dates and P genealogies, and the term “day” (!Mוי) demonstrates a similar calendrical concern. Furthermore, in the genealogies, we find the names of the patriarchs considered to belong to P ( !חנ, !לאללהמ, !Nניק, !Kונח, !חלשׁותמ, !שׁונא, !Mדא, !תשׁ, !לאעמשׁי, !Kמל, !וער, !גלפ, !גורס). The term “son” (!Nב) appears in genealogies but also in typical P expressions such as “son of X year” (!הנשׁ Nב) to indicate the age of someone, “sons of Israel” ( !לארשׂי ינב), etc. The root !דלי (to beget or to give birth) is found in the genealogies in Gen 5; 11; Exod 6 but also in other P narratives of the patriarchs (Gen 16–17; 21; 25; 35; 36; 46; 48) which focus on affiliation. The term “generation” ( !רוד) is also recognized as typically P (Gen 6:9; 9:12; 17:7,9,12; etc. ), as well as the term “annals” (!תודלות), which serves to introduce a narrative section or a genealogy and. This term structures the narrative and genealogical sections in the book of Genesis (Gen 2:4; 5:1; 6:9; 10:1,32; 11:10,27; 25:12–13,19; 36:1,9; etc.). The terms “fowl” ( !Pוע), “beast/flesh” (!רשׂב), “creeping” (!שׂמר), “swarming” (!Zרשׁ), “living being” ( !היח שׁפנ), “cattle” (!המהב), “kind” (!Nימ) are found in the typically P expression “living creatures of every kind: cattle and creeping things and wild animals of the earth of every kind” (Gen 1:24; cf. Gen 1:25-26; 6:7,20; 7:14,23; etc.). These expressions are often associated with “multiplication” ( !הבר), an essential theme for P that also appears in the blessings of P narratives as in Gen 17; 48; etc. The term “being” (!שׁפנ) is also used in P texts to refer to a person, e.g., in Gen 12; 17; 36; 46. As for the term “all” (!לכ), it
5https://github.com/YoffeG/PnonP
is used overwhelmingly in both P and D texts. In P texts (Gen 1:27; 5:2; 6:19), humanity (!Mדא), in the image ( !Mלצ) of God, is conceived in a dichotomy of “male” ( !רכז) and “female” ( !הבקנ). The root !רכז in its second sense, that of “remembering”, also plays a role in the P narratives (Gen 8:1; 9:15–16; 19:29; Exod 2:24; 6:5) when God remembers his covenant and intervenes to help humanity or the Israelites. The covenant ( !תירב; cf. also Gen 9; 17), the sign of which is the circumcision (!לומ) of the foreskin (!הלרע) is correctly characterized as P. According to P, God’s covenants are linked to a promise of offspring (!ערז; cf. Gen 17; etc.) and valid forever ( !Mלוע; Gen 9; 17; 48:4; Exod 12:14; etc.). The term “seed/descendant” (!ערז) is also used by P in the creation narratives in Gen 1. The term “between” (!Nיב) is used several times to indicate the parties concerned by the covenant in Gen 9 and Gen 17. The term is also found frequently in Gen 1 in the creation story, where creation is the result of separation “between” ( !Nיב) different elements – presenting God as the creator is not typical of a national god whose role primarily guarantees protection, military success, and fertility. The transformation of the God of Israel into a creator God appears only in the exilic or postexilic texts. Thus, the root “to create” (!ארב) is rightly associated with P (Gen 1:1-2:4; 5:1-2). The use of divine names is particular in the priestly narratives. “God” (!Mיהלא) is the term used in the origin stories (Gen 1–11), “El Shaddai” for the patriarchs (Gen 12–Exod 6), and finally, “YHWH” from Exodus 6,2-3 on. Here, the algorithm did understand a particular use by P of the term “God” (!Mיהלא). One of the differences with Holzinger’s list is the fact that the algorithm considers the terms Noah (!חנ), flood (!לובמ), and the ark (!הבת) as typically P. This is probably because the flood narrative is much more developed in P than in non-P or because the semantic environment is attached to other P expressions. Nevertheless, all three terms appear in non-P texts as well. The term “daughter” (!תב) should be considered P not because of its frequency, which is admittedly somewhat higher in the P narratives of Genesis, but probably because of its semantic environment. Thus, the term appears in the expression “sons and daughters” (!Mינבו תונב), which is very frequently used in Gen 5; 11. The preposition “after” ( !רחא) appears in the expression “after you” (!Kירחא) in the promise to Abraham in Gen 17 or the expression “after his begetting” (!ירחא ודילוה) in the genealogies in Gen 5; 11. The appearance of the term “to die” (!תומ) as a characteristic of P is explained by its presence in the genealogies of Gen 5; 11 but also in the succession of each of the generations of the patriarchs. Finally, the terms “water” (!Mימ) and “heaven” (!Mימשׁ) play a major role in the creation narrative P (Gen 1:1-2:4) and the flood narrative (Gen 6-9*). These two terms also appear in Exodus, where water is mentioned in the account of the duel of the magicians (Exod 7-9*), in the passage of the sea of reeds which is paralleled in the creation of Gen 1 (Exod 14), and as a means of purification during the building of the tabernacle (Exod 29-30; 40). This latter function of water probably builds its symbolism in the other narratives. The term firmament ( !עיקר) is associated with heaven and appears only in the creation story P of Gen 1 but is of little significance elsewhere. On the non-P side, terms like “Joseph”, “pharaoh”, and “Egypt” are non-P features since the story of Joseph is non-P. Similarly, the presence of “Jacob” is explained by an account of only a few verses for this story in P as opposed to several whole chapters for the non-P account of Jacob. The terms “brother” (16P /179nonP ), “father” (19P /213nonP ), and “mother” (4P /33nonP ) as nonP features can be understood by a greater emphasis on family in the original patriarchal accounts, whereas P emphasizes genealogy. The terms “master” (!Nודא), “slave/servant” ( !דבע), and “boy/servant” (!רענ) reflect the hierarchical structures of the household of the wealthy landowners in the narratives of the patriarchs but are of no interest to the priestly editors. Similarly, non-P texts show more interest in livestock, with terms such as camel (!למג), donkey (!רומח), or small livestock ( !Nאצ) considered non-P features. The dialogues are more present in the non-P stories than in the P stories. Thus the terms that open the direct discourses “speak” ( !רבד), “say” (!רמא), and “tell” ( !דגנ) are considered typical non-P terms as well as the set of Hebrew propositions in direct discourse ( !המ, !התע, !ה, !יכ, !Mג, !הנה, !לא, !אנ, !Mא, !אל). The term “man” ( !שׁיא) can be used in many ways: man, husband, human; someone. Its use and expressions using it are significantly more frequent in non-P texts (42P/213nonP). This may be an evolution of the language rather than a deliberate or theological change on the part of P. Finally, for the terms “to enter” (!אוב) or “to go” (!Kלה) to be characteristic of non-P, this may reflect a stronger interest in place, in travel in the original
texts probably composed to legitimize sanctuaries or as etiological narratives whereas these aspects are less marked in the P texts.

heading: E.2 Exodus

heading: E.2.1 Semantics
text: For the P-texts of Exodus, the algorithm has extracted the semantic features of the tabernacle construction in Exod 25-31; 35-40 but does not give features of the P-texts that would be found elsewhere. We find in the features: the different names of the holy place, “the holy one”, “the dwelling”, “the tent of meeting”; the materials used for the construction, “acacia wood”, “pure gold”, “bronze”, “linen”, “blue, purple, crimson yarns”, etc.; the spatialization, “around”, “outside”; the dimensions, “length”, “cubit”, “five”; the components, “altar”, “curtain”, “ark”, “utensils”, “table”, and YHWH’s orders to Moses, “you shall make”. Thus, the algorithm has a good understanding of the terms specific to the construction of the Tabernacle but is not susceptible to a more general understanding of the characteristics of P in Exodus. The non-P features are more interesting. For example, the use of the word “people” ( !Mע) appears primarily in the non-P texts because the priestly redactors usually preferred to use the term “assembly” ( !הדע). The word “I” in the long form (!יכנא) is considered non-P because the word ’ny, the short form, appears in P texts. The expression “to YHWH” does appear 24 times in non-P texts, e.g., “to cry out to YHWH”/“to speak to YHWH”/“to turn to YHWH”, whereas P avoids the expression. This is easily understandable by a desire to give YHWH the initiative in all interactions. In P, it is he who demands, commands, and speaks. There are few dialogues. As for the terms Egypt ( !Mירצמ) and Pharaoh (!הערפ), they are indeed quantitatively more frequent in the non-Priestly texts of Exodus (respectively 36P/139nonP et 26P/89nonP) as in Genesis.

heading: E.2.2 Grammar
text: As we have already seen, non-P texts more often adopt the protagonists’ point of view by including dialogues or their thoughts, whereas P texts prefer a third-person narration. One of the consequences is the privileged use of 3rd singular or plural suffixes, unlike non-P texts where 1st singular or 2nd singular suffixes are more often used. Moreover, the massive use of the 3rd person in P texts can also be explained by the presence of
pleonasms which use a form with this suffix: !ומע, !ותא, etc. (Holzinger, 1893). Concerning verbs, the form Qal or Piel, qatal in the 2nd masculine singular, is prevalent in P texts. This is understandable because of P’s theology, according to which God orders using the second person, and then the protagonists carry out according to YHWH’s order. On the side of the non-P texts, the narrative form, i.e., Qal, wayyiqtol in the 3rd masculine singular, is significant, although these forms are also very present in P texts. Another peculiarity is the use in non-P of “name in the constructed form + place name”. P seems to have avoided topical constructions because of reduced interest in localizations. The remaining terms are persistent elements. Further analysis would be needed to understand the relevance of the distinction made by the algorithm.

heading: E.3 Summary
text: As we can see, the pipeline could extract typical features of priestly texts in Genesis, easily recognizable for a specialist. In addition, other P features have also been found that may be specific to a single narrative, correspond to repeated use of an expression, or be a significant theological theme such as water. On the other hand, the features of non-P texts do not indicate a coherent editorial milieu or style but rather allow us to better distinguish between P texts and non-P texts by particular theological or linguistic features. The data provided by the algorithm allow for the detection of particularities that require an explanation. More detailed investigations than those presented above are necessary to better understand specific instances of the results. For the texts of Exodus, the excessive importance of the chapters devoted to the construction of the Tabernacle (Exod 25-31; 35-40) did not allow us to obtain satisfactory results in the characterization of P, which could indicate an originally independent document. Nevertheless, the characterization of non-P texts is relevant, as well as the results on grammar.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Left blank.
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.
3 A3. Do the abstract and introduction summarize the paper’s main claims? Left blank.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? Not applicable. Left blank.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.
C 3 Did you run computational experiments? Left blank.
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Left blank.
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Left blank.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. Left blank.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: It is important to note that this work aims to set a benchmark for future endeavors using strictly traditional machinery throughout our analysis. To ensure collaboration with biblical scholars, our methodology allows for full interpretability of the exploration process (see §D. 4). This has threefold importance: (1) the field of text stylometry, especially that of ancient Hebrew texts, has hitherto been explored statistically and computationally to a limited extent, such that even when utilizing conservative text-embeddings, such as in this work, considerable insight can be gained concerning both the quality of the analysis and the philological question. (2) Obtaining benchmark results using traditional embeddings is a pre-condition for implementing more sophisticated yet convoluted embeddings, such as pre-trained language models (e. g. Shmidman et al. , 2022) or self-trained/calibrated language models (Wald et al. , 2021), which we intend to apply in future works. (3) Finally, the interdisciplinary nature of this work and our desire to contribute to the field of biblical exegesis (and traditional philology in general) requires our results to be predominantly interpretable, such that they can be subjected to complementary analysis by scholars from the opposite side of the interdisciplinary divide (Piotrowski, 2012)