TopWORDS-Poetry: Simultaneous Text Segmentation and Word Discovery for Classical Chinese Poetry via Bayesian Inference
Authors: Changzai Pan
Feiyue Li
Ke Deng
Venue: EMNLP 2023 Main (2023)
Keywords: Word Discovery
Text Segmentation
Classical Chinese Poetry
Bayesian Inference
Unsupervised Method

Abstract: As a precious cultural heritage of human beings, classical Chinese poetry has a very unique writing style and often contains special words that rarely appear in general Chinese texts, posting critical challenges for natural language processing.
Little effort has been made in the literature for processing texts from classical Chinese poetry.
This study fills in this gap with TopWORDS-Poetry, an unsupervised method that can achieve reliable text segmentation and word discovery for classical Chinese poetry simultaneously without pre-given vocabulary or training corpus.
Experimental studies confirm that TopWORDS-Poetry can successfully recognize unique poetry words, such as named entities and literary allusions, from metrical poems of《全唐诗》(*Complete Tang Poetry*) and segment these poetry lines into sequences of meaningful words with high quality.

WoRDSS-S-::::::: Simultaneous Text Segmentation and Word Discovery for Classical Chinese Poetry via Bayesian Inference
Changzai Pan
Feiyue Li Department of Chinese Language and Literature, School of Humanities, Tsinghua University
Ke Deng"
Center for Statistical Science & Department of Industrial Engineering, Tsinghua University
Center for Statistical
Science & Department of Industrial Engincering,
Tsinghua University kdengêtsinghua www.nnggu..uuuu...o
pc21 80maals...................lll. tsinghua. .du. en
lifeiyue@tsinghu.. edu.
Abstract
of syllables. Containing 1.3 million Chinese char- acters in total, the collection of metrical poems in CTP (referred to as P hereinafter) is the most essence of Tang poetry and our focus in this study. A typical metrical poem in CTP is composed of ei- ther 4 or 8 lines, each of which contains either 5 or 7 Chinese characters, and can be classified into 4 cat- egories Pax5, P4x7, P8x5 and P8x7, where Pmxn represents the poems with m lines and n Chinese characters in each line. In P, there are 3,225 po- ems in P4x5, 9,820 poems in P4x7, 13,536 poems in Pax5. and 7,646 poems in Psx7- Fig (d)-(g) show 4 famous poems (each for one of the 4 cate- gories) by the famous poet Du Fu (#t H 712-770 A.D.) of the Tang Dynasty, with the word bound- aries (highlighted by symbol 'f') manually labelled by domain experts. For the majority of poems in CTP, however, no word boundaries are labelled yet.
As a precious cultural heritage of human beings, classical Chinese poetry has a very unique writ- ing style and often contains special words that rarely appear in general Chinese texts, posting critical challenges for natural language process- ing. Little effort has been made in the liter- ature for processing texts from classical Chi- nese poetry. This study fills in this gap with SAWORDS-Poetry an unsupervised method that can achieve reliable text segmentation and word discovery for classical Chinese poetry si- multancously without pre-given vocabulary or training corpus. Experimental studies confirm that TopWORDS-Poce
can successfully rec- ognize unique poetry words, such as named entities and literary allusions, from metrical poems of i (Complete Tang Poetry) and segment these poetry lines into sequences of meaningful words with high quality.
The absence of word boundaries posts critical challenges in literary study of classical Chinese poetry at a large scale, and leads to a great appeal to develop efficient algorithms to achieve reliable text segmentation and word discovery for poetry texts(Li et al., 2016; Liu, 2019). However, such a task is very challenging, because classical Chinese poetry is composed of special domain texts with many unique features.
1 Introduction
Expressing complicated thoughts and feelings in refined language, poctry is the most shining pearl on the crown of human literature. As a special literary style, poetry has unique characteristics in rhythm, metre and language use, making it a critical challenge to process poetry texts via algorithms (Agirrezabal et al., 2016; Fang et al., 2009; Ahmed and Trausan-Matu, 2017).
First, classical Chinese poetry has very unique writing style and often contains special words that rarely appear in general Chinese texts, such as unique named entities and literary allusions. More- over, to express complicated thoughts and feelings with few words, poets were often forced to com- press long phrases into shorter "pseudo words" for putting them into short poetry lines. Although these "pseudo words" are phrases instead of words from the grammatical point of view, many of them are treated as "poetry words" in practice because they have specific meanings and function as words (Liu, 1966; Li et al., 2016). Due to these facts, word discovery becomes a critical problem in processing
Rising the Spring and Autumm Period (770- 476 B.C.) and flourishing the Tang Dynasty (618- 907 A.D.), classical Chinese poetry has a long his- tory and profound cultural heritage. Composed of nearly 50,000 poems by over 2000 poets of the { (Complete Tang Poetry, CTP) displayed in Fig (a) is the ideal object for studying classical Chinese poetry. Fig (b) and (c) visualize the 200 poets with the most poems and the 300 most frequent Chinese characters in CTP. Among the ncarly 50,000 poems in CTP, 34,227 are metrical poems that have recognizable patterns
Tang Dynasty,
Corresponding author.

---

KB
4
16.
*
34
[a] APRRR
[b]
I M Maeting Li G Guinian n jangnon
SEGGO Climbing Yuoyang Tower
S Shu's Minsttr
The Plan the Eight Formations
1y PR/E Wh dande covpreed kingrien chree,
BEEE/ / W /1/0/:1 / FL. To the Toda fae of the Priccr
H / I hrurrd long not Dongting"
4 HOL////O////TTT /R The staine holl to the Miniater, where con in found?
PA Lis Dooe complesed the Plan of the Eight Formstions
/ 92 RE/BE)
M ///// outiide the volls Brocads City. Kiore the eypeess tre stanco danse.
at the hoad of the hill of Out Nine hoard sm mey simn.
and this day I cliad Tooyang Townr.
/T/////////
TE / AR rollly froe that in Henggnn the scenery io fine. 18 1872/ 45 // ////////FF Fi-
BARE W//B///
The River floes ot, the stoned not tiza,
No and Cha to it apart in the southcost.
Half hiding the stalrs, ayshies prasoeee take
milors spring. RADE PA///////// ////////// Si.
bit
Beaven Earth Fleat day night
yollow ortoles give
Leaves nnught.
Fae moll' (d)
(e)
SERR /B8////// /8/-7,
///0// F/al,
Frezs kin and friends not eingle word.
Thrioe enllee urand repentadly: place
Pattern
Freg
Ratio
the world. 380 18 / / /E/--
2-1-2
1525
50, 83%
E /1/ old and stchi do have and Stery Snat..
Five Five-chrracter lines
for rei gm founding
2-2-1
1162
38.
custaining: on old officer'
heart.
other
224
10. 44%
1& M tll Las -ootoss
H/ #:30 //EEE victorious, the not hisselt diot.
2-2-1-2
1670
54. 77%
barrier
Seven-character lines
2-2-2 2-1
1071
34. 000
H8 HT/ /SSHN// Isor the rulling,
Re JL /8/ 85 It alsore makes Iold Ioldrapirite Fill their clechne with
other
259
11. 23%
tewms ing from
(h)
IF)
(5)
Figure I: Illustration of classical Chinese poetry in ( (Complete Tang Poetry, or CTP). (a) The book series of CTP. (b) The 200 poets with the most poems in CTP. (c) The 300 most frequent characters in CTP. (d)-(g): Four famous poems by the famous poet Du Fu (#L 712-770 A.D.) belonging to P4x5, P4x7, Psx5 and Pax7 respectively, with word boundaries (highlighted by symbol 'Y') manually labelled by domain experts. (h) Major segmentation patterns of metrical poems in CTP discovered hased on 1,000 manually segmented poems that were randomly sampled from the pool of all metrical poems in CTP via stratification sampling.
classical Chinese poetry.
proportion of these segmentation patterns in P, we randomly sampled 1,000 poems from P (referred to as P1000) by stratification sampling (i.e., 250 poems for each of the 4 poem types in P), and manually segmented them as the demonstrating examples shown in Fig (d)-(g). Figure (h) sum- marizes the obtained results, confirming that most poetry lines indeed precisely follow these 4 seg- mentation patterns.
Second, to keep proper rhythm, text segmenta- tion of classical Chinese poetry lines follows few major segmentation patterns. Previous studies have revealed that 5-character poetry lines mainly follow two segmentation patterns 2-1-2 and 2-2-1, while 7-character poetry lines mainly follow another two segmentation patterns 2-2-1-2 and 2-2-2-1 (Chen, 1979; Yip, 1997; Feng, 2019; Liu, 2019). In this study, we refer to these 4 major segmentation pat- terns as @212, @221 @221 and @2221, respectively. Apparently, @2212 and @212 correspond to the same segmentation pattern for poetry lines of different lengths, and form a general segmentation pattern 2-12- Similarly, @2221 and 0221 form another gen- eral segmentation pattern 6.21- To investigate the
Third, on the other hand, however, about 10% of poctry lines violate these major segmentation patterns due to appearance of long words contain- ing more than 2 Chinese characters, such as named entities and literary allusions. For example, in the poems shown in Fig 1 (d)-(g), there are 5 po- etry lines that violate the 4 major segmentation

---

patterns because of long words with 3 Chinese char- acters, Due to this phenomenon, text segmentation becomes a non-trivial issue.
posed for processing classical Chinese poetry. Li et al. (2016) proposed recognize meaningful words in classical Chinese poetry by filtering out fake words from word candidates via a modified support vector machine trained by manually la- belled training samples. This method relies heavily on manually labelled samples to capture features of poetry words, and completely ignores the segmen- tation patterns of poetry lines. Liu (2019) proposed to formulate the text segmentation problem of po- etry line into a pattern selection problem under the assumption that every poetry line must precisely follow one of the two general segmentation patterns 2+21 or 2+12. and achieve text segmentation by se- lecting the plausible pattern based on only the last three characters of each poetry line via weighted point-wise mutual information. Although such method takes good advantage of the segmentation patterns and performs reasonably well, it oversim- plifies the problem via a restrictive assumption and tends to miss all named entities and literary allu- sions with more than 2 Chinese characters.
These unique features of poetry texts make it in- appropriate to directly apply popular tools for pro- cessing general Chinese texts based on supervised learning, such as Jieba (Sun, 2012), StanfordNLP (Manning al., 2014), THULAC (Sun et al., 2016), PKUSEG (Luo et al., 2019), and LTP (Che al., 2021), to process classical Chinese poetry. As we will show in the result Section, due to the lack of training for poetry texts, these methods suffer from serious performance degradation in text segmenta- tion, and tend to miss many poetry words because of the limited ability in discovering unregistered words. And, it is not realistic to tune these tools for this specific task, because no qualified training corpus that contains enough well-segmented clas- sical Chinese poetry lines is publicly available yet, to the best of our knowledge. Some methods based on unsupervised learning, e.g.. TopWORDS (Deng et al., 2016), GTS (Yuan et al., 2020) and others (Goldwater al., 2009; Mochihashi et al., 2009; Pearl et al., 2010), are strong in discovering unreg- istered words, but suffer from poor performance on text segmentation in the current scenario because they do not take the special features of poetry texts into consideration.
To better resolve this challenging problem, we need an algorithm that can wisely utilize the segmentation patterns of poetry lines. is flexible enough to deal with situations where the regular segmentation patterns violated, and has strong word discovery ability recognize special poetry words with little training information. In this study, we fill in this gap with TopWORDS-POeete an algorithm with all these desired features. Extend- ing the Bayesian framework of TopWORDS-SSE proposed by Pan et al. (2022) to a more sophis- ticated hierarchical Bayesian model to reflect the constraints on text segmentation due to the major segmentation patterns, H-WORDS-Poetrr is able to achieve effective text segmentation and word dis- covery simultaneously for classical Chinese poems with metrical patterns.
As pointed out by Pan et al. (2022), when pro- cessing domain-specific Chinese texts like poetry that potentially contain many unregistered words, word discovery and text segmentation become two critically important tasks entangled with each other. Most existing methods for Chinese text segmenta- tion work under the support of a pre-loaded vocab- ulary, either explicitly given apriori or implicitly encoded in a pre-trained large language model. If the pre-loaded vocabulary covers the target corpus well, these methods typically perform reasonably well; once the underlying vocabulary of the target corpus exceeds the pre-given vocabulary seriously, however, we often observe significant performance degeneration of these methods, as demonstrated in Pan et al. (2022). Because classical Chinese poetry contains many unique named entities and literary allusions that never appear in general texts and follow specific segmentation templates, meth- ods without considering these features tend to miss many poetry words and segment poetry lines in an inappropriate manner.
An experiment study on metrical poems in CTP confirms that NEWORRDS-Poetee can correctly segment these poetry texts, with various named entities, literary allusions and special poetry words effectively recognized. Compared to existing meth- ods in the literature, TopWORDS-Pon PPeery is supe- rior processing classical Chinese poetry with significant performance margin. The outputs of TopWORDS-Poetry may promote the applications of AT approaches the study of classical Chinese literature and inspire more research efforts to study classical Chinese poetry via the paradigm of digital
In the literature, few methods have been pro-

---

humanity.
complete initial dictionary D {w W2, W2..... ..... WN) by enumerating all qualified word candidates in This can achieved in three steps: first, we enu- merate all possible sub-strings in 7 whose length I TL and frequency 2 TP: as suggested by Top- WORDS, to form a collection of word candidates denoted as Dy; second, we segment each poetry line Тj according to the two general segmentation patterns (+21 and 2*12 respectively get col- lection of 7y-specific word candidates Dj. and as- semble them into an additional collection of word candidates Dp D finally, we unite DE g and Dp to get paper, we assume that the initial dictionary D gen- erated in this way covers the underlying vocabulary Dy of interest, i.e., Dy C DD
2 TopWORDS-Poctry
Proposed by Pan et al. (2022), TopWORDS-Sey is an effective method for processing open-domain achieve high-quality text
Chinese texts that segmentation and word discovery simultaneously. Modelling the target texts by the uni-gram language model with an over-complete initial dictionary D obtained by enumerating all qualified word candi- dates, TopWORDS-Seg utilizes the word bound- aries suggested by pre-given text segmenter (e.g., PKUSEG) as prior information to guide usage fre- quency estimation and pruning of word candidates in D under Bayesian framework, and finally achieves text segmentation based on the pruned word dictionary with respect to the prior word boundarics.
Dr U Dp. Throughout this
Define 0 {0 WED, where An. is the usage probability of word candidate w in T. Under the uni-gram language model with D as the vocabu- lary, we have the following likelihood function for poetry lines in T when D, B and are given:
As demonstrated by Pan et al. (2022), the Bayesian framework of TopWORDS--S Seg inte- grates the advantages of strong word discov- erer TopWORDS and an effective text segmenter PKUSEG, leading stronger word discoverer and more robust text segmenter for open-domain Chinese texts.
P(T D:O,B)
II PTT D;), j=1
II WED
0 (T,,B))
(1)
In this section, we extend the Bayesian frame- work of TopWORDS-Seg with more flexible way to utilize prior information text segmentation. Instead of using the word boundaries suggested by a pre-given text segmenter, e.g., PKUSEG, as prior information to guide learning, we encode the constraints on text segmentation due to various seg- mentation patterns (i.c., Q+12 and (+21) into a more sophisticated prior distribution, coming up with a hierarchical Bayesian model to infer the unknown segmentation pattern of each poetry line.
where nww (T,, By) is the number of appearance of in Tÿ under segmentation Bj-
To learn the unknown model parameters (0, B)
via Bayesian inference, we follow Pan al. (2022)
to adopt the following independent prior:
π (0, B) =
II (B,),
where
2.1 The Bayesian Framework
(0) Dirichlet a))
(2)
Following the notations in Pan et al. (2022), we let T (Ti..... } be the unsegmented Chinese poctry lines of interest, A {a1.-- .arr be the set of Chinese characters that appears in T. and D be the underlying vocabulary T. For each poetry line T3 of length L,, let B- = (b.n...., b,t,)) be its word boundary profile, where byz = 1 means there is word boundary behind the /th character of Tj. and by: 0 otherwise. Apparently, B, de- termines how T, is segmented. Our goal is to learn the unknown vocabulary Di and word boundary profiles [Bi..-. B₂) from T.
is the same conjugate prior as in TopWORDS- with a = { being the parameter vector of a n-dimensional Dirichlet distribution. Here, we choose to specify Ow = for all w E DD lead- ing to a non-informative plat prior distribution for 0, as suggested by Pan et al. (2022). The prior distribution of B, i.e., T [) 11 क (B;), how- ever, needs to be properly specified to fit the special features of poetry lines.
Different from general Chinese sentence, whose segmentation usually does not follow ob- vious patterns, a poctry line this study typically follows few major segmentation patterns as we
To discover Dy, we first construct an over-

---

have pointed out previously. To be concrete, let I.. be the collections of segmentation patterns for poetry lines of length L, and B& = (bg1.-.. ......... boL) be the word boundary profile of segmentation pattern E L. where b.. = 1 if puts a word boundary behind the 1-th Chinese character of a poetry line, and 0 otherwise. In this study, we have two collections of segmentation patterns
patterns in Ps and 9, respectively. In case that the proportion of segmentation patterns in P is precisely known, we can simply specify X to the known proportion vector to reflect the prior knowl- edge. In practice, however, we usually do not have such knowledge. In such scenario, a reasonable strategy is to specify the following conjugate prior distribution for X and X
P3 {221,, @212) and 3 (e2221 (2212),
π (A.)) = Dirichlet 2o 1g,,, π (0)) = Dirichlei(A 3gg,), (6)
(5)
with the following word boundary profiles:
and infer them along the learning process. Such Bayesian framework leads to hierarchical Bayesian model (Good et al., 1966; Gelman et al., 2013) for By's with (B ( G 2. 1)) hyper- parameters. Here, we choose to specify 3 and B 8 with constant vectors whose elements all equal to 1.
Ben (0, 1, Beans (0, 1, 1, 1, 0,
Ben (0, 1,), B Bezzi (0, 0,,1,,
To reflect the impact of these segmentation pat- terns text segmentation, we use the following mixture distribution as the prior distribution of By:
xBB))
Σ Xe (Byle)) PEPL
Finally, we come up with the following joint prior distribution for = {OwlwED. À (A ) and B = [Byhsssnn
where X = (ee) LE a probability vector defined over segmentation patterns in L, and
π - ((A) II (B, A1
j=1
T( (Bylg)
II Binary(byleg)
where
x (A) = (1 T (A
with Per being e-specific probability value deter- mined by Be the word boundary profile of segmen- tation pattern g. Here, we specify
with (0). . (B, (),,)) and π (Xg²) defined
Eq. (2)-(6), respectively. Accordingly, have the following posterior distribution of (0, X, B) given and D based on the Bayes theorem:
Pal (1 R) bot + K. P: 1< Leg.
(4)
Log
f(0, B, B D,T)) & π (0, (, PPT D)O,B).
(7)
where E 1) is the probability of placing a word boundary at each position randomly by pseudo segmenter, and к E 10 is smooth- ing parameter mix the segmentation pattern and the pscudo segmenter. In this study, we set p 0.5 by default, and leave k as a tunable param- eter. Apparently, н closer to 0 means a stronger prior preference to follow the segmentation pattern 0. Such a prior setting sufficiently considers the constraints in segmenting poetry lines due to the segmentation patterns, while gives us the flexibil- ity to violate these constraints, leading to ideal framework for processing classical Chinese poetry. Because there are two collections of segmenta- tion patterns, i.e., D3 and Pr. in this study, we need two groups of parameters A and A to re- flect the prior proportion of different segmentation
2.2 Word Discovery and Text Segmentation
Based on the posterior distribution in Eq. (7), word discovery and text segmentation can achieved by following the strategy recommended by Deng et al. (2016).
0.5
First, we maximize the marginal posterior distri- bution below with B integrated out
ʃ(0, A D,T) 1
by the EM algorithm (Dempster al., 1977) get the Maximum a Posteriori (MAP) estimate of (A, A) below:
(0. Å) arg max f(O,A (DT).
(8)
(0.X)

---

Second, get the significance score the of each word candidate w E D by calculating the likeli- hood ratio between the full model with all word candidates in D and the reduced model with w removed from D:
To ease the computation challenges, we pro- pose to introduce an additional group of auxiliary latent variables Z = {2j)issn with Z stand- ing for the underlying segmentation pattern of po- etry line Tj. With the help of Z, we can expand the target posterior distribution f(0, X, B D. T) in Eq. (A.1) to higher dimensional distribution F0 Z. D.T), which takes f(0, X, B D, D) as its marginal distribution and is computa- tionally more friendly. and thus implement com- putations based on the expanded posterior distri- bution instead of the original onc. We leave all these computation details to Appendix A-C, with intuitions behind some key formulas explained in Appendix D.
II P(2) D; !T P()) ow=
Vi log
where 0 (w==o) is modification of by setting d. O with all the other elements of l unchanged, and
P(T ID;0,
E <B3 λ) ([T3] By),
=
B,c
- Σ Σ
A क (B, e) P(T, 0; Bj)-
BycB, pE.
2.4 TopWODDS-Poetry Pipeline
Because a larger the means that word candidate is more important for fitting the poetry texts and thus more likely to be meaningful word, we achieve word discovery by keeping only word candidates whose th 2 Tw- Since ~2 x² asymptotically under the null hypothesis that the reduced model with w removed the true model, we set the threshold Ty to the (1 - 0.05
Assembling the above ingredients, we come up with the TopWORDS-POety algorithm composed of four stages: a dictionary initialization stage to generate the initial word dictionary D, a prior spec- ification stage specify prior distributions based on pre-given segmentation patterns and hyper- parameters, word discovery stage to estimate (0, A) and prune the initial word dictionary into a more concise final dictionary DF. and a text seg- mentation stage to segment poetry lines based on the refined model. Figure 2 illustrates the algorithm pipeline of TopWORDS- Poetry.
N -quantile of x2 distribution by default, based on the Bonferroni correction principle for multiple hypothesis testing. Third, based the MAP estimate (0, Å) of the pruned word dictionary D obtained in the previous step, we can segment T, according
In this pipeline, TL, TF and k are three control pa- rameters to be specified by the users. Since words with more than 3 Chinese characters are very rare in classical Chinese poetry and it is not realistic to discover rare words that appear only once, we set Tz. = 3 and TP by default. Following the strategy proposed by Pan et al. (2022), we choose a larger x in the word discovery stage (referred to as Kg)) to encourage the discovery of unregis- tered words, and smaller x in the text segmenta- tion stage (referred to K2) to show more respect to the segmentation patterns. In this study, we find that TopWORDS-Poetr works well by set- ting KW 0.5 and n, 0.001, and thus use these values as the default setting.
B, max P[B| D, D,Tr;0,A. BEB,
(10)
where By stands for the set of all possible word boundary profiles of Tj. A more robust strategy recommended by Deng et al. (2016) is to calculate
7/1
Σ ba. D,T;O,A), BEB,
(11)
and put word boundary behind the Ith position of Tj if 7y is larger than pre-given threshold TS.
2.3
Computational Issues
Although TopWORDS- follows a similar strategy as in TopWORDS-SSC -Seg to achieve word discovery and text segmentation, it involves more computational challenges due to the more compli- cated prior distribution 5 (B,), which is mixture distribution with unknown parameter A
Note that ToPWORDS-Peoot -Pettyy is unsupervised method in nature, because it only takes set of segmentation patterns (i.e., P, and 3r) and few hyper- paper--prameters as inputs, and does not need any training corpus or pre-given vocabulary.

---

Dictionary intialization Stage Word candidates 10
Parameter Estimation Stage
Poetry lines
Sub-string
Initial dictionary Dy '2p
Posterior distribution P A. 81 B1)-D) (7100.000
Parameter estimation
Word candidates 10p
Seamented by patterns
Boundary profiles
Prior word discovery c(8. a(8.2.81k.)
Bayestan framwwork
Estimated parameters R,
Sogmentation patterns (
Prior distributiin (1A)
specilication
Final dictonary Dy
selection with
Kg
Prior distribution s(()
Prior for text segmentation x18,1,8
segmentation
Sagmented text &
Prior Specifictiion Stage
Teat Segmentation Stage
Figure 2: Pipeline of the TopWORDS-Poety algorithm
2.5 Considering Correlation within Couplet the TopWORDS-Poeny algorithm, we assume that the choice of segmentation pattern com- pletely independent across different poetry lines. In practice, however, a metrical poem with 4 or 8 lines is composed of 2 or 4 couplets, each of which contains two poetry lines with positively correlated segmentation patterns (see Fig E1 for detailed ev- idence). Such fact means that can slightly modify TopWORDS-Poce Peetry for more efficient algorithm by modelling the correlation accordingly. Hereinafter, we refer to the modified TopWORDS- Poetry algorithm as TopWORDS-POetry"
picks up segmentation pattern from (e-12, (+21) for each poetry line. We also include RandomPat- tern in our experiment.
Furthermore, we also evaluate the performance of GPT-style methods on segmenting classical Chi- nese poetry by asking ChatGPT (model version: gpt-35-turbo) to segment all poetry lines in the test- ing dataset P1000. Appendix F reports the prompt designing for calling ChatGPT for this task in de- tail.
3.1 Performance Evaluation Criteria
Because no gold standard publicly available for text segmentation of poems in CTP, we utilize the manually segmented poems in P1000 as the test data for performance evaluation. Randomly sampled from P, the pool of all metrical poems in CTP, via stratification sampling, P1000 covers 250 poems for each of the 4 poem types of interest and is group of representative samples from P. Let Trest be the indices of all poetry lines in P1000. B = {B; be the true word boundary profiles of poetry lines in Trest and Vreat be the true vocabulary of Tiest obtained by collecting all words in the segmented poetry lines in Treat-
3 Experimental Study
We applied TopWORDS-Poete on the poetry texts in P, the collection of all metrical poems in CTP, under the default setting of тL 3. TF 2. p 0.5, Kg = 0.5, K₂ 0.001, TΦ 0.05 and TS 0.5. For comparison purpose, we also applicd competing methods to the same data set, including Jieba (Sun, 2012). StanfordNLP (Manning et al.. 2014), THULAC (Sun et al., 2016), PKUSEG (Luo et al., 2019), LTP (Che et al., 2021), TopWORDS (Deng et al., 2016) and TopWORDS-Seg under their default settings.
For each competing method, let Va2 be the dis- covered vocabulary based all poetry lines in T, B {B;}eT be the predicted word boundary profiles, and V,, the collection of words that ap- pear in the segmented poetry lines Trest based on Biest = B JeT We measure its performance on word discovery by discovery recall R((.() and segmentation recall (!l.1) defined as following:
Unfortunately, however, no executable program is available for either the modified SVM method (referred to as ModSVM) for discovering poetry words by Li et al. (2016) or the pattern selection method (referred to as SelectPattern) for segment- ing poetry texts by Liu (2019). Although it is straightforward to implement SelectPattern by our- selves, is non-trivial to implement ModSVM due many subtle details. Therefore, we only include SelectPattern the experimental study. Moreover, Liu (2019) also proposed a baseline method (re- ferred to as RandomPatter here) that randomly
n ((.) test
test n ((.0) Veest
((+.1)
(1.)) R((e
https//ccht.opeeai.com/

---

Word discovery (1))
12
Text segmentation (Fs)
Method
[Val
R2
[Va]
P1o08
P250
LTP
39K
59.33%
57.32%
2.5K
19.28%
94%
20.00%
21.16%
18.72%
21.72%
19.65%
StanfordNLP
77K
77.04%
63.69%
6.0K
43.50%
45.54%
39.49%
40.45%
40.26%
41.07%
37.80%
THULAC
76K
77.35%
43.63%
6.7K
53.28%
35.03%
49.08%
50.27%
50.14%
49.25%
47.78%
PKUSEG
83K
81.82%
48.09%
7.6K
59.26%
36.31
52.17%
52.62%
52.96%
52.65%
51.20%
Jicba
71K
78.21%
4
6.8K
57.30%
37.58%
53.62%
56.40%
52.60%
54.23%
53.04%
TopWORDS
54K
77.96%
89.81%
8.5K
71.22%
89.49%
63.71%
63,34%
62.49%
65.08%
64.07%
TopWORDS-Seg
86K
83.27%
53.50%
8.6K
68.06%
44.90%
60.67%
60.26%
61.27%
60.35%
60.55%
RandomPattem
96K
95.12%
0.00%
9.7K
81.61%
0.00%
69.09%
63.62%
64.70%
72.80%
72.56%
SelectPattern
73K
94.19%
0.00%
88.71%
0.00%
12%
81.27%
81.57%
86.46%
85.91
ChatGPT
8.0K
77.12%
66.24%
73.48%
72. 19%
73.17%
71.99%
74.96%
TopWORDS-Poety
82K
96.19%
83.76%
9.3K
88.98%
62.74% 56.69%
84.48% 84.71%
81.50% 80.68%
81.95% 82.70%
86.49%
86.46%
TopWORDS-Poety"
82K
96.25%
82.80%
9.4K
89.36
86.85
86.63%
Table I: Performance of TopWORDS-Pooe Poctry word discovery and text segmentation for classical Chinese poctry with comparison to competing methods.
LaL L8 PE
F.M
1.7M
#
+ #
N
444 85 $
#
MEA
ST.
4
1b6
47
(a) Figure 3: Results of the Top 100 most significant discovered names (a), addresses (b), and literary allusions (c) with more than 2 Chinese characters discovered by TopWORDS-Poetry
(c)
where and I are natural numbers, and (e,)) - fw E V freq(w)= 2 tt len(w)= 2 4,
Patterns in Pg
Proportion
Patterns in Pt
Proportion
2-1-2
55.88%
2-2-1-2
59.57%
2-2-1
35.71%
2-2-2-1
30.92%
2-3
7.37%
2-2-3
8.82%
is a subset of V composed of selected words, with freq(w) being the raw frequency of string w in T and (en(w) being the length of w. Ra and R, eval- uate word discovery from different perspectives, and provide us a more comprehensive view on this task in this application.
3-2
1.03%
2-3-2
0.69%
other
0.01%
other
0.00%
Table 2: The distribution of segmentation patterns re-
ported by TopWORDS- for poctry lines in T.
For test data Trest, let F, be the Fi score of Btest with respect to BLest. We measure the performance on text segmentation for Trest by F.. In practice, by using poetry lines in P1000 and its subsets (e.g.. PAx5 páxx pAx7 and påx7 which cover differ- 250 250 250
methods word discovery and text segmentation for the CTP data with 10 competing methods, from which we can see: (1) tools for processing general Chinese texts, such as LTP, Jieba and so on, per- form poorly in this application; (2) TopWORDS recognizes most long words but at the price of rela- tive low performance on short words, and cannot segment poetry lines well, with TopWORDS-SOL Seg performing worse in most dimensions because the prior information provided by PKUSEG is mislead- ing: (3) SelectPattern achieves good performance on segmenting poetry lines in general, but misses all long words with more than 2 Chinese charac- ters and all irregular patterns for text segmentation
250 ent types of poems in it) as the test data, we can evaluate text segmentation performance at the over- all as well as more detailed levels.
3.2 Results
It took 80 minutes in cluster with 10 CPU cores (Platinum82, 2.5GHz) with a RAM of 28GB to finish all computations involved in this experiment. Table compares the performance of proposed

---

beyond the major patterns; (4) The Rs and Fs scores of ChatGPT concentrate narrow region of around 70%, leaving a wide performance mar- gin (>10%) with respect to SWWORDS-Poeryy (5) Popeoryyy and TopWORDS-Poery
classical Chinese poetry.
Additionally, we also report in Appendix G more detailed performance evaluation results on word discovery by word types (e.g., name, address, and literary allusion), where labelled words in test dataset P1000 are manually classified into different categories. Moreover, to investigate whether the ad- vantage of TopWORDS-Pooe Poetry over the other meth- ods is statistically significant, we further calculated the performance margin of SipWORDS-Poetry" between the second best competing methods in ev- ery performance evaluation dimension and tested the statistical significance of the obtained perfor- mance margin. Detailed results are reported in Appendix H.
however, both achieve balanced performance on word discovery and text segmentation that outper- forms all competing methods, with TopWORDS- Poetry" being slightly superior in most dimensions. These results confirm that TopWORDS-Poens is an effective word discoverer for poetry texts. Although Re of TopWORDS-Poetry which 71,,, is much smaller than 89.19% reported by TopWORDS, it is the third largest value in the column and outperforms SelectPattern with huge margin. We note that R!!23) 62.74% means that TopWORDS-Poene may miss 1/3 long words in Treat in the segmented texts. But, considering that still capture the majority of these long words in the discovered word dictionary.
Conclusions and Discussions
In this paper, we proposed TopWORDS Poeetr,, powerful unsupervised method for processing clas- sical Chinese poetry that can segment poetry lines wisely with respect to few major segmentation patterns, and recognize unique poetry words such as name entities and literary allusions successfully with little training information. Relying on hierar- chical Bayesian model to utilize prior information about known segmentation patterns of poetry lines, and rigorous statistical principles achieve param- eter estimation and model selection, TopWORDS- Poctry leads an elegant solution to challenging NLP problems in an interpretable way.
R(233) 83.76% for TopWORDS-Poeryy
The results of TopWORDS- Pooetry reported in Table 1 are under the default setting where hyper- parameter K2 is set to K3 0.001. In this setting, TopWORDS-Poetry reports a relatively low 22(2.3) score (~60%), with respect to the high R!2.2) score (~88%), because the low parameter H₂ 0.001 discourages long words in text segmentation. In practice, however, we can adjust the granularity of poetry line segmentation in TopWORDS-Poece by specifying larger Ka tends to reserve more long words in text segmentation, and thus leads to higher chance break through the restrictive segmentation tem- plates. Therefore, if we could accept little bit of decrease in the Fs scores from the current ~55 % to 880% we can get (223) that is as large as 80% by specifying K3 = 0.1 instead.
We hope this proposed method can serve as a use- ful tool for scholars in the field of Chinese literary or digital humanity to conduct in-depth study on classical Chinese poetry via distant reading. For ex- ample, the literary allusions discovered from CTP by TopWORDS-Poot -eeetry may lead to very exciting research in near future, which may extend the re- search efforts discussed by Moritz al. (2016); Sturgeon (2018); Burns et al. (2021). We also hope this work can promote more and more interactions between literary scholars and NLPers, and applica- tions of NLPP technology in cultural education and dissemination, helping more people to better enjoy the charm of Chinese traditional culture.
K, differently:
Table reports the segmentation patterns recog- nized by TopWORDS-Poere from T, confirming that SWORDD--PPetr can successfully recog- nize rich segmentation patterns and correctly es- timate their proportion. Figure 3 (a)-(c) visualize the most significant names, addresses and literary allusions with more than Chinese characters dis- covered by fopWORDS-Poetry suggesting that SSpWORDS-Poetrys indeed captures lot of mean- ingful words that are often missed by most compet- ing methods. These results confirm the superiority of the proposed method over competing methods and its ability as a powerful tool for processing
Limitations
The proposed method relies on regular segmenta- tion patterns in poetry lines to guide text segmen- tation and thus may fail to process poetry lines without such regular segmentation patterns. More- over, the proposed method tends to miss rare words

---

that appear only one or two times in the target cor- pus due its nature as an unsupervised approach, and may result in improper segmentation for poetry lines containing rare named entities.
Ke Deng, Peter K. Bol, Kate J. Li, and Jun S. Liu, 2016. On the unsupervised analysis of domain-speciff Chi- nese texts. Proceedings of the National Academy of Sciences, 11 113((2):615-- 654-
Bradley Efron and Robert J Tibshirami. 1994, An intro- duction to the bootstrap. CRC press.
Acknowledgements
Alex Chengyu Fang, Fengju Lo, and Cheuk Kit Chinn. 2009. Adapting NLP and corpus analysis techniques to structured imagery analysis in classical Chinese poetry. In Proceedings of the Workshop on Adapta- tion of Language Resources and Technology to New Domains, pages 27-34, Borovets, Bulgaria. Associa- tion for Computational Linguistics.
This research is partially supported by the Na- tional Scientific and Technological Innovation 2030 Major Project (No: 2020AAA0106501), the Guo Qiang Institute of Tsinghua University, and the Beijing Natural Science Foundation (Z190021). Changzai Pan was supported by China Scholar- ship Council during the early stage of this work. The authors would like to thank Miss Yafei Han, Miss Jiafei Song, Miss Linyue Zhang, Mr Yang- gang Zhang, Mr Ning Cheng, Miss Xiting Song and other members in Ke Deng's lab and Feiyue Li's lab for their great help on generating manually labelled data for performance evaluation.
Shengli Feng. 2019. Prosodic syntex in Chinese: theory and facts. Routledge.
Andrew Gelman, John B Carlin. Hal S Stern, David B Dunson, Aki Vehtari, and Donald B Rubin. 2013. Bayesian data analysis. CRC press.
Sharon Goldwater, Thomas L. Griffiths, and Mark John- son. 2009. A Bayesian framework for word segmen- tation: Exploring the effects of context. Cognition, 112(1):21-54.
References
Irving John Good, Ian Hacking, RC Jeffrey, and Håkan Törnebohm. 1966. The estimation of probabilities: An essay modern Bayesian methods. Synthese, 16(2).
Manex Agirrezabal, Iñaki Alegria, and Mans Hulden. 2016. Machine learning for metrical analysis of En- glish poetry. In Proceedings of COLING 2016, the 26th International Conference on Computational Lin- guistics: Technical Papers, pages 772-781, Osaka, Japan. The COLING 2016 Organizing Committec.
Xia Li, Bin Wu, and Bailing Zhang, 2016. Unknown word detection in Song poetry. In 2016 IEEE First International Conference on Data Science in Cy- berspace (DSC), pages 544-549. IEEE
Munef Abdullah Ahmed and Stefan Trausan- Trausan-Matu. 2017. Using natural language processing for analyz- ing Arabic poetry rhythm. In 2017 16th RoFduNet Conference: Networking in Education and Research (RoEduNet), pages 1-5. IEEE
Chao-Lin Liu. 2019. Onto word segmentation of the complete Tang poems. arXiv preprint arXiv: 10621.
James JY Liu. 1966. The art of Chinese poetry. Univer- sity of Chicago Press.
Patrick J Burns, James A Brofos, Kyle Li, Pramit Chaud- huri, and Joseph P Dexter. 2021. Profiling of inter- textuality in Latin literature using word embeddings. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 4907.
Ruixuan Luo, Jingjing Xu, Yi Zhang, Xuancheng Ren, and Xu Sun. 2019. PKUSEG: A toolkit for multi-domain Chinese word segmentation. CoRR, abs/ 1906. 1455.
Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford Cor natural language pro- cessing toolkit. In Proceedings of 52nd Annual Meet- ing of the Association for Computationll Linguis- tics: System Demonsirations, pages 55-60, Balti- more, Maryland. Association for Computational Lin- guistics.
Wanxiang Che, Yunlong Feng, Libo Qin, and Ting Liu. 2021. N-LTP: An open-source neural language technology platform for Chinese. Proceedings of the 2021 Conference on Empirical Methods in Nat- ural Language Processing: System Demonstrations. pages 42-49.
Matthew Y Chen. 1979, Metrical structure: evidence from Chinese poetry. Linguistic Inquiry, 10(3):371- 420.
Daichi Mochihashi, Takeshi Yamada, and Naonori Ueda. 2009. Bayesian unsupervised word segmentation with nested Pitman-Yo language modeling, In Pro- ceedings of the Joint Conference of the 47th Annual Meeting of the ACI. and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 100-108. Suntec, Singapore. Associa- tion for Computational Linguistics.
Arthur P Dempster, Nan M I Laird, and Donald B Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1):1 1-22..

---

Maria Moritz, Andreas Wiederhold, Barbara Paviek. Yuri Bizzoni, and Marco Büchler. 2016. Non-literal text reuse in historical texts: An approach to iden- tify reuse transformations and its application bible reuse. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1849-1859.
(0, X), the E-step computes the Q-function below:
Q x; 0 > (0)
- DT
E
T 1(0))
=C
Σ
(log Bw n (n (0()) 2(0))
WCD
Changzai Pan, Maosong Sun, and Ke Deng. 2022. TopWORDS-see Simultancous text segmentation and word discovery for open-domain Chinese texts via Bayesian inference. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Pupers), pages 558-169, Dublin, Ireland. Association for Computational Lin- guistics.
(lgg X no 1())
@E.PAU3
where C is a constant that does not change with A
n
nw (0 (11)
Σ
na (0
A 11),
j=1
no(0())
Σ
(0(() 1
Lisa Pearl, Sharon Goldwater, and Mark Steyvers. 2010. Online learning mechanisms for Bayesian models of word segmentation. Research on Language and Computation, 8:107-132.
ne
m (0(t) ( (()))
Donald Sturgeon. 2018. Unsupervised identification of text reuse in early Chinese literature. Digital Schol- arship in the Humanities, 33(3):670-684.
= (n (Tj, Bj) D, D;; 000) (1())
(A.2)
= Σ Nww (Ty, (B) DD ,(1)),
ByEB;
Junyi Sun. 2012. Jieba Chinese word segmentation tool.
nes ((1))
=P(Z, 21 D, Tj; ((0) A())
(A.3)
Maosong Sun, Xinxiong Chen, Kaixu Zhang, Zhipeng Guo, and Zhiyuan Liu, 2016. THULAC: An efficient lexical analyzer for Chinese.
Σ (B, e) P(T, [D; 00) ,B))
x e
BYEB
The M-step updates (0 A ()) by maximizing
Wai-lim Yip. 1997. Chinese poetry, revised: an anthol- ogy of major modes and genres. Duke University Press.
(0, x; ((t) ( with respect to (0, A), leading
to the updating function below:
Zheng Yuan, Yuanhao Liu, Qiuyang Yin, Boyao Li, Xi- aobin Feng. Guoming Zhang, and Sheng Yu. 2020. Unsupervised multi-granular Chinese word segmen- tation and term discovery via graph partition. Journal of Biomedical Informatics, 10:103352..
0ft11)
(n(t))))A00)) EWED no (010)
VwE E D; (A.4)
x(++1
Ve IA..)
E E 1010 X
B Calculation of Significance Score
A EM Algorithm for Estimating (0,
In order to compute vhe efficiently through dynamic programming, we have
By adding latent variables Z, the target posterior distribution f(A, X, B | D,T) can be extended to f(0,X. B D. T) as following:
log (1 Twg) Σ j=1
4h
where
1(0, X. Z. B D,T)
A = =0)) (()) II
Twj
= P w Bj Tú, D.
(B.1)
(Azz- TB 2z)
Σ 1( BB) PPBB) D; 0, A),
j=1
BYEB,
IP[T| D)
(A.1)
with notation "w ~ B," meaning that word can- didate w appears in the segmented version of Tj based on By.
Given (0())(00) (0)) the current estimation of

---

C Efficient Computation via Dynamic Programming
all computation issues in the following recursion:
(T(()))) =
Σ
(To st)
ISoSmin(t 1,71)
To implement the TopWORDS-Poctry algo- rithm, we need to calculate Rej in (A.2), Roj in (A.3), jwj in (B.1) and Pyt in (11) for v T3 E T.
OM---e-11 II (1-Pat)
pot) P(e),e-1)
t-eskct1
plo)(T (T(24) =
E
For a specific - T 01 ... a, by defining Tites] = a4- ... ag. we can shown that R. Neg. Tuj and Yit. which are all functions of T,, have the following recursive formula:
ISoSmin(L t.T.)
f (e-ee4t
II t+ISkct+s
- Patts)
D Insights behind the Mathematical Formulation
nw (T)
Xe
(T),
The mathematical formulation of TopWORDS- Poetry composed of four parts. The first part, which contains just formula (I), defines the basic Unigram Language Model (ULM) which serves as the backbone of the TopWORDS-scric meth- ods. Formula (2)-(6) form the second part, which defines the informative prior distribution for word boundary based on the potential segmentation pat- terns of poetry lines. The third part composed by formula (7)-(11) provides detailed results about Bayesian inference of the unknown model parame- ters and word boundary prediction under the ULM and the specified prior. And, the fourth part com- posed by formula (A.1)-(B.)) in the Appendix pro- vides more computational details about the EM algorithm and significant score.
(TT)
DE LE
(T) Σ
ple) (Tien) p10) (Tiss)
ISICEIL
Ow
II (1 - Pet) Pes: [(Tm.p=D)
ISK<s
(T) Ag plel (T)
P(T)
Tw (T) p(T')
Σ
Ag . r (T),
PL
(T) Σ [ [>1] ((Tpe + ((Tu.==w)
ple) (Tix))
Orissn II (1-Pet) Pot (T))
Among these four parts, part 3 and are all standard mathematical formulas about ULM, Bayesian inference and EM algorithm with fast cal- culation via dynamic programming, which are very similar to their counterparts in the TopWORDS- Seg algorithm proposed by Pan et al. (2022). The design of prior distribution in the second part, i.e.. formula (2)-(6), however, makes a key difference and plays an essential role in this study. Among these formulas, the conjugate Dirichlet prior in for- D; -x(B Poi): mula (2) is a natural choice that has been adopted by Pan et al. (2022). Formula (3), however, speci- fies for a mixture prior distribution with multiple components to reflect the fact that each poetry line can potentially follow multiple segmentation pat- terns. Formula (4) corresponds to the fact that segmentation pattern may not be exactly followed and we have a chance to break through the pat- tern template by placing a word boundary at each position randomly. Formula (5)- (6), on the other hand, specify the prior distribution for the relative
15ke
(T) Lecor, A p(T(ss))) p(T()))
P(T)
where
pl0)
(Tpaa) (TO)(Tp)))
Σ P(
BCB
p(T) = Σ >p.20 plee(())))) (T),
DEC
with B|al being the collection of all possible word
boundary profiles of T [
We can compute pl (T)) and (T)))) in
linear time via dynamic programming to implement

---

weights of the mixture components defined in for- mula (3). And, formula (7), at last, assembles all these piecewise prior distributions into a joint prior distribution to guide the learning.
patterns:
10 2212/212 2212/221,
2221|212
@221 221)
= (2221 2212: 22212/2221 /2221
22221
(E.1)
E More Details on TOpDORSS-Poetry
Segmentation patterns such as 2212/212 means that the two poetry lines in one couplet have the same pattern 2-1-2, while 2212/22 means that the two poetry lines have different patterns. Let X10 and A14 be the proportion vectors of Lo0 and 9 10. respectively. Replacing Pg and Py by P and P00 we can implement TopWORDS- Poetry* by following exactly the same pipeline in TopWORDS-Poetry, except for fixing Pe.t.// 1 for any e E P1 U P4 because there is always a word boundary between the two poetry lines in a couplet.
Let (72--1, T2:) be one couplet composed of two poctry lines. We can estimate the transition proba- bility (T222 12 based on couplets in training data P1000. Figure E1 visualizes the estimated tran- sition probability matrices, suggesting there exists positive correlation between the usage of segmen- tation patterns within couplet.
212
80.0%
15.0%
5.0%
0.8
0.6
F Details on Calling ChatGPT
221
29.5%
60.7%
9.8%
To segment classical Chinese poetry lines by Chat- GPT, we asked ChatGPT (model version: gpt- 35-turbo) to segment all poetry lines in the test- ing dataset P1000 under the parameter setting where temperature = 0.0, mar_lokent 800, top_p - 0.95, frequency_penone
0.2
other
33.6%
25.0%
41.4%
212
221
other
0, presence presence_penalty- = 0 (all other parameters take the default values) with the prompt template in Figure F1.
(a)
2212
75.2%
18.5%
6.3%
0.8
0.6
Prompt template:
2221
37.5%
54.2%
8.3%
0.4
TRE
0.2
A
<
6T ill
*
other
30.7%
20.9%
48.5%
1
SR/AEM
/T//
/
A
2212
2221
other
(b)
English translation:
Figure El: Transition probability matrix between patterns within one couplet on Trow (a) Transi- tion probability matrix for 5-character poem couplets. (b)Transition probability matrix for 7-character poem couplets.
"Please do text sogmentation for the sentences in separated by "y", without providing any explanation. Input: His deeds covered kingdom split in three, his fame completed the
Eight Formations, The River flows on, stones not turn remnant bittemess at his failure to swallow Wu.>
Answen doods covered a kingdom split three J, his fame completed the River flows on stones do not turn a remnant bitterness at his failure to swallow Wu. Inpul: <f Target Text]>"
Eight Formations.
These facts suggest that we can further improve TopWORDS-Pocee by modelling such correlation. In practice, this can achieved conveniently by merg- ing two poetry lines in the same couplet into longer concatenated line with 10 or 14 Chinese characters, and extending Py and Py to the fol- lowing larger collections of longer segmentation
Figure Fl: The prompt template calling ChatGPT and its English translated version.

---

G Performance Evaluation on Word Discovery by Word Type
and standard deviation of the bootstrap distribu- tion of each performance measurement for every involved method.
Performance evaluations on word discovery re- ported in Table I are pooled results for all word types including both common words and technical words, e.g., names, addresses and literary allusions, Here, we would like report more detailed results for different types of technical words. For this pur- pose, we manually picked up all names, addresses and literary allusions in the 1000 segmented poems in the P1o0n. and summarized Table GI the per- formance of different approaches on discovering technical words of different types. From this table, we can see that TopWORDS-PORe discovering all types of technical words, particu- larly for literary allusions that are often missed by competing methods.
If TopWORDS Peettyy works significantly bet- ter than competing methods, we would like to observe an obvious shift in its empirical distribu- tions with respect to the competing methods, which can be tested by standard paired 2- 2-sample t-test. 'he last two mance margins between TopWORDS PPettry (the more robust member of the TopWORDS-Poery family) and the best competing method outside the SeWWORDS-Peetee family, and the p-values of the corresponding paired 2-sample (-test. Be- cause most of these p-values are highly signifi- cant (<0.001), we are comfortable to claim that TopWORDS Sootr indeed works better than isting methods in most performance evaluation di- mensions. Although such strategy is not very rigorous, provides us with strong empirical evi- dence to gain useful insights about the statistical significance of TopWORDS-Poed with respect to competing methods.
of Table H1 reports the perfor-
is effective in
Method
Name
Adress
Allusion
Entity Count
1303
924
723
LTP
20.64%
22.29%
19.78%
StanfordNLP
50.42%
52.71%
44.67%
THULAC
61.17%
63.31%
59.06%
PKUSEG
63.70%
64.94%
60.86%
Jicha
62.55%
64. 18%
55.19%
TopWORDS
84.96%
84.20%
82.99%
TopWORDS-Seg
73.68%
76.95%
72.48%
RandomPattern
80.05%
76.84%
69.85%
SelectPatter
86.19%
85.50%
78.56%
ChatGPT 80.97%
83.23%
78.98%
TopWORDS Poctry
87.18%
88. 64%
88.38%
TopWORDS-Poetry
87.72%
88.96%
88.38%
Table G1: Performance of different methods on discov- ering technical words of different types.
H Does Top WORDS-Poetry Perform Significantly Better?
To check whether TopWORDS-Pocs performs significantly better than competing methods, we conducted an additional experimental study based on the idea of bootstrapping (Efron and Tibshirani, 1994). For this purpose, we generated 20 pseudo testing datasets by randomly picking up 80% el- ements of P1000 without replacement. For each of these pseudo testing datasets, one version of the nulti-dimensional performance measurements was calculated, resulting an empirical distribu- tion of performance measurements for every in- volved method. Table H1 summarizes the mean

---

1: 0.30%
0.322
0
0.336
1 0.20
64. 0
0.40%
0.23%
1: 0.26%
0.20%
0.21%
Table HI: Performance of ToPWORSS Poetry for classical Chinese poetry by bootstrapping on word discovery and text segmentation compared to competing methods.
200
0.73
19.67%
37.7 77.74%
47. 47.84%
$1.30%
00.72%
22.566:
55.97%
86.54%
86.69%
6.27
21 21,839 1 0.333
41.18% 1: 0.48%
= 0.40%
52.92% H. 0411
0318
0.00
0443
0.555
1.33%
+ 0.4%%
1 n428
7.85 x 10-0
270
0.34
49.43 49.43%
54.31%
64,99%
72.725
66.48%
86.47%
86.
86.48
Text segnentition (Fs)
0.31%
4: 0.41%
0.31%
0.41%
0,366
62 # 0,325
0.40%
0.27%
0,26%
0.30%
10-17
520
1.14
18.78%
40.160 16%
50.16%
52.92%
81 $1538
82.6799
2.80
61
S1
21.18% = 0..0
40. 00.33% = 0.70%
50.0898 = 0.79%
52.48% # 0.57%
=: 0.548
63.3 33.1%% = 0.46%
63.78% 0.47%
11.14% 2: 0.53%
81.53% = 0.46%
N0.72 M0.2% # 0.45%
2.25 x 10
200
-042
20.05% + 0. 15%
39.45% 1: 0. 0.88
49.1 19.118 + 0.15%
52.23% 1: 0.17%
1.0.00
0.22%
84.1 11% 0.17%
84.33% 1 0.0. 1788
1.06 10-12
P1o0
53.00% 1
0.62
63.08 33.08%%
84.50%
1.005
44.61 88 # 1.53%
34.22 42.22 = 1.27%%
36.16 16.168 #: 1.30%
= 1.37%
84% 1.01%
1.50%
0.00%
1.64%
a 1.57%
1:15 x 10
RC R2.3
-32.7 0.17%
29.94%
57.78%
0.00%
63 28.8
57.07%
0.12%
# 0.25%
= 0.22%
58.88 0.23%
57. 0.1%%
0.25%
0.23%
0.14%
0.20%
4.10 x 10
R
0.58
19.099
43. 19%
52.88%
71.5%%
88 80%
99.47%
89.47
Word discovery (P)) R!! "
67.
=: 26
:: 24
= 25
24
22
27
24
7688 21
21
24
2088
4921
5502
6318
3677
7080
7106
7803
7W28
1.4%%
63.71 1: 1.90%
# 1.40%
48.36% 1a 1.49%
1.27%
1.51%
0.00%
1.0%%
1.24%
2.22 x 10
R
44,58
58.45%
43.62%
3380
15
$4.05%
0.00%
44.47%
X3.57%
60. 303 1 0. 19%
# 0.23%
0.21%
0.15%
0.15%
0.25%
0.13%
0.00%
0..0%
9:57 x 10
(222
1.15
77.69'5
82.08
78.61%
78.70%
83.51%
44.33%
96. 32%
39.39%
77.5
39
77K
76K
83K
71K
54K
R6K
73K
82K
K2K
SMpWORDS-SSS Sug
TepWORDS Pheery"
Method
StanforaNL.P
THULAC
PKUSEG
Top WORDS
RundomPattem
SeleetPoteem
Performance
p-valuc
LTP
Jieba
TopWORDS


Reviews:
[{"review_id": "YbSTvaijMo", "reviewer": "Reviewer_m3qs", "summary": "", "questions": "A: What might account for the large difference in R_s(2,3) between TopWORDS and TopWORDS-Poetry?\n\nB: In their discussion of off-the-shelf segmentation tools, the authors write that \"it is not realistic to tune these tools for this specific task, because no training corpus for classical Chinese poetry are publicly available yet, to the best of our knowledge.\" I was confused by this claim - why does Complete Tang Poetry not count as a suitable corpus in this context? I think it would be helpful to provide a short rationale for not attempting fine-tuning of a general-purpose tool and instead proceeding directly to creation of a bespoke method. ", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your positive comments, constructive suggestions and insightful questions. Below are our point-to-point responses to your comments and questions. \n\nFirst, to provide more quantitative evidence to compare the performance of different approaches on discovering unique lexical terms such as named entities and literary allusions by word or phrase type as you suggested, we manually labelled all names, addresses and literary allusions in the 1000 manually segmented poems in the $P_{1000}$ data set, and summarized in the table below the performance of different approaches on discovering these lexical terms:\n\n| Method | Name | Adress | Allusion  |\n|---|---|---|---|\n| entity count | 1303  | 924  | 723   |\n| LTP | 20.64% | 22.29% | 19.78%  |\n| StanfordNLP | 50.42% | 52.71% | 44.67%  |\n| THULAC | 61.17% | 63.31% | 59.06%  |\n| PKUSEG | 63.70% | 64.94% | 60.86%  |\n| Jieba | 62.55% | 64.18% | 55.19%  |\n| TopWORDS | 84.96% | 84.20% | 82.99%  |\n| TopWORDS-Seg | 73.68% | 76.95% | 72.48%  |\n| RandomPattern | 80.05% | 76.84% | 69.85%  |\n| SelectPattern | 86.19% | 85.50% | 78.56%  |\n| TopWORDS-Poetry | 87.18% | 88.64% | **88.38%**  |\n| TopWORDS-Poetry^* | **87.72%** | **88.96%** | **88.38%**  |\n\nFrom this table, we can see that TopWORDS-Poetry is effective in discovering all types of lexical terms, particularly for literary allusions that are often missed by most competing methods. We would be very happy to add these new results to the manuscript as main texts or supplementary materials if our paper can be accepted by the conference.\n\nSecond, in response to your request for more discussions about how the proposed method might enable new studies of literary allusions, we have carefully studied the references you recommended and come up with the insight that our work and these references have quite different focuses although they are in general related. Due to the high conciseness of classical Chinese poetry, in most cases, literature illusions in classical Chinese poetry are short phrases with no more than 4 Chinese characters highlighting particular events or literary imageries. For example, the literature illusion \"老萊衣\" (Laolai’s Clothes) refers to the colorful clothes worn by 老萊 (Laolai Zi), a hermit in the country of Chu in the Spring and Autumn period who was famous by amusing his parents wholeheartedly by wearing colorful clothes and making funny movements when he was 70 years old. In Chinese culture, \"老萊衣\" has become a symbol of filial piety, because the story behind the phrase indicates the spirit of sacrificing oneself for the benefit of parents, which is the key element of filial piety. Literature illusions like this one can deliver complicated meanings with concise texts, increasing the charm of classical Chinese poetry greatly. Intertextuality studied by Moritz et al. (2016), Sturgeon (2018) and Burns et al. (2021), however, mainly focuses on identifying parallel sentences or passages that are literally different but have similar meanings. According to Sturgeon (2018), below is a typical example of such parallel sentences:\n\n\"使乌获疾引牛尾，尾绝力勯，而牛不可行，逆也\" (If let the strongman Wu Huo pull the tail of the ox, and if the tail breaks and Wu Huo is exhausted but the ox does not move forward, it would be considered a reverse situation) in《吕氏春秋》(*Lvshi Chunqiu*);\n\n\"今使乌获、藉蕃从后牵牛尾，尾绝而不从者，逆也\" (Today, if let the strongman Wu Huo and the helper Jie Fan pull the tail of the ox from behind, and if the tail breaks and the ox does not move forward, it would be considered a reverse situation) in《淮南子》(*Huainanzi*).\n\nApparently, the task of discovering parallel sentences/paragraphs is more like a matching problem based on sentence/paragraph meaning, which is different from the word/phrase discovery task in this paper. On the other hand, however, these two tasks are also closely related, if two parallel sentences/paragraphs correspond to the same literature illusion, which can be effectively summarized into a common short phrase representing the literature illusion. We would be very happy to cite the references you recommended in our paper with in-depth discussions, if our paper can be accepted by the conference.\n\nThird, in response to your question on the large difference in $R_s^{(2,3)}$ between TopWORDS and TopWORDS-Poetry, we would like to provide the following explanation. In general, TopWORDS is a more aggressive word discoverer and text segmenter than TopWORDS-Poetry, which tends to report more longer words/phrases in word discovery and reserve more longer words/phrases in text segmentation. Although such a property makes TopWORDS a powerful tool for Chinese word/phrase discovery in open domains, it often under-segments Chinese texts as pointed out by Pan et al. (2022). Similar to TopWORDS-Seg, the proposed TopWORDS-Poetry tries to re-balance the ability of TopWORDS on word discovery and text segmentation tasks by introducing informative prior information on text segmentation under the Bayesian framework. Such an effort would naturally enhance the performance of TopWORDS-Poetry on poetry line segmentation at the price of performance degeneration on word discovery. That is why we observe the large difference in $R_s^{(2,3)}$ between TopWORDS and TopWORDS-Poetry. In fact, a similar phenomenon has been reported by Pan et al. (2022). In practice, we can adjust the granularity of poetry line segmentation in TopWORDS-Poetry by selecting the hyper-parameters $\\kappa_s$: if we increase $\\kappa_s$ from the current 0.001 to 0.1, the $R_s^{(2,3)}$ of TopWORDS-Poetry would increase to 83%, a value very close to TopWORDS. But, if we do so, the $F_S$ scores of TopWORDS-Poetry would decrease from the current ~85% to ~80%, which is slightly smaller than the 84.12% of SelectPattern. In the manuscript, we choose to specify $\\kappa_s=0.001$ instead of 0.1 to make sure that TopWORDS-Poetry can dominate SelectPattern in all dimensions. If we could accept a little bit of relaxation in the $F_S$ scores, we can easily get a $R_s^{(2,3)}$ that is as large as 80% by choosing $\\kappa_s=0.1$ instead. We hope these explanations can help better clarify things.\n\nFourth, about your question on why the *Complete Tang Poetry* itself cannot serve as a suitable training corpus to tune existing methods, we would like to respond as follows. To serve as a qualified training corpus to tune existing methods, a corpus must contain well-segmented Chinese texts with all word boundaries explicitly labelled. The original *Complete Tang Poetry*, however, only contains raw poetry lines without any word boundaries. That is why we cannot tune existing methods with the *Complete Tang Poetry* itself.\n\nAt last, we would correct the typos you pointed out, split Fig 3 into separated parts as you suggested, and enhance the word clouds with English translations, once we get the chance to update the manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "-The authors report a detailed empirical evaluation of TopWORDS-Poetry and show that it outperforms a range of other methods, including off-the-shelf word segmenters and TopWORDS-Seg.\n\n-The method is well-described, and ample details are provided for replicating the analyses. \n\n-Classical Chinese remains an under-resourced language, and the method should be immediately useful both for NLP researchers and for digital humanities scholars interested in the corpus. I appreciate that the authors have taken genre differences seriously and exploited the unique structure of classical Chinese verse to develop a bespoke segmentation tool. ", "reasons_to_reject": "I think the paper makes a strong contribution overall. My main suggestion for improvement would be to provide some finer-grained evaluation data. In particular, the authors emphasize that their method performs better with lexical items rarely found outside of poetic corpora, such as named entities and literary allusions. While the word clouds in Figure 3 are interesting, I would like to see more quantitative evidence for this claim, including performance information broken down by word or phrase type. How, for instance, does the tool do specifically on allusive passages? I appreciate that doing so would require additional manual work, but I think having a breakdown across a subset of the test set would be very informative. \n\nAlso, I would be interested to read further discussion about how the method might enable new studies of literary allusions. As the authors are no doubt aware, quantitative study of intertextuality is an topic of ongoing, cross-lingual interest in cultural analytics and digital humanities (e.g., Sturgeon 2018 Digital Scholarship in the Humanities, Burns et al. 2021 NAACL, Moritz et al. 2016 EMNLP), so such discussion might help excite and orient readers who are primarily interested in applications. ", "questions_to_authors": "A: What might account for the large difference in R_s(2,3) between TopWORDS and TopWORDS-Poetry?\n\nB: In their discussion of off-the-shelf segmentation tools, the authors write that \"it is not realistic to tune these tools for this specific task, because no training corpus for classical Chinese poetry are publicly available yet, to the best of our knowledge.\" I was confused by this claim - why does Complete Tang Poetry not count as a suitable corpus in this context? I think it would be helpful to provide a short rationale for not attempting fine-tuning of a general-purpose tool and instead proceeding directly to creation of a bespoke method. ", "paper_topic_and_main_contributions": "The paper describes TopWORDS-Poetry, a Bayesian text segmentation algorithm optimized for classical Chinese poetry. The method is an adaptation of TopWORDS-Seg (Pan et al. 2022) that incorporates genre-specific information about segmentation patterns into the prior distribution. Performance is evaluated on word discovery and text segmentation tasks using the Complete Tang Poetry corpus. \n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "I would recommend splitting Figure 3 into multiple parts. (a) and (b) should be standalone tables, and the font size should be increased to improve readability. The word clouds in (c) - (e) can then be kept as Figure 3. \n\n\"Corpus\" is singular, \"corpora\" plural, so, e.g., \"no training corpus...are publicly available\" is incorrect (lines 130-131). \n\nWould it be possible to provide translations of the word clouds to improve accessibility of the paper for the full audience of EMNLP? "}}, {"review_id": "aw5NihdYeN", "reviewer": "Reviewer_qve5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your sharp comments and constructive suggestions. Below are our point-to-point responses to your comments. \n\nWe fully agree with you that the proposed TopWORDS-Poetry is indeed a unique approach designed for a very specific application. But we believe that the research in this study still makes significant academic contributions to the NLP community, since classical Chinese poetry covers a body of texts with very unique features for which little research has been done. First, by formulating potential segmentation patterns of poetry lines as informative prior to guide text segmentation and word discovery under the framework of TopWORDS, we come up with an effective approach to process classical Chinese poetry and fill a blind spot in NLP literature. Second, experiments in this study confirm that existing methods established for general Chinese texts have various limitations in processing classical Chinese poetry. Third, applying TopWORDS-Poetry to the nearly 50,000 poems in the *Complete Tang Poetry*, we came up with a large collection of discovered words/phrases containing many named entities and literary allusions that have not been reported before, which may greatly promote the study of classical Chinese poetry via the paradigm of digital humanity.\n\nWe are terribly sorry that we were not able to provide sufficient explanation about the intuition behind the mathematical formulas involved in this paper due to the limited space of the conference paper. We would be very happy to explain the reasoning behind the TopWORDS-Poetry algorithm by providing an additional supplementary material once the paper is accepted. In fact, the 17 major formulas in our paper can be divided into four groups. The first group, which contains just formula (1), defines the basic *Unigram Language Model* (ULM) which serves as the backbone of the TopWORDS-series methods. Formula (2)-(6) form the second group of formulas, which defines the informative prior distribution for word boundary based on the potential segmentation patterns of poetry lines. The third group composed of formula (7)-(11) provides detailed results about Bayesian inference of the unknown model parameters and word boundary prediction under the ULM and the specified prior. The fourth group composed by formula (12)-(17) in the Appendix provides more computational details about the EM algorithm and significant score. \n\nAmong these four groups of formulas, group 1, 3 and 4 are all standard mathematical formulas about ULM, Bayesian inference and EM algorithm with fast calculation via dynamic programming, which are very similar to their counterparts in the TopWORDS-Seg algorithm proposed by Pan et al. (2022). Formulas in the second group, however, make a key difference and play an essential role in this study. Below, we explain in detail why we specify prior distributions as in formula (2)-(6). In fact, the conjugate Dirichlet prior in formula (2) is a natural choice that has been adopted by Pan et al. (2022). Formula (3), however, specifies for $B_j$ a mixture prior distribution with multiple components to reflect the fact that each poetry line can potentially follow multiple segmentation patterns. Formula (4) corresponds to the fact that a segmentation pattern may not be exactly followed and we have a chance to break through the pattern template by placing a word boundary at each position randomly. Formula (5)-(6), on the other hand, specify the prior distribution for the relative weights of the mixture components defined in formula (3). And, formula (7), at last, assembles all these piecewise prior distributions into a joint prior distribution to guide the learning.\n\nWe hope the above explanations would help better clarify potential confusions. We would be more than happy to upload an additional supplementary material for this issue if our paper is fortunately accepted.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* a novel method for joint text segmentation and vocabulary induction in classic Chinese poetry; even though the method is to a large extent based on an existing model, there is sufficient novelty in the adaptations to the specific genre, which I find interesting\n\n* authors provide an extensive evaluation and comparison to related work", "reasons_to_reject": "* the application of the method is fairly limited\n\n* descriptions are fairly technical; technical adequacy is a good thing, but I feel some more effort could have been taken to better explain the reasoning behind the formulae\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper presents an unsupervised Bayesian model to jointly perform text segmentation and word discovery (i.e. automatic vocabulary induction). The method is based on an existing Bayesian method (Pan et al., 2022) with some adaptations specifically tailored towards the specific genre of classical Chinese poetry. One of the key ideas is to exploit the regularity of the poetry's meter, modeled as a Bayesian prior. The method yields good results in both segmentation and vocabulary induction.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "1gHLWwxypM", "reviewer": "Reviewer_BAFH", "summary": "", "questions": "- Please explain why the segmentation tool you are developing is required. In particular who is it for? Is it for scholars studying these\npoems? Would the scholars have issues with unknown words and with segmentation? I would have thought it is their job to understand these poems. So if it is not scholars you are targetting who is it? Do you want to help lay people understand these poems? Is it common? Or you are developing a literary analysis tool? \n- I am having trouble understanding how word discovery is different from text segmentation. Isn't segmentation in Chinese inserting boundaries between characters which would in turn give you word discovery?\n- Can you please explain clarify part of your model is different from Peng et al (2022). Is it a prior that also takes into account segmentation patterns? In other words equation (3)?\n- It would be good to report aggregate scores in Figure 3(a) and also whether differences between TopWORDS-Poetry(*) and the next best comparison system are statistically significant. \n- did you try to see how a GPT style model performs your task using in context learning?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your insightful comments and detailed suggestions, which have stimulated us to think in depth about the academic positioning and representation of this paper. We fully agree with you that the proposed TopWORDS-Poetry method is an extension of the TopWORDS-Seg method in Pan et al. (2022), aiming at a niche task. But we believe that the research in this study still makes significant academic contributions to the NLP community, since classical Chinese poetry covers a body of texts with very unique features for which little research has been done. First, by formulating potential segmentation patterns of poetry lines as informative prior to guide text segmentation and word discovery under the framework of TopWORDS, we come up with an effective approach to process classical Chinese poetry and fill a blind spot in NLP literature. Second, experiments in this study confirm that existing methods established for general Chinese texts have various limitations in processing classical Chinese poetry. Third, applying TopWORDS-Poetry to the nearly 50,000 poems in the *Complete Tang Poetry*, we came up with a large collection of discovered words/phrases containing many named entities and literary allusions that have not been reported before, which may greatly promote the study of classical Chinese poetry via the paradigm of digital humanity. Moreover, the extension from TopWORDS-Seg to TopWORDS-Poetry is non-trivial, as the design of informative prior to count for the segmentation patterns of poetry lines involves deliciated modelling techniques on top of deep insights into the unique task.\n\nWe totally agree that the Bayesian formulation involved in this paper is not friendly for readers who are not familiar with Bayesian inference. We are terribly sorry that we were not able to provide sufficient explanation about the intuition behind the mathematical formulas involved in this paper due to the limited space of the conference paper. In fact, the mathematical formulation of this paper is composed of four parts. The first part, which contains just formula (1), defines the basic *Unigram Language Model* (ULM) which serves as the backbone of the TopWORDS-series methods. Formula (2)-(6) form the second part, which defines the informative prior distribution for word boundary based on the potential segmentation patterns of poetry lines. The third part composed by formula (7)-(11) provides detailed results about Bayesian inference of the unknown model parameters and word boundary prediction under the ULM and the specified prior. And, the fourth part composed by formula (12)-(17) in the Appendix provides more computational details about the EM algorithm and significant score. \n\nAmong these four parts, part 1, 3 and 4 are all standard mathematical formulas about ULM, Bayesian inference and EM algorithm with fast calculation via dynamic programming, which are very similar to their counterparts in the TopWORDS-Seg algorithm proposed by Pan et al. (2022). The design of prior distribution in the second part, i.e., formula (2)-(6), however, makes a key difference and plays an essential role in this study. Among these formulas, the conjugate Dirichlet prior in formula (2) is a natural choice that has been adopted by Pan et al. (2022). Formula (3), however, specifies for $B_j$ a mixture prior distribution with multiple components to reflect the fact that each poetry line can potentially follow multiple segmentation patterns. Formula (4) corresponds to the fact that a segmentation pattern may not be exactly followed and we have a chance to break through the pattern template by placing a word boundary at each position randomly. Formula (5)-(6), on the other hand, specify the prior distribution for the relative weights of the mixture components defined in formula (3). And, formula (7), at last, assembles all these piecewise prior distributions into a joint prior distribution to guide the learning. We would be very happy to explain the insights by uploading an additional supplementary material for this issue once the paper is fortunately accepted. \nBelow are our point-to-point responses to your detailed questions.\n\n\n\n**Q1**. Please explain why the segmentation tool you are developing is required. In particular who is it for? Is it for scholars studying these poems? Would the scholars have issues with unknown words and with segmentation? I would have thought it is their job to understand these poems. So if it is not scholars you are targetting who is it? Do you want to help lay people understand these poems? Is it common? Or you are developing a literary analysis tool?\n\n**Response**：Thanks for raising this important question, which we did not discuss sufficiently in the current manuscript. The primary goal of this study is to develop a powerful tool for literary scholars to analyze classical Chinese poetry at a large scale via distance reading, a research paradigm that has been widely adopted by the community of digital humanity. Under the traditional paradigm of literary research, an individual scholar typically conducts an in-depth study on a small collection of poems written by specific poets, in particular periods, or in similar styles. Few efforts have been made to study a large number of poems at once from a global perspective due to the lack of technical tools that can effectively process classical Chinese poetry. In the history of Chinese literature, approximately 800,000 classical poems have been passed down, including 50,000 poems in *Complete Tang Poetry*. Although such a corpus is not as big as Wikipedia, it definitely exceeds the capability limit of individual scholars, leading to a great appeal for developing effective algorithms to process Chinese poetry texts. At present, there is still no widely adopted method for such a purpose. Once we make a technological breakthrough in this direction, we would expect an exciting interaction between research on classical Chinese poetry and NLP technology, which may lead to a new era of literary study and promote many exciting applications for cultural education and dissemination, helping more people to better enjoy the charm of classical poetry. The proposed method has the potential to play such a role.\n\n**Q2**. I am having trouble understanding how word discovery is different from text segmentation. Isn't segmentation in Chinese inserting boundaries between characters which would in turn give you word discovery?\n\n**Response**: Sorry for not discussing this important issue sufficiently in the current manuscript, as we believe that Pan et al. (2022) have already provided an insightful discussion on this important issue. At the conceptual level, word discovery and text segmentation are two highly related tasks in Chinese NLP, because inserting word boundaries between Chinese characters can naturally generate the segmented words. In practice, however, as pointed out by Pan et al. (2022), these two tasks have different focuses and often entangle with each other. This is because classical methods for Chinese text segmentation usually require the support of a preloaded vocabulary, which is either an explicit list of words, or an implicit word collection encoded in the large language models. When the preloaded vocabulary fits the target corpus well, we typically expect reasonably good results on text segmentation. Once the preloaded vocabulary fails to cover the underlying vocabulary of the target corpus, however, we often observe significant performance degeneration on text segmentation, as demonstrated in Pan et al. (2022). The task in this study happens to fall into the second case, as classical Chinese poetry contains many unique named entities and literary allusions that never appear in general texts. In this scenario, we would have to concern the word discovery task and consider it with the text segmentation task together in pursuit of high-quality results.\n\n**Q3**. Can you please explain clarify part of your model is different from Peng et al (2022). Is it a prior that also takes into account segmentation patterns? In other words equation (3)?\n\n**Response**: We believe that the paper you mentioned should be Pan et al. (2022). Indeed, as an extension of the key ideas in Pan et al. (2022), TopWORDS-Poetry is very similar to TopWORDS-Seg in model structure and learning strategy. The key difference is that TopWORDS-Poetry introduces a more complicated mixture prior distribution to count for the potential segmentation patterns for Chinese poetry lines, which also leads to new computational challenges on parameter estimation, model selection and post Bayesian inference. \n\n\n**Q4**. It would be good to report aggregate scores in Figure 3(a) and also whether differences between TopWORDS-Poetry(*) and the next best comparison system are statistically significant.\n\n**Response**: Thanks for these constructive suggestions. We totally agree that it would be ideal if we can integrate the multi-dimensional evaluation measurements into a one-dimensional score. To the best of our efforts, however, we found that it’s non-trivial to define such an integrated score due to the absence of a reasonable criterion to conduct the integration. Totally, two types of evaluation measurements are reported in Fig 3 (a): one for the task of word discovery, and one for the task of text segmentation. Because the two tasks are quite different in nature, there does not exist a straightforward way to combine their performance measurements. Although trivial aggregation such as simple or weighted averaging is always possible, it would be very difficult to interpret the averaged value obtained in this way and determine the weights wisely. Thus, to avoid potential confusions and disputes, we chose to report detailed, instead of integrated, results.\n\nTo evaluate the statistically significance you suggested, we conducted an additional experimental study based on the idea of bootstrapping. To be concrete, generated 20 pseudo testing datasets based on the 1000 manually segmented poems in original testing dataset $P_{1000}$ by randomly picking up 80% elements of $P_{1000}$. For each of these pseudo testing datasets, we can calculate one version of the multi-dimensional performance measurements, resulting in an empirical distribution of performance measurements for every involved method. If TopWORDS-Poetry works significantly better than competing methods, we would like to observe an obvious shift in its empirical distributions with respect to the competing methods, which can be tested by a standard paired 2-sample t-test. After implementing such a protocol, we found that the p-value of the paired 2-sample t-test is highly significant (<0.001) in almost all dimensions, indicating that TopWORDS-Poetry indeed works better than existing methods. Although such a strategy is not very rigorous from the theoretical point of view, it provides us with empirical evidence to gain insights about the statistical significance of the results by TopWORDS-Poetry with respect to the competing methods. We hope these efforts can help ease your concerns.   \nDetailed performance (mean and standard error) with differences from the next best methods and p-values by paired 2-sample t-test are shown below:\n\n| Method | $R_d^{(t,l)}$ |  |  | $R_s^{(t,l)}$ |  |  | $F_S$ |  |  |  |   |\n|---|---|---|---|---|---|---|---|---|---|---|---|\n|  | $\\|V_d\\|$ | $R_d^{(2,2)}$ | $R_d^{(2,3)}$ | $\\|V_s\\|$ | $R_s^{(2,2)}$ | $R_s^{(2,3)}$ | $P_{1000}$ | $P_{250}^{4\\times5}$ | $P_{250}^{8\\times5}$ | $P_{250}^{4\\times7}$ | $P_{250}^{8\\times7}$  |\n| LTP | 39K | 60.39% ± 0.19% | 58.45% ± 1.47% | 2088 ± 26 | 19.08% ± 0.12% | 29.94% ± 1.05% | 20.05% ± 0.15% | 21.18% ± 0.8% | 18.78% ± 0.31% | 21.83% ± 0.33% | 19.67% ± 0.3%  |\n| StanfordNLP | 77K | 77.54% ± 0.23% | 63.71% ± 1.9% | 4921 ± 24 | 43.19% ± 0.25% | 44.61% ± 1.53% | 39.45% ± 0.18% | 40.33% ± 0.7% | 40.16% ± 0.41% | 41.18% ± 0.48% | 37.74% ± 0.32%  |\n| THULAC | 76K | 77.69% ± 0.21% | 43.62% ± 1.4% | 5502 ± 25 | 52.88% ± 0.22% | 34.22% ± 1.27% | 49.11% ± 0.15% | 50.08% ± 0.79% | 50.16% ± 0.31% | 49.43% ± 0.4% | 47.84% ± 0.23%  |\n| PKUSEG | 83K | 82.08% ± 0.15% | 48.36% ± 1.49% | 6318 ± 24 | 58.88% ± 0.23% | 36.16% ± 1.3% | 52.23% ± 0.17% | 52.48% ± 0.57% | 52.92% ± 0.41% | 52.92% ± 0.41% | 51.3% ± 0.36%  |\n| Jieba | 71K | 78.61% ± 0.15% | 43.8% ± 1.27% | 5677 ± 22 | 57.29% ± 0.21% | 37.78% ± 1.37% | 53.66% ± 0.14% | 56.5% ± 0.54% | 52.59% ± 0.36% | 54.31% ± 0.31% | 53.08% ± 0.29%  |\n| TopWORDS | 54K | 78.7% ± 0.25% | **90.15%** ± 0.94% | 7080 ± 27 | 71.53% ± 0.29% | **89.84%** ± 1.01% | 63.68% ± 0.21% | 63.31% ± 0.46% | 62.34% ± 0.32% | 64.99% ± 0.5% | 64.17% ± 0.32%  |\n| TopWORDS-Seg | 86K | 83.51% ± 0.14% | 54.05% ± 1.51% | 7106 ± 19 | 67.88% ± 0.25% | 45.2% ± 1.54% | 60.75% ± 0.2% | 60.01% ± 0.5% | 61.34% ± 0.31% | 60.53% ± 0.43% | 60.72% ± 0.4%  |\n| RandomPattern | 96K | 95.24% ± 0.11% | 0.0% ± 0.0% | 8015 ± 24 | 81.26% ± 0.23% | 0.0% ± 0.0% | 69.11% ± 0.22% | 63.78% ± 0.47% | 64.74% ± 0.4% | 72.72% ± 0.55% | 72.56% ± 0.23%  |\n| SelectPattern | 73K | 94.33% ± 0.13% | 0.0% ± 0.0% | 7688 ± 21 | 88.89% ± 0.14% | 0.0% ± 0.0% | 84.11% ± 0.17% | 81.14% ± 0.53% | 81.53% ± 0.27% | 86.48% ± 0.33% | 85.97% ± 0.26%  |\n| TopWORDS-Poetry | 82K | 96.32% ± 0.06% | 84.47% ± 1.08% | 7803 ± 21 | 89.09% ± 0.2% | 63.28% ± 1.64% | 84.5% ± 0.14% | **81.53%** ± 0.46% | 81.91% ± 0.26% | 86.47% ± 0.42% | 86.54% ± 0.2%  |\n| TopWORDS-Poetry $^*$ | 82K | **96.39%** ± 0.06% | 83.57% ± 1.24% | 7828 ± 24 | **89.47%** ± 0.17% | 57.07% ± 1.57% | **84.73%** ± 0.17% | 80.72% ± 0.45% | **82.67%** ± 0.3% | **86.82%** ± 0.42% | **86.69%** ± 0.21%  |\n| Difference between TopWORDS-Poetry$^*$ and next best method |  | 1.15 | -6.58 |  | 0.58 | -32.77 | 0.62 | -0.42 | 1.14 | 0.34 | 0.73  |\n| p value by paired 2-sample t-test  |  | 9.57E-23 | 2.22E-19 |  | 4.10E-13 | 1.15E-28 | 1.06E-12 | 2.25E-03 | 2.80E-17 | 7.85E-06 | 6.27E-12  |\n\n\n**Q5**. did you try to see how a GPT style model performs your task using in context learning?\n\n**Response**：Thanks for this insightful suggestion! To evaluate the performance of GPT-style models on segmenting classical Chinese poetry, we asked ChatGPT (model version: gpt-35-turbo) to segment all poetry lines in the testing dataset $P_{1000}$ under the following prompt template: \n\n\"请对<>中的句子进行分词，用\"/\"隔开，不需要提供任何解释。  \n输入：<功蓋三分國。名成八陣圖。江流石不轉。遺恨失吞吳。>  \n回答：功蓋/三分國。名成/八陣圖。江流/石/不/轉。遺恨/失/吞吳。  \n输入：<[目标文本]>\"\n\nBelow is the English translation of the above prompt template:  \n\n“Please do text segmentation for the sentences in <>, separated by \"/\", without providing any explanation.  \nInput: < His deeds covered a kingdom split in three, his fame completed the Plan of the Eight Formations. The River flows on, the stones do not turn, a remnant bitterness at his failure to swallow Wu.>  \nAnswer: His deeds covered / a kingdom split in three /, his fame completed / the Plan of the Eight Formations. The River flows on /, the stones / do not / turn /, a remnant bitterness / at his failure / to swallow Wu.  \nInput:<[Target Text]>\n\n| Method | $R_s^{(t,l)}$ |  |  | $F_S$ |  |  |  |   |\n|---|---|---|---|---|---|---|---|---|\n|  | $\\|V_s\\|$ | $R_s^{(2,2)}$ | $R_s^{(2,3)}$ | $P_{1000}$ | $P_{250}^{4\\times5}$ | $P_{250}^{8\\times5}$ | $P_{250}^{4\\times7}$ | $P_{250}^{8\\times7}$  |\n| LTP | 2.5K | 19.28% | 29.94% | 20.00% | 21.16% | 18.72% | 21.72% | 19.65%  |\n| StanfordNLP | 6.0K | 43.56% | 45.54% | 39.49% | 40.45% | 40.26% | 41.07% | 37.80%  |\n| THULAC | 6.7K | 53.28% | 35.03% | 49.08% | 50.27% | 50.14% | 49.25% | 47.78%  |\n| PKUSEG | 7.6K | 59.26% | 36.31% | 52.17% | 52.62% | 52.96% | 52.65% | 51.20%  |\n| Jieba | 6.8K | 57.30% | 37.58% | 53.62% | 56.40% | 52.60% | 54.23% | 53.04%  |\n| TopWORDS | 8.5K | 71.22% | **89.49%** | 63.71% | 63.34% | 62.49% | 65.08% | 64.07%  |\n| TopWORDS-Seg | 8.6K | 68.06% | 44.90% | 60.67% | 60.26% | 61.27% | 60.35% | 60.55%  |\n| RandomPattern | 9.7K | 81.61% | 0.00% | 69.09% | 63.62% | 64.70% | 72.80% | 72.56%  |\n| SelectPattern | 9.2K | 88.71% | 0.00% | 84.12% | 81.27% | 81.57% | 86.46% | 85.91%  |\n| TopWORDS-Poetry | 9.3K | 88.98% | 62.74% | 84.48% | **81.50%** | 81.95% | 86.49% | 86.46%  |\n| TopWORDS-Poetry $^*$ | 9.4K | **89.36%** | 56.69% | **84.71%** | 80.68% | **82.70%** | **86.85%** | **86.63%**  |\n| **ChatGPT** | 8.0K | 77.12% | 66.24% | 73.48% | 72.19% | 73.17% | 71.99% | 74.96%  |\n\n\nAfter summarizing the results output by ChatGPT, we found that the $R_S$ and $F_S$ scores of ChatGPT concentrate in a narrow region of around 70%, leaving a wide performance margin (>10%) with respect to TopWORDS-Poetry. Such results confirm that TopWORDS-Poetry achieves the SOTA on processing classical Chinese poetry, even when GPT-style models are taken into consideration.\n\n\n**Q6**. There is a lot of work on Bayesian word segmentation which the authors do not cite https://www.semanticscholar.org/paper/A-Bayesian-framework-for-word-segmentation%3A-the-of-Goldwater-Griffiths/46f31f9069bb934498a288126053bcab01ff34aa https://aclanthology.org/P09-1012/ https://link.springer.com/article/10.1007/s11168-011-9074-5\n\n**Response**：Thanks for listing these references missing in the current manuscript. Based on our investigation, these references have been well discussed by Pan et al. (2022). We would add these references into the manuscript with discussions if the paper is fortunately accepted.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The authors present a Bayesian unsupervised model of word segmentation which is a nice respite from supervised neural network models\n- The model might be useful of other segmentation tasks (e.g., for documents)\n- The paper servers as a good introduction to unsupervised word segmentation", "reasons_to_reject": "- Slightly impenetrable, very young readers will struggle with the Bayesian formulation as formulas are presented without any intution\n- The task and model are rather niche \n- Added value on Peng et al. (2002) is unclear to me. The current model seems like an extension", "questions_to_authors": "- Please explain why the segmentation tool you are developing is required. In particular who is it for? Is it for scholars studying these\npoems? Would the scholars have issues with unknown words and with segmentation? I would have thought it is their job to understand these poems. So if it is not scholars you are targetting who is it? Do you want to help lay people understand these poems? Is it common? Or you are developing a literary analysis tool? \n- I am having trouble understanding how word discovery is different from text segmentation. Isn't segmentation in Chinese inserting boundaries between characters which would in turn give you word discovery?\n- Can you please explain clarify part of your model is different from Peng et al (2022). Is it a prior that also takes into account segmentation patterns? In other words equation (3)?\n- It would be good to report aggregate scores in Figure 3(a) and also whether differences between TopWORDS-Poetry(*) and the next best comparison system are statistically significant. \n- did you try to see how a GPT style model performs your task using in context learning?", "paper_topic_and_main_contributions": "This paper focuses on devising tools for processing classical Chinese poems. It is argued that standard Chinese processing tools are not directly applicable to classical poems as they contain many new words, named entities, and poetry specific patterns. The authors present a Bayesian unsupervised method of discovering new words in classical Chinese poems --- it is essentially a word segmentation problem. The authors propose an improvement of the method of Pan et al. (2022). The proposed method starts with a heuristic segmentation based on an initial word dictionary and the most frequent segmentation patterns. Prior distributions are then specified  based\ngiven segmentation patterns;  a word discovery stage estimates word boundaries  and prunes the initial word dictionary D (see  Figure 2 for details). The target posterior is estimated by introducing latent variables and using an EM-style algorithm (derived in Appendix A).", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "There is a lot of work on Bayesian word segmentation which the authors do not cite\nhttps://www.semanticscholar.org/paper/A-Bayesian-framework-for-word-segmentation%3A-the-of-Goldwater-Griffiths/46f31f9069bb934498a288126053bcab01ff34aa\nhttps://aclanthology.org/P09-1012/\nhttps://link.springer.com/article/10.1007/s11168-011-9074-5"}}]

Author rebuttal:

Meta-review: This paper presents an unsupervised Bayesian model designed for text segmentation and word discovery. They extend an existing method, with the idea that the regularity of the poetry's meter can be incorporated as a Bayesian prior. The proposed method shows good performance both on word discovery and text segmentation.

The reviewers concur that the authors have introduced a clever idea that is empirically justified. Personally, I love to see a new Bayesian model in 2023. While younger readers may face challenges in comprehending the approach, this should not pose a barrier to its acceptance.
Decision: Accept-Main