GreedyCAS: Unsupervised Scientific Abstract Segmentation with Normalized Mutual Information
Authors: Yingqiang Gao
Jessica Lam
Nianlong Gu
Richard Hahnloser
Venue: EMNLP 2023 Main (2023)
Keywords: Abstract Segmentation
Mutual Information
Argument Mining

Abstract: The abstracts of scientific papers typically contain both premises  (e.g., background and observations) and conclusions. Although conclusion sentences are highlighted in structured abstracts, in non-structured abstracts the concluding information is not explicitly marked, which makes the automatic segmentation of conclusions from scientific abstracts a challenging task. In this work, we explore Normalized Mutual Information (NMI) as a means for abstract segmentation. We consider each abstract as a recurrent cycle of sentences and place two segmentation boundaries by greedily optimizing the NMI score between the two segments, assuming that conclusions are strongly semantically linked with preceding premises. On non-structured abstracts, our proposed unsupervised approach GreedyCAS achieves the best performance across all evaluation metrics; on structured abstracts, GreedyCAS outperforms all baseline methods measured by $P_k$. The strong correlation of NMI to our evaluation metrics reveals the effectiveness of NMI for abstract segmentation.

GreedyCAS: Unsupervised Scientific Abstract Segmentation with Normalized Mutual Information
Yingqiang Gaot, Jessica Lam', Nianlong Gu Richard H.R. Hahnloser
Institute of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland (yinggiang, gan, lamjessi rich @iniiettz...............e.... LLnguisstic Research Infrastructure, University of Zurich, Switzerland nianlong, guêuzh.ch
ethz.ct
Abstract
start of the conclusions
The abstracts of scientific papers typically con- tain both premises (e.g., background and ob- servations) and conclusions. Although conclu- sion sentences are highlighted in structured ab- stracts, in non-structured abstracts the conclud- ing information not explicitly marked, which makes the automatic segmentation of conclu- sions from scientific abstracts a challenging task. In this work, we explore Normalized Mu- tual Information (NMI) means for abstract segmentation. We consider each abstract as a recurrent cycle sentences and place two segmentation boundaries by greedily optimiz- ing the NMI score between the two segments, assuming that conclusions are strongly seman- tically linked with preceding premises. On non- structured abstracts, our proposed unsupervised approach GreedyCAS CAS achieves the best perfor- mance across all evaluation metrics; struc- tured abstracts, GreedyCAS outperforms baseline methods measured by Pt. The strong correlation of NMI to our evaluation metrics reveals the effectiveness of NMI for abstract segmentation.'
conclusions
start/end of the abstract
premises
stitch
Cyclic Abstract Segmentation
Figure Left: An abstract that contains premise (blue) and conclusion (red) sentences. The task is identify the start and end segmentation boundaries of the con- clusions. Right: We regard each abstract as recurrent cycle of sentences by stitching its start and end together. Best view color printing.
Many abstracts, especially those from the biomedical domain, are structured to help the reader extract the conclusions (e.g.. abstracts fol- low the IMRaD format (Nair and Nair, 2014; Der- noncourt and Lee, 2017) or the CONSORT format (Hopewell et al., 2008)). contrast, abstracts from many other research domains do not explicitly in- dicate the position of conclusions, which means that readers must perform the potentially cogni- tively demanding task of identifying the conclu- sions themselves. We are therefore interested in probing approaches for splitting scientific abstracts into conclusion and premise sentences.
1 Introduction
Abstracts of scientific papers are short texts that summarize the findings reported in the body text (Bahadoran et al., 2020). A well-formulated ab- straet forms scientific inference that extends from premises (e.g., shared knowledge, experimental ev- idence, or observation) to conclusions (e.g., sugges- tions, claims, Ripple et al. (2012)). splitting of an abstract into a conclusion segment and a premise segment can help readers better comprehend how conclusions are drawn (Bahadoran et al., 2020) and is of interest for downstream research tasks such as argument generation (Schiller al., 2021), knowl- edge retrieval (Hua et al., 2019), opinion analysis (Hulpus al., 2019), and text summarization (Cho et al., 2022).
Existing text segmentation approaches (Soma- sundaran et al., 2020; Lo et al., 2021; Barrow et al., 2020; Koshorek et al., 2018) 1) can be applied to sci- entific abstract segmentation. However, fine-tuning such models typically requires large amounts of la- belled data that are expensive collect. In contrast, unsupervised approaches require no annotated data and can segment large numbers of texts with mini- mal human involvement. Thus, we primarily test
'Code and data available at https::/// ://p://///// www.gthh.boe.eo com/ wtt.aarrddaddde//Geeeeeeee

---

unsupervised frameworks for segmenting scientific abstracts.
Unsupervised approaches (Alemi and Ginsparg. 2015) usually make use of metrics based on tex- tual coherence or topic contiguity and take the following strategies: 1) to greedily seek the best segmentation based on text similarity at each step (Choi, 2000; Hearst, 1994); and 2) to iteratively approach global optimum of segmentation ob- jective (c.g. semantic relatedness) via dynamic programming (Fragkou al., 2004; Bayomi and Lawless, 2018). These methods use similarity mea- sures or lexical frequencies between segments determine segmentation boundaries or convert the enter-sentence similarities into a semantic graph and perform graph search (Glavaš et al., 2016) to find the optimal segmentation.
Given a set of abstracts, we want to determine their splits into premise segments and conclusion segments. Combining the premises from all ab- stracts gives a premise set, and similarly, obtain conclusion set from combining all conclu- sion segments. We hypothesize that the abstracts are best segmented when the Normalized Mutual Information (NMI) between the conclusion set and premise set is maximized. Our intuition is that con- clusions follow from the remainder of the abstract, which is redundancy that is well captured by mu- tual information. To maximize NMI, use exhaustive greedy approach that iterates over all abstracts and determines the best segmentation for each. To test how NMI deals with known text boundary, the end of an abstract, we stitch the start and end of each abstract together to form a cycle, then select two segmentation boundaries with con- straints based on prior knowledge (see Figure 1). We name our approach Greedy Cyclic Abstract Segmentation (GreedyCAS).
Supervised methods are usually deployed when sufficiently many annotated examples available. These methods typically use language models encode sentences and perform binary classifica- tion to predict whether a sentence is on the seg- mentation boundary (Somasundaran et al., 2020; Banerjee et al., 2020; Aumiller et al., 2021; Bad- jatiya et al., 2018; Lukasik et al., 2020; Koshorek et al., 2018). Banerjee et al. (2020) fine-tuned hierarchical sentence encoder using structured abstracts to classify sentences into discourse cat- egories (BACKGROUND, TECHNIQUE, and OB- SERVATION). In general, supervised approaches achieve good performance, but they consume a large amount of annotated data, which is expensive to collect. In this work, we set out to test generic unsupervised approaches that require no training data.
To test our proposed approach, we create two datasets. One dataset comprises non-structured ab- stracts with human-annotated conclusion sentences. The other dataset contains structured abstracts in which conclusion sentences have been explicitly marked by the authors of the abstract.
Our main contributions are as follows:
We propose GreedyCAS, an unsupervised ap- proach for scientific abstract segmentation that optimizes NMI.
3 Methodology
On dataset of non-structured abstracts, we show that GreedyCAS achieves promising segmentation results.
We formulate the task of segmenting scientific ab- stracts as follows. Given an abstract A = (s:) containing sentences, we define C = و
as the set of all ma possible segmentations of A. Each segmentation 9 (PA) C3²) con- tains premise segment Pá and a conclusion seg- ment CdA. with boundaries given by two indices ag E3 E Nin (range of integers from 1 to n):
We find a strong correlation between NMI and other evaluation metrics, in support of NMT being useful for segmentation.
2 Related Works
C = (=: af sis
Abstract segmentation is particular case of text segmentation. The task of text segmentation is to insert separation markers into the text such that the segmented fragments are topically coherent and comprehensive (Hazem et al., 2020). Traditional methods can be categorized into supervised and unsupervised methods.
PA = { 8 & CAA.
Associated with a corpus A. {Ai)ieria of A abstracts, there is set of G -GG) HENt
of I1 mA, possible segmentations. Search- ing for the best ensemble of segmentations of A

---

prior configurations
substitute
compute PMIP:C) for each config.
config. with NMI(F.C)
abstracts with initial segmentations
randomly fetch one abstract
randomly fetch the next abstract
iterate until N abstracts are felched
update
construct premise and conclusion ensemble
Figure 2: Our proposed GreedyCAS pipeline for segmenting scientific abstracts. Premise sentences are colored in blue, whereas conclusion sentences are colored in red. Best view color printing.
within G may involve exhaustive enumeration over Ih M,, segmentations, which is impossible der limited computational costs. Therefore, in this work, we concentrate on greedily approaching reasonably good segmentation that is tight lower bound of the actual global optimum.
config.
laheling
0001001
1000100
0100010
3.1 Cyelic Abstract Segmentation
config.
To reduce the search space, make two assump- tions: firstly, that conclusion sentences are located the end of abstracts; and secondly, that each ab- stract contains at most three conclusion sentences. This results in m possible segmentations per abstract. The segmentations an example abstract with n = 7 sentences are depicted in Table I. Because scientific abstracts typically end with conclusion sentences, we expect the stitching point of our cyclic abstracts to form boundary. In other words, we read out the segmentation of interest from the first segment boundary a² the sccond segment boundary E expect to coincide with the abstract end. Thus, by optimizing the second segment boundary E/¹, we perform a sanity check that the unsupervised segmentation method is capa- ble of detecting the abstract end, which is a natural boundary of the abstract.
labeling
0000101 1000010
000001
Table Labeling of segmentation boundaries for an example abstract with seven sentences, among which at most three sentences are conclusion sentences (colored in red). 1 indicates sentence the segment boundary. whereas O) indicates a non-hounde non-houndary sentence. We aim to reduce the complexity by searching within six possible segmentations for each abstract.
Mutual information I(X;Y) a measure of the absolute reduction in information uncertainty (in bits) for a random variable X after observing an- other correlated random variable Y. Our proposed greedy approach is based on the assumption that the uncertainty of the conclusion is minimized after the premise is observed, i.c. that the segmentation maximizes mutual information.
3.2 Normalized Mutual Information
Our next step is to choose optimization objec- tive for the greedy search. Inspired by work text summarization (Padmakumar and He, 2021) and birdsong analysis (Sainburg et al., 2019), we explore mutual information as the optimization ob- jective.
We denote C {CA, HENT possible con- clusion ensemble spanned by all conclusion seg- ments from the k abstracts, and P {PAA Jen,a as one possible premise ensemble obtained the same way. Note that the segmentation js can be different for each abstract A;

---

The task can now be formulated as follows: given a corpus of abstracts A. determine the premise and conclusion ensembles that maxi- mize the mutual information (IP; C). More formally, we compute [PP; C)) as follows:
Here, we have
Up = Σ p(up) (ogp(wp) = (IP)
up
and
C(P.C)=
Uc - Ep(wp log log H(C)
Σ Σ Σ plupi we) log p(up: we) p (Ww))(w)))
essentially being the entropy of the premise ensem- ble and the conclusion ensemble, respectively. For the least upper bound (a = -X),, we have
A,EA upCP w.toy
where Wp and We are unigram tokens in the i-th premise segment 3 and the i-th conclusion seg- ment CA, respectively. p(wp;ws) indicates the joint probability of the premise word Wpp appear- ing in the premise segment PA, and the conclu- sion word we appearing in the conclusion segment CrA. p(up)) and p(w,) denote marginal probabili- ties. Making use of language modeling statisties, we compute the marginal probabilities as follows:
U-x = lim Ua = min (c))
a--20
In this work, we use U oo to normalize [[P; C) to ensure that the maximal attainable NMI value is 1. This brings us the benefit of comparable NMI scores for different corpus sizes k.
3.3 Greedy Cyclic Abstract Segmentation
We now introduce our GreedyCAS approach to segment abstracts of scientific papers. GreedyCAS performs search, where we first explore the best segmentation of particular abstract that maxi- mizes NMI C), then iterate over all abstracts perform the same maximization.
p(wp) = dwp.PP P)
Eng, dwy, PP) c(wo:9 ()
p(we) Eng cur C)'
where P(w,P) denotes the number of occurrences of w within the tokenized premise segments in P and w/, is a token from the premise segment of any abstract. The terms c(w, C) and w' are defined analogously.
Algorithm I deseribes the basic segmentation ap- proach GreedyCAS-base. Given the input abstract corpus A, the algorithm greedily searches for the segmentation that leads to the maximal NMI(P, C). The output is the optimized segmentation G* Algorithm 2 illustrates the advanced approach GreedyASS-NN, where we first split the abstract corpus A into a series of chunks (denoted as A chhukk in size of c; then, for each seed abstract Aí, sampled from the current chunk A chunk, we perform embedding-baseee based nearest neighbour (NN) search within the chunk to construct the batch (de- noted as hatch) comprising the most semanti- cally relevant abstracts for AG, by computing the cosine similarity using their abstract embeddings:
The joint probability is then computed as E Pi C E E(ug, wo)) clwy, P)c(w', C)
puupi we)
Because mutual information is an unbounded measure that increases with the size of A, is not directly comparable across different I and en- sembles (Poole et al., 2019). We therefore normal- ize I(P; C) by mapping it onto the interval [o, 1] and use Normalized Mutual Information (NMI) as the final optimization objective.
AA A chunk
Taken from Kvålseth (2017), we compute NMI follows:
e(Ag) e((A)
sin,A [|e(A;))] I11 -|e(AA]]]]]]
NMI(P; C) = I(P: C)
A batch = =f chuukk rank (sim (A A)) < b)
Uo
where U₂ denotes the non-decreasing theoretical upper bound of (P; C) and is parametrized by the a-order arithmetic mean
We use a pre-trained Sentence-BERY model? (Reimers and Gurevych, 2019) to acquire the ab-
"We use the sentence-trunssormers encoder (pre-trained model all-Mini.M--L-- with model size 80 MB). Apache-2.0 I License, available at gf thub. cct/UKP1.ab/ sentence trans formers
Uo (
Mu + Uc
1/a

---

Algorithm 1: GreedyCAS-base base: unsuper-
Algorithm 2: GreedyCAS-NN: unsuper-
vised cyclic abstract segmentation
vised cyclic abstract segmentation with nearest neighbor search
Input: abstract corpus (A
Output: optimized segmentation G = {G* Jeeu
Input: abstract corpus A = (A.)er, chunk size
A {- A:
G, batch size b
while A # Ø do
Output: optimized segmentation
0 < 0:
optimization objective.
G" (G* heNTa
G" + Ø;
G x- 2:
A; 4 sample(A):
{Aqtum JiENr.e, + truncate(AAc]:
Am t A (AG
A chhnk { HIEN11
[9 J5=1 t- configure(A));
4. while len(G') + do
GAA < 19
foreach A, chmt do
foreach epoch do
G + 2:
G + Ø;
6
10
{AN, + sample(A, chumk
11
foreach A, E A do
10 (0,"))
contigure(AA):
8
AC +- be
seed abstracts
foreach Ai, A do
0," A sample(10) }=:);
(4i, ImeN +- NN--sarch((
14
Gh, t ( 1:
11
Abatch + {A
15
G - G UGAr;
12
Ghach SrredyCAS-bass(AAA (A))e));
end
Gehunk +- Gehank U Chach
17
foreach
9/ E Gh do 93t =
end
18
G" G' U UG
19
foreach G E eGa do
end
9 Gtv;
end
9j = (pfr (
is return G"
21
22
end
Phu u (1ftt)
C C, CuiUCC UOC
entific papers on SARS-CoV-2 coronavirus-related research published since March 2020. These papers share higher lexical commonality than biomedical papers in general due to the focused research inter- est in COVID-19.
25
Og + (MMpptte-NM((P: C);
26
2 > 0g then
27
0 - Oz:
G < 4 UCA.
28
29
end else
30
We worked with abstracts whose sentences have been categorized into BACKGROUND, METHODS, RESULTS, and CONCLUSION discourse categories, We trusted the categories of these structured ab- stracts from the CORD-19 corpus since scientific papers are peer-reviewed and multi-round revised. We automatically aggregated the dataset CAS- auto from 697 structured scientific abstracts whose paper titles contained the keyword vaccine. In- spired by Shieh et al. (2019), we took sentences in BACKGROUND, METHODS, RESULTS categories as premises, and sentences in the CONCLUSION category as conclusions.
continue;
end
end
34
end
35 end
36 return G"
stract embeddings. Finally, the same greedy strat- egy as described in GreedyCAS-base is applied to find the best segmentation for each abstract of the batch. To fully utilize the power of parallel computing, we use multi-threading? to optimize NMI.
In addition, we manually constructed a dataset CAS-human of 196 non-structured abstracts from CORD- 19, using the keyword antigen to find target abstracts. We then asked four human annotators to label the conclusion sentences within those ab- stracts. All human annotators were not instructed about the potential positions of conclusion sen- tences scientific abstracts. By doing this, we avoided biasing them. To facilitate the annotation process and reduce the annotators' workload, we
4 Dataset
Since we calculate NMI scores using lexical co- occurrences of words, we constructed a corpus of related abstracts based on the COVID-19 Open Re- search Dataset (CORD-19) released by Wang et al. (2020). This dataset is massive collection of sci-
*Wo use the Python MultiThreadin library https:/ org/3//////// /3/ Abbrry/ rarrary/tat htte/threaaddee ng. html
docs docs.python python. wowhhoooorr/////oo

---

used the interactive data labeling platform Doc- cano" (Nakayama al., 2018) for constructing the CAS-human dataset.
conclusion positions that align with our prior knowledge
160
outliers
Table 2 shows the overall statistics of our pro-
140
#sentence
posed datasets for scientific abstract segmentation. During data preprocessing,
4-
120
intentionally re-
#sentence
100
2
moved stop words, numbers, and punctuations (ex-
80
cept ",, which is essential for the sentence tok-
5 conclusion positions
8
enizer we used) in the abstracts. We also low-
60
ercased all tokens in both datasets for increased computational efficiency.
20
-2
1
2
Dataset abs.
# con.
pre.
avg. labs.]
conclusion positions
CAS-auto 697
1,267
4,755
8.64
CAS-human
196
263
1,220
7.57
Figure 3: Statistics the positions of the conclusion sentences within the abstracts in the PAS-human dataset. The minus sign denotes the positions counting from the end of the abstract (-1 denotes the sentence of the abstract, -2 the second last sentence, and so on).
Table 2: Statistics
two datasets. abs. denotes the number of abstracts, # con. and # pre. indicate the number of conclusion and premise sentences, respec- tively. avg. Jabs.| average number of sentences of the abstracts.
measures. Finally, we use Jaccard index to mea- sure the similarity between the set of segmented conclusion sentences and the set of ground-truth conclusion sentences. Lower P3: and WD scores in- dicate better segmentation results, whereas higher ROUGE and Jaccard indexes represent better seg- mentation results.
Figure 3 shows the positions of the conclusion sentences within the abstracts in the CAS-human dataset as labelled by the human annotators. Simi- larly, as shown in previous works (Fergadis et al.. 2021; Achakulvisut et al., 2019), in 95% of our annotated non con-structured abstracts, the positions of conclusion sentences consistent with prior assumption (they were among the last 3 sentences of the abstract).
5.2 Baselines
We present three unsupervised baseline methods for abstract segmentation as baselines. To ensure the comparability of the results, we manually added an additional segmentation boundary at the end of the abstract for any approach that provides only one boundary.
5 Evaluation
5.1 Metrics
To evaluate the segmentation results, we use both set similarity and textual relevance as metrics. We test the performance of our approaches on both automatically (CAS-auto) and manually (CAS- human) aggregated data. To evaluate the segmenta- tion boundaries,
Random To test prior knowledge of the po- sition of conclusions sentences, set up two dom baselines: a) Random-base: following the idea initially proposed by Beeferman et al. (1999), we place segmentation boundaries after two randomly selected sentences; b) Random--plas we segment an abstract by randomly selecting one from the six possible segmentations described in chapter 3.1.
text segmentation metrics P& (Beeferman et al., 1999) and WindowDiff (WD, Pevzner and Hearst (2002)). Then, we use ROUGE score (Lin, 2004) to measure the textual relevance between the segmented and ground conclu- sion sentences. We compute the arithmetic mean of ROUGE-1, ROUGE-2, and ROUGE-Lsum f-
TextTiling" As proposed in Hearst (1997) and serving as the classic text segmentation approach, TextTiling utilizes lexical information to detect topic changes within given text. In case, Text-
'MIT License, available https:/ /gi twu.hhub.ccc com/ doccano/ doccano/doccane doccano
"We used the sentence-splitter by Philipp Koehn and Josh Schroeder, GNU Lesser General Public License, available at https://f //////ttbbb med ac ouu/ Sentence-splicee
"We use the HarvestText implementation for the English language, MIT License, available at https: thub. com/blmoi st

---

Tiling places segmentation boundary between sentences.
abstracts. The results are shown in Table 4.
CAS-auto
Pk. 4
WD 4
Jaccard f
ROUGE 1
SBERT-sim Inspired by Solbiati et al. (2021), we use the same Sentence-BERT encoder as GreedyCAS-NN to segment abstracts using sen- tence semantics. Each abstract is split into two segments such that the cosine similarity of their Sentence-BERT embeddings is maximized.
Random-buut buse
.3958
.4990
.1169
.278
Random-plus
.2022
.2251
.5171
.6569
TextTiling SBERT-sim
.242
.3311
.0099
.5271
.1930
2101
.6013
.7274
GreedyCAS CCSS-
.1656
.2341
.4878
.8366
GreedyCAS-NN'
.6622
.2317
.4300
.5717
Table 4: Segmentation results on the CAS-auto dataset. The best results are statistically significantly better than the closest approach.
6 Results and Discussion
6.1 Non-structured Abstracts
First, we test our supervised GreedyCAS ap- proaches against the baselines the CAS-human dataset of non-structured abstracts. We list the ex- periment results in Table 3. For the GreedyCAS approaches, we report the empirical performance with the best batch size. The best batch size hints at how many closely related abstracts together can bring benefits to the estimation of word probabili- ties, and essentially, the final segmentation results. Intuitively, the batch size grows, the relatedness among the abstracts drops because is not possible get large number of abstracts that study pre- cisely the same research question. For the Random baselines, we report the best segmentation results from 11 random trials.
First, we found that compared the Random- base baseline, Random-plus improves the results across different measures by large margins.
We then observe that the SBERT-sim baseline, which segments abstracts based on the cosine sim- ilarity between the premise and conclusion seg- ments, achieves the best performance on three out of four metrics by large margin. Our best model GreedyCAS-NN only achieves the leading perfor- mance measured by Pk. while achieving lower per- formance measured by other metrics. Table 5 and in appendix B, we show two abstracts that were wrongly segmented by GreedyCAS-NN: the first abstract has one additional sentence from the B BCKGROUND category, whereas the second ab- stract has one additional sentence from the RE- SULTS category. These sentences were misat- tributed by GreedyCAS to the conclusion segment because they increase the NMI score; however, due to the complexity of NMI, understanding the exact reasons why NMI increases is non-trivial.
CAS-human Ps L
Jaccard t
ROUGE f
Random-base
.293
.5366
.0642
.1925
Random-plee -luus
2594
.3678
.4724
6115
TexTTiling
2555
.3444
.4971
6153
SBERT-sim
3009
.4199
.4491
.5355
GreedyCAS-hase!
.1937
.3089
.5670
.6631
G
.1655
.2543
6020
Table Segmentation results on CAS-humaa dataset. J indicates that the lower the value is, the better the performance, whereas f means the opposite. The superscripts of GreedyCAS approaches indicate the best empirical hatch size. The hest results are statistically significantly better than the closest baseline (Wilcoxon signed-ranked test).
6.3 Analysis
In Figure 4, we compute the correlation coefficients between NMI scores and each evaluation metric. We plot the NMI scores and the evaluation metrics w.r.t the batch size (ranging from 2 to 12).
Figure 4 shows that NMI scores are strongly neg- atively correlated with the text segmentation met- rics P& and WindowDif? but are strongly positively correlated with the set similarity metric Jaccard in- dex and the lexical metric ROUGE.
We see that the GreedyCAS-NN achieves the best performance. The results suggest that this approach works well on non-structured abstracts. Thus, NMI is able capture the conclusion- relevant information at both the abstracts' begin- ning and end.
In Figure 5, rather than studying the complex sce- nario where entire sentence gets re-attributed, showcase the impact on NMI when just one word moved from the premise segment the conclusion segment. We studied whether placing segmentation boundaries between words can pro- vide similar NMI(P; C) scores compared to placing
6.2 Structured Abstracts
Next, we test GreedyCAS approaches against the baselines on the CAS-auto dataset of structured

---

mentation boundaries after complete sentences might be the optimal choice when optimizing with NMI.
0.300
Jaccard Index
Pe score
0.250
0.95
0.600
r= 0.82
0.200
0.500
7 Conclusion
0.150
0.30
0.40
0.30
0.40
In this explorative work, we propose an unsuper- vised approach, GreedyCAS, for automatically seg- menting scientific abstracts into conclusions and premises. We introduce the cyclic abstract segmen- tation pipeline, which be applied structured and non-structured abstracts. Our approach lever- ages the lexical information between words that co-occur the conclusion and premise segments and finds the best segmentation a set abstracts using NMT as an optimization objective. Our empir- ical results show that NMI is an effective indicator for the segmentation results of scientific abstracts.
nomalized MI
normalized MI
*+
ROUGE score
0.700
WD score
0.88
0.75
0.350
0.650
0.300
0.600
0.250
0.30
normalized MI
normalized
Figure Correlation coefficients between NMI and other metries. We fit linear regression models the data points. indicates the significance level p < 10 and #* p 10 (Pearson correlation test).
Limitations
In this work, we explored the use of normalized mutual information for scientific abstract segmentation. The main limi- tations of our work are listed below:
them within sentences. To do this, we randomly pick one abstract from the CAS-human dataset and calculate the changes in NMI(P; C) due to the re- location of the segmentation boundary caused by word time moved the right.
optimization objective
The input abstracts GreedyCAS need to be on similar research topics; otherwise, their shared vocabulary is limited and the word probabilities in the computation of NMI can- not be estimated well.
1e-4
change normalized MI
0
GreedyCAS has a high time and space com- plexity because it involves searching for the best segmentation and enumeration over all possible word pairs at each iteration. As a result, GreedyCAS takes long time to exe- cute and using larger batch sizes is a challenge with limited computational resources.
-2
0
50
100
150
1e
3.55
original normalized aalizddd
3.54
0
We did not try assess the reliability of word- pair probability estimation. Presumably, the more abstracts are considered, the better the probability estimates of frequent word pairs, but the more infrequent outlier pairs creep with biased probability estimates. Thus, it seems not obvious that more abstracts will necessarily yield better probability and mutual information estimates.
50
100
word position
Figure 5: Change in NMI(P: C) when moving one word from the premise to the conclusion segment for a fixed abstract. Dashed lines denote the end positions of sen- tences in the abstract. We smooth the data with a Gaus- sian filter with o
We see in Figure 5 that the slope at word posi- tions near sentence boundary is not always steeper than at word positions within the sentences, which indicates that segmenting abstracts by putting seg-
In the future, we will analytically study how to increase the efficiency and the applicability of GreedyCAS by considering other ways of estimat- ing the word probabilities.

---

9
Acknowledgements
Sangwoo Cho, Kaiqiang Song, Xiaoyang Wang. Fei Liu, and Dong Yu. 2022. Toward unifying text seg- mentation and long document summarization. arXiv preprint arXiv:2210. 16422.
We acknowledge the support from Swiss National Science Foundation NCCR Evolving Language, Agreement No.5INF40 1000440110888 180888. We also thank the anonymous reviewers for their constructive com- ments and feedback.
Freddy YY Choi. 2000. Advances in domain indepen- dent linear text segmentation. In 1st Meeting of the North American Chapter of the Association for Com- putational Linguistics.
Franck Dernoncourt and Ji Young Lee. 2017. PubMed 200k RCT: a dataset for sequential sentence clas- sification in medical abstracts. In Proceedings of the Eighth International Joint Conference on Natu- ral Language Processing ( (Volume 2: Short Papers), pages 308-313, 13, Taipei, Taiwan. Asian Federation of Natural Language Processing.
References
Titipat Achakulvisut, Chandra Bhagavatula, Daniel Acuna, and Konrad Kording. 2019. Claim ex- traction in biomedical publications using deep dis- course model and transfer learning. arXiv preprint arXiv:1907.0090
Aris Fergadis. Dimitris Pappas, Antonia Karamolegkou, and Harris Papageorgiou. 2021. Argumentation min- ing in scientific literature for sustainable develop- ment. In Proceedings of the Sth Workshop on Argu- ment Mining, pages 100-111.
Alexander A Alemi and Paul Ginsparg. 2015. Text segmentation based on semantic word embeddings. arXiv preprint 1rX
Dennis Aumiller, Satya Almasian, Sebastian Lackner, and Michael Gertz. 2021. Structural text segmen- tation of legal documents. In Proceedings of the Eighteenth International Conference on Artificial In- telligence and Law, pages 2-11.
Pavlina Fragkou, Vassilios Petridis, and Ath Kehagias. 2004. A dynamic programming algorithm for linear text segmentation. Journal of Intelligent Information Systems, 13
Goran Glavaš, Federico Nanni, and Simone Paolo Ponzetto. 2016. Unsupervised text segmentation us- ing semantic relatedness graphs. In Proceedings of the Fifth Joint Conference on Lexical and Computa- tional Semantics, pages 125-130.
Pinkesh Badjatiya, Litton J Kurisinkel, Manish Gupta, and Vasudeva Varma. 2018. Attention-based neu- ral text segmentation. In European Conference on Information Retrieval, pages 180-193. Springer.
Zahra Bahadoran, Parvin Mirmiran, Khosrow Kashfi,
Amir Hazem, Béatrice Daille, Dominique Stutz- mann, Christopher Kermorvant, and Louis Cheva- lier. 2020. Hierarchical text segmentation for me- dieval manuscripts. In Proceedings of the 28th Inter- national Conference on Computational Linguistics. pages 024006251.
and Asghar Ghasemi. 2020. The principles of biomedical scientific writing: abstract and key- words. International Journal of Endocrinology and Metaholism, 18(1).
Soumya Banerjee, Debarshi Kumar Sanyal, Sami- ran Chattopadhyay, Plaban Kumar Bhowmick, and Partha Pratim Das. 2020. Segmenting scientific ab- stracts into discourse categories: deep learning- based approach for sparse labeled data. In Proceed- ings of the ACM/IEEE Joint Conference Digital Libraries 2020, pages 429 -29-
Marti A Hearst. 1994. Multi-paragraph segmentation expository text. In 32nd Annual Meeting of the Asso- ciation for Computational Linguistics, pages 9-16.
Marti A Hearst. 1997. Text tiling:
Segmenting text into multi-paragraph subtopic passages. Computational linguistics, 23( 3:33-64.
Joe Barrow, Rajiv Jain, Vlad Morariu, Varun Manju- natha, Douglas W Oard, and Philip Resnik. 2020. A joint model for document segmentation and segment labeling. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 313-322.
Sally Hopewell, Mike Clarke, David Moher, Blizabeth Wager, Philippa Middleton. Douglas G Altman, Ken- neth Schulz, and Consort Group. 2008. Consort for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS medicine, 5(1):220.
Mostafa Bayomi and Séamus Lawless. 2018 C-hts: A concept-based hierarchical text segmentation ap- proach. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
Xinyu Hua, Zhe Hu, and Lu Wang. 2019. Argument generation with retrieval, planning, and realization. In Proceedings of the 57th Annual Meeting of the As- sociation for Computational Linguistics, pages 2661- 2672.
Doug Beeferman, Adam Berger, and John Lafferty. 1999. Statistical models for text segmentation. Ma- chine learning, 177-2-177-210.
Toana Hulpus, Jonathan Kobbe, Christian Meilicke, Heiner Stuckenschmidt, Maria Becker, Juri Opitz, Vivi Nastase, and Anette Frank. 2019. Towards

---

explaining natural language arguments with back- ground knowledge. In PROFILESISEMEXC ISWC, pages 62-77.
Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. 2019. On variational bounds of mutual information. In International Con- ference on Machine Learning, pages 5171-5180. PMLR.
AP Huovila, Astrid M Eder, and Stephen D Fuller. 1992. Hepatitis b surface antigen assembles in a post-er, pre-golgi compartment. The Journal of cell biology.
Nils Reimers and Tryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing. Associa- tion for Computational Linguistics.
(116):1305-1333
Omri Koshorek, Adir Cohen, Noam Mor, Michael Rot- man, and Jonathan Berant. 2018. Text segmentation supervised learning lask. In Proceedings of the 2018 Conference of the North American Chap- ler of the Association for Computational Linguisties: Human Language Technologies, Volume 2 (Short Pa- pers). pages 469 473.
Anna M Ripple, James G Mork, John M Rozier, and Lou S Knecht, 2012. Structured abstracts in med- line: Twenty-five years later. National Library of Medicine.
Tarald O Kvälseth. 2017. On normalized mutual infor- mation: measure derivations and properties. Entropy, 19(1 1)1631...
Tim Sainburg, Brad Theilman, Marvin Thielk, and Tim- othy Q Gentner. 2019. Parallels in the sequential organization of birdsong and human speech. Nature communications, 10(1):1-11.
Blaženka Letinić, Marinela Contreras, Yael Dahan- Moss, Ingrid Linnekugel, José de la Fuente, and Lizette Koekemoer. 2021. Additional evidence on the efficacy different akirin vaccines assessed on anopheles arabiensis (diptera: Culicidae). Parasites & vectors, 14(1):1-13.
Benjamin Schiller, Johannes Daxenberger, and Iryna Gurevych. 2021. Aspect-controlled neural argument generation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 380-396.
Chin- Yew Lin. 2004. ROUGE: A package for auto- matic evaluation of summaries. In Text Summariza- tion Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
Alexander Te-Wei Shieh, Yung-Sung Chuang, Shang- Yu Su, and Yun-Nung Chen. 2019. Towards under- standing of medical randomized controlled trials by conclusion generation. In Proceedings of the Tenth International Workshop on Health Text Mining and Information Analysis (LOUHI 2019), pages 108-117.
Kelvin Lo. Yuan Jin, Weicong Tan, Ming Liu, Lan Du, and Wray Buntine. 2021. Transformer over pre- trained transformer for neural text segmentation with enhanced topic coherence. In Findings of the Associ- ation,for Computational Linguistics: EMNLP 2021, pages 5334-3340.
Alessandro Solbiati, Kevin Heffernan, Georgios Damaskinos, Shivani Poddar, Shubham Modi, and Jacques Cali. 2021. Unsupervised topic segmentation of meetings with hert embeddings. arXiv preprint urXiv:2106.1299
Michal Lukasik, Boris Dadachev, Kishore Papineni, and Gonçalo Simões. 2020. Text segmentation by cross segment attention. In Proceedings of the 2020 Con- ference Empirical Methods in Natural Language Processing (EMNLP)), pages 4707-4716.
Swapna Somasundaran et al. 2020. Two-level trans- former and auxiliary coherence modeling for im- proved text segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 34, pages 7797-7804.
PK Nair and Vimala D Nair. 2014. Organization of a research paper: The imrad format. In Scientific writ- ing and communication in agriculture and natural resources, pages 13-25. Springer.
Shufang Sun, Danhua Lin, and Don Operario. 2021. In- terest in covid-19 vaccine trials participation among young adults china: Willingness, reasons for hes- itancy, and demographic and psychosocial determi- nants. Preventive Medicine Reports, 22:101350.
Hiroki Nakayama, Takahiro Kubo, Junya Kamura, Yasu- fumi Taniguchi, and Xu Liang. 2018. doccano: Text annotation tool for human. Software available from http://gttub..oo/////co/////////
Vishakh Padmakumar and He He. 2021. Unsupervised extractive summarization using pointwise mutual in- formation. In Proceedings of the 16th Conference of the European Chapter of the Association for Com- putational Linguistics: Main Volume, pages 2505- 2512.
Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar, Russell Reas, Jiangjiang Yang. Doug Burdick, Darrin Eide, Kathryn Funk, Yannis Katsis, Rodney Michael Kinney, Yunyao Li, Ziyang Liu, William Merrill, Paul Mooney, Dewey A. Murdick, Devvret Rishi, Jerry Sheehan, Zhihong Shen. Brandon Stilson. Alex D. Wade, Kuansan Wang, Nancy Xin Ru Wang, Christopher Wilhelm, Boya Xie, Douglas M. Ray- mond, Daniel Weld, Oren Etzioni, and Sebastian Kohlmeier. 2020. CORD-19: The COVID-19 -19 open
Lev Pevzner and Marti A Hearst. 2002. A critique and improvement of evaluation metric for text segmentation. Computational Linguistics, 28(1):19- 36.

---

research dataset. Proceedings of the 1st Work- shop on NLP for COVID-19 at ACL 2020, Online. Association for Computational Linguistics.

---

A Exhaustive Search
We tested GreedyCAS on five sell-structured abstracts (in total 7,776 configurations) from the CAS-auto dataset, where we use different a-orders to normalize the MI. To do this, we brute-forcely calculated NMI values for all configurations. Figure shows the progressive change of the running maximum NMI in the exhaustive search, where the global maximum was reached at around 5, 5,100.th configuration. For all three cases, GreedyCAS managed reach the same global maximum. However, due the exponential increase of exhaustive search cost number of abstracts, testing GreedyCAS over larger amount of abstracts was difficult.
running max of NMI for different orders of arithmetic mean (k = 5)
2
0.18
0.18
0.18
normalized MI
normalized MI
normalized MI
0.16
0.16
0.16
0.14
0.14
0.14
2000
4000
2000 4000
2000 4000
number of configuration
number of configuration
number of configuration
Figure 6: Progressive development of NMI on enumerating 7,776 configurations constructed from 5 abstracts.
We further tested GreedyCAS on five abstracts with different numbers of trials. Each trial involved different random segmentation initializations. Figure shows the distributions of the number of itera- tions under different numbers of trials that GreedyCAS required to achieve maximal NMI. We see that GreedyCAS is able to find the maximal NMI within 60 iterations during all trials.
number of iterations at max. NMI (k 5)
500 random trials
1,000 random trials
2,000 random trials
number trials
number of trials
number of trials
75
150
300
50
100
200
25
50
100
0
20
40
60
20
40
60
20
40
60
number of iterations
number of iterations
number of iterations
Figure 7: Distribution of number of iterations GreedyCAS performed on five abstracts when reaching the maximal NMI, with different number random trials. Every trial begins with random initial segmentations five abstracts.

---

B Dataset Example
In the following tables, sentences in the premise segment are highlighted in blue, whereas sentences in the conclusion segment are highlighted in red.
Title: Additional evidence on the efficacy of different Akirin vaccines assessed on Anopheles arabiensis vention for the current vector control activities. Previous results have displayed the elficacy of using Subolesin/Akirin ortholog vaccines for the control of multiple ectoparasite infestations. This made Akirin potential antigen for vaccine development against An. arabiensis,
(Diptera: Culicidae) (Letinić al., 2021)
BACKGROUND Anopheles arabiensis is an opportunistic malaria vector that rests and feeds outdoors, circumventing current indoor vector control methods. Furthermore, this vector will readily feed on both animals and humans. Targeting this vector while feoding on animals can provide an additional inter
METHODS The efficacy of three antigens, namely recombinant Akirin from An. arabiensis, recombinant Akirin from Aedes albopictus, and recombinant Q38 (Akirin/Subolesin chimera) were evaluated as novel interventions for An. arabiensis vector control. Immunisation trials were conducted based on the concept that mosquitoes feeding on vaccinated balb/c mice would ingest antibodies specific to the target antigen. The antibodics would interact with the target antigen in the arthropod vector, subsequently disrupting its Function.
RESULTS All three antigens successfully reduced An. arabiensis and reproductive capacities, with vaccine efficacy of 68-73%.
CONCLUSIONS These results were the first to show that hosts vaceinated with recombinant Akirin vac- cines could develop a protective response against this outdoor malaria transmission vector, thus providing step towards the development of novel intervention for An. arabiensis vector control.
Table 5: Example abstract in CAS-auto segmented by GreedyCAS-NN, where the first sentence of the BACK- GROUND category is attributed to the conclusion segment. Best view in color printing.
Title: Interest in COVID-19 vaccinc trials participation among young adults in China: Willingness, reasons for hesitancy, and demographic and psychosocial determinants (Sun el al., 2021)
BACKGROUND With the demand for rapid COVID-19 vaccine development and evaluation, this paper aimed describe the prevalence and correlates of willingness to participate COVID-19 vaccine trials among university students in China.
METHODS cross-sectional survey with 1,912 Chinese university students was conducted during March and April 2020. Bivariate and multivariate analyses were performed to identify variables associated with willingness participate.
RESULTS The majority of participants (64.01%) () indicated willingness participate in COVID-19
vaccine trials. Hesitancy over signing informed consent documents, concerns over time necessary for participating in medical study, and perceived COVID-19 societal stigma were identified as deterrents, whereas lower socioeconomic status, female gender, perception of likely COVID- 19 infection during the pandemic, and COVID- 19 prosocial bchaviors were facilitative factors. Further, public health mistrust and hesitancy over signing informed consent documents had significant interactive effect on vaccine trial willingness.
CONCLUSIONS High standards of ethical and scientific practice are needed in COVID-19 vaccine re- search, including providing potential participants full and accurate information and ensuring participation free of coercion, socioeconomic inequality, and stigma. Attending to the needs of marginalized groups and addressing psychosocial factors including stigma and public health mistrust may also be important to COVID-19 19 vaccine development and future uptake.
Table 6: Example abstract in CAS-aurt segmented by GreedyCAS-NN, where the last sentence RESULTS category is attributed the conclusion segment. Best view in color printing.
Table shows an example abstract that contains a word pair, whose contribution to the overall NMI

---

score is the greatest.
Title: Hepatitis B surface antigen assembles in post-ER, pre-Golgi compartment (Huovila et al., 1992)
Expression of hepatitis B surface antigen (HBsAg)) the major envelope protein of the virus, in the absence
of other viral proteins leads its secretion oligomers in the form of disk-like or tubular lipoprotein particles. The observation that these lipoprotein particles are heavily disulphide crosslinked is paradoxical since HBsAg assembly is classically believed to occur in the ER, and hence in the presence of high levels of protein disulphide isomerase (PDI) which should resolve these higher intermolecular crosslinks. Indeed. incubation of mature, highly disulphide crosslinked HBsAg with recombinant PDI causes the disassembly of HBsAg dimers. We have used antibodics against resident ER proteins in double immunofluorescenee studies to study the stages of the conversion of the HBsAg from individual protein subunits to the secreted. crosslinked, oligomer. We show that HBsAg is rapidly sorted to a post-ER, pre-Golgi compartment which exclluees PDI and other major soluble resident ER proteins although it overlaps with the distribution of rab2, an established marker of an intermediate compartment. Kinetic studies showed that disulphide-linker HBsAg dimers began to form during a short (2 min) pulsc, increased in concentration to peak at 60 min and then decreased as the dimers were crosslinked to form higher oligomers. These higher oligomers are the latest identifiable intracellular form of HBsAg before its secretion (t 1/2 = 2 h). Brefeldin treatment does not alter the localization of HBsAg in this PDI excluding compartment, however, it blocks the for- mation of new oligomers causing the accumulation of dimeric HBsAg. Hence this oligomerization must occur a pre- vre-G compartment. These data support a model in which rapid dimer formation, catalyzed by PDI, occurs in the ER, and followed by transport of dimers to a pre-Golgi compartment where the absence of PDI and a different lumenal environment allow the assembly process to be completed.
(1pp w)) pair that contributes the most to the overall NMI(I,; C) score: (HBsAg, compartment)
Table 7: Example abstract in CAS- human segmented by GreedyCAS-NN, The word pair that contributes the most
to (M; C) bold. Best view color printing.

---

C Impact NMI When Moving One Word
A Let Wp E P be any premise word, Wy E C be any conclusion word, A, fixed abstract, and Øj one possible segmentation (we use the index j to represent the segmentation). By moving one arbitrary word from the premise segment PA to the conclusion segment C T(IP; C) changes. We investigate the major terms in the equation of mutual information. We aim find those word pairs that predominantly contribute to / (P; C) that the computation can be simplified, which essentially will allow the algorithm run on larger batch size.
First, we examine what happens to the marginal probabilities p(wp) and p(we):
p(u,)) = c c(Up, P) Tiu = w)
Eup, cuyy P)
p(we) cWe,, C) + Ilwe w] Euj clw C)
Here I denotes an indicator function and counter function. The indicator function takes the value if the condition in the bracket is fulfilled, otherwise, it takes the value O. For the marginal probabilities. we have the following cases:
If up w, then p(upp) decreases; if up # wo then p(up) increases.
If w w, then p(w.) inereases; if we o w, then p(w)) decreases.
Then, examine what happens to p(upi we), the main term in computation of [PP C)
p(upi we) = Ejss c(up pay pa C2) + (c(mp, pdo Twup = w)) (c(we, Cdi) + Tw = w])
Eimgane) (c(w/, P) - Ilu = w|) (c(w's C) + 1lw2 = w))
constant
Σ Cup pdg Jo(we. (AA)) + (c(wpp pai) Ilw = wi) (c(we, (rAi) + I[we = wl)
cl P)a(w(, C)+
Σ
{elwt P)J[w( = w] - -c(w C)](w( = w| - -[uu = w|[lu[ = wi]
comstant
+ a3 b
here a and the constant terms within the fraction, since moving u in Ag will not affect other abstracts. For the joint probability, we have the following cases:
If Up * w and We f w, pl Wip we) remains unchanged.
If wp = w and we * w, then
(W)p:W) + (a 1)3 c(we, C)
If up # w and w. w, then
+ a(3 +1) b+ c(wp, IP)
p(
If wp w and w. w, then
+ (a - 1)(33 + bb c(wp. IP) c(we, C)
p(upi we)

---

Till this point, found it very difficult predict how p(upi we) will change when moving w, The reasons are:
For the case of wp = w and wi = w, we cannot tell a priori whether c(w, P) - c(w, C) is positive, i.c. whether w appears more frequently within premise segments or conclusion segments. This leads to the uncertainty of determining the change in the sign of mutual information.
The normalizing factor of NMI, which is essentially the minimum between H (P) and H (C), cannot be determined after moving w.
Also, because for any research domain, it is nearly impossible to get large number of papers that study exactly the same research question (e.g., it's not possible to get thousands of papers that study the effectiveness of COVID-19 vaccines, due to limited number of clinical trials that have been done far), therefore, further increasing the batch size is not feasible.
Due to the above reasons, we only computationally studied how NMI would change when moving one word from the premise segment to the conclusion segment (see Figure 5).


Reviews:
[{"review_id": "qw0FGsA6lh", "reviewer": "Reviewer_m93t", "summary": "", "questions": "1. Is this paper the first to solve the abstract segmentation in an unsupervised manner?\n2. Why stitching the start and end together?\n3. How to calculate NMI in GreedyCAS-NN?\n4. Does the abstract segmentation has publicly available datasets? Why constructing the dataset in your own?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer m93t,\n\nWe sincerely thank you for the valuable comments. We kindly ask you to review our answers to your queries provided below:\n\n**Q: Datasets used are relatively small and do not represent a broad range of possible scenarios.**\n\nA: By definition, NMI requires word probabilities to be well-estimated. To make sure that the word probabilities are reliable, we use abstracts from papers in the CORD19 dataset, since they all focus on covid related research topics, and close-related papers have higher cooccurrences of premise and conclusion words. Nevertheless, to further narrow down the research topics, we fetch structured abstracts whose paper titles contain the keyword *vaccine*, and non-structured abstracts, the keyword *antigen*. This results in having nearly 700 structured abstracts and 200 non-structured abstracts, which is the best we can do with the CORD19 dataset.\n\n**Q: The method is heuristic.**\n\nA: We aim to investigate the suitability of NMI as an optimization objective for segmenting abstracts into premises and conclusions. The assumptions about conclusion sentence location and number rely on heuristics, which are necessary to prevent excessive computational costs in GreedyCAS. Additionally, Figure 3 illustrates that these heuristics closely correspond to human annotations on the location of conclusion sentences, indicating their validity.\n\n**Q: For Structured Abstracts, GreedyCAS seems to have no advantage and the paper does not conduct depth discussions.**\n\nA: Thank you for the comment. When writing structured abstracts, we hypothesize that authors take text directly from the corresponding sections in the full text instead of writing novel text. This may result in the sentences of structured abstracts being less coherent with one another (in the sense of having fewer words in common), which in turn may affect the performance of GreedyCAS. We will edit our manuscript to include this discussion. \n\n**Q: Is this paper the first to solve the abstract segmentation in an unsupervised manner?**\n\nA: To the best of our knowledge, it is. However, we are delighted to know and cite any related works, if you have any suggestions. \n\n**Q: Why stitch the start and end together?**\n\nA: Conclusion sentences are typically located at the end of the abstracts, so segmenting original abstracts into premises and conclusions is often a matter of finding just one conclusion boundary sentence. By stitching the start and end of each abstract together, we essentially evaluate GreedyCAS on the harder task of having to identify both conclusion boundary sentences. \n \n**Q: How to calculate NMI in GreedyCAS-NN**\n\nA: The computation of NMI within GreedyCAS-NN mirrors that of GreedyCAS-base. The distinction, however, lies in the approach. In GreedyCAS-NN, during batch creation, a pre-trained sentence encoder SBERT is employed to retrieve abstracts that carry the greatest semantic relevance to the seed abstract. The rationale here is rooted in the belief that semantically similar abstracts possess a higher degree of shared vocabulary. This, in turn, plays a pivotal role in enhancing the precision of word probability estimation during the NMI calculation process.\n\n**Q: Does the abstract segmentation has publicly available datasets? Why constructing the dataset in your own?**\n\nA: To the best of our knowledge, no existing publicly accessible datasets align precisely with our objectives. To ensure accurate word probability estimation, it's essential to source abstracts that pertain closely to our research themes. Consequently, we made the determination to compile these abstracts on our own.\n\nWe hope that we have addressed your concerns and we are happy to discuss further.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed GreedyCAS approach optimizes Normalized Mutual Information (NMI) for unsupervised scientific abstract segmentation, offering a new solution to an important NLP task.\n2. Experimental results on two datasets demonstrate the effectiveness of the approach, with promising results on non-structured abstracts and strong correlation between NMI and other evaluation metrics.\n3. The discussion of limitations and suggested future work demonstrates the author's commitment to improvement, contributing to the overall quality of the paper.", "reasons_to_reject": "1. The datasets used for testing are relatively small and do not represent a broad range of possible scenarios.\n2. The method is heuristic.\n3. For  Structured Abstracts, GreedyCAS seems to have no advantage and the paper does not conduct depth discussions.", "questions_to_authors": "1. Is this paper the first to solve the abstract segmentation in an unsupervised manner?\n2. Why stitching the start and end together?\n3. How to calculate NMI in GreedyCAS-NN?\n4. Does the abstract segmentation has publicly available datasets? Why constructing the dataset in your own?", "paper_topic_and_main_contributions": "This paper proposes an unsupervised approach for scientific abstract segmentation, named Greedy Cyclic Abstract Segmentation (GreedyCAS), which optimizes Normalized Mutual Information (NMI). The approach uses an exhaustive greedy approach to iterate over all abstracts and determine the best segmentation for each. To test how NMI deals with a known text boundary, the end of an abstract, the start and end of each abstract are stitched together to form a cycle, then two segmentation boundaries are selected with constraints based on prior knowledge. Two datasets are created to test the proposed approach, one comprises non-structured abstracts with human-annotated conclusion sentences, and the other contains structured abstracts in which conclusion sentences have been explicitly marked by the authors of the abstract. The results show that GreedyCAS achieves promising segmentation results on the dataset of non-structured abstracts, and there is a strong correlation between NMI and other evaluation metrics.\nMain contributions:\n1. Propose GreedyCAS, an unsupervised approach for scientific abstract segmentation that optimizes NMI.\n2. Show that GreedyCAS achieves promising segmentation results on a dataset of non-structured abstracts.\n3. Find a strong correlation between NMI and other evaluation metrics, in support of the effectiveness of the proposed approach.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kSMQD0DJ8G", "reviewer": "Reviewer_RqZY", "summary": "", "questions": "a. The current method section needs a bit more detailed description for reproducibility. For example, in section 3.1, the authors argued that 6 possible segmentations per abstract exist in current methods, but it reads that there could be more combinations of boundary and non-boundary sentences in an abstract. Could the authors elaborate more on it?\n\nb. Since the conclusion sentences normally occur towards the end of the abstracts - I wonder if separating the abstracts into two parts (first half and second half) first, and applying the current methods in the section half of the abstracts only would make it more efficient?\n\nc. In section 4, could the authors provide more details of the inter-coder agreement checking results between the annotators when labeling the sections?\n\nd. Could the authors elaborate more on the motivation of why segmenting the abstracts into only two subsections instead of a more fine-grained segmentation?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer RqZY,\n\nWe sincerely thank you for the valuable comments. We kindly ask you to review our answers to your queries provided below:\n\n**Q: Identifying the conclusion from the abstracts might not provide enough information that is needed.**\n\nA: In this study, our focus does not revolve around categorizing premises into distinct discourse segments (Background, Methods, Results, and Conclusion). Rather, our objective is to investigate whether NMI, relying solely on word statistics, possesses the capability to differentiate between conclusions and premises. Early efforts in discourse sentence classification often introduce external language information not inherent to the abstracts. In contrast, our approach relies solely on the estimation of word probabilities derived from the abstract texts.\n\n**Q: Due to the large number of scientific papers, the efficiency of the method might be an issue in real practice.**\n\nA: Thank you for the comment. The high computational cost of GreedyCAS is largely due to the need for reliable word probabilities, which in turn is only possible when many abstracts are processed simultaneously. In a separate ongoing work, we explore whether using pretrained language models, which have already processed large text corpora, have reliable word probability estimations without needing to process as many abstracts.\n\n**Q: There could be more combinations of boundary and non-boundary sentences in an abstract?**\n\nA: To reduce computational cost, we assumed that 1) abstracts contain at most three conclusion sentences and 2) conclusion sentences are not located within the middle of abstracts. These assumptions lead us to six possible combinations of boundary and non-boundary sentences per abstract. However, without these assumptions, the total number of possible combinations per abstract grows exponentially because the two boundary sentences may be anywhere within the abstract. \n\n**Q: Separating the abstracts into two parts (first half and second half) first, and applying the current methods in the section half of the abstracts only would make it more efficient?**\n\nA: Thank you for this comment. Indeed, adopting your suggestion could enhance efficiency by extracting conclusions solely from the second half of the abstracts. We applied GreedyCAS to the full abstract not for efficiency, but to analyze its ability to segment abstracts under a more difficult setting (where there are more possible conclusion boundary positions to choose from). \n\n**Q: Provide more details of the inter-coder agreement checking results between the annotators when labeling the sections. Why segmenting the abstracts into only two subsections instead of a more fine-grained segmentation.**\n\nA: Within the CAS-human dataset, we refrained from requesting annotators to assign a section (Background, Results, Methods) to each sentence of the abstract. This omission stemmed from the fact that section details were not considered into GreedyCAS's input, and section classification is not the goal of this work.\n\nWe hope that we have addressed your concerns and we are happy to discuss further.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Segmenting unstructured abstracts or scientific papers into subsections is an important task. Developing methods in this task could potentially be beneficial to the community in other related tasks, such as segmenting electronic health records, etc.\n\nThe study proposed a new approach that utilized mutual information for abstract segmentation.", "reasons_to_reject": "1. Current approach segments the abstracts into two parts - premise and conclusion. However, most early efforts have been tried to segment the abstracts into sections with more fine granularity. Comparatively, just identifying the conclusion from the abstracts might not provide enough information that is needed. \n\n2. As discussed by the authors - the proposed methods has a high time and space complexity. Due to the large number of scientific papers, the efficiency of the method might be an issue in real practice.", "questions_to_authors": "a. The current method section needs a bit more detailed description for reproducibility. For example, in section 3.1, the authors argued that 6 possible segmentations per abstract exist in current methods, but it reads that there could be more combinations of boundary and non-boundary sentences in an abstract. Could the authors elaborate more on it?\n\nb. Since the conclusion sentences normally occur towards the end of the abstracts - I wonder if separating the abstracts into two parts (first half and second half) first, and applying the current methods in the section half of the abstracts only would make it more efficient?\n\nc. In section 4, could the authors provide more details of the inter-coder agreement checking results between the annotators when labeling the sections?\n\nd. Could the authors elaborate more on the motivation of why segmenting the abstracts into only two subsections instead of a more fine-grained segmentation?", "paper_topic_and_main_contributions": "This work proposed an unsupervised approach that utilizes mutual information to segment unstructured abstracts into premise (sections other than the conclusion subsection) and conclusion. The study treats every abstract as a recurrent cycle of sentences. To optimally determine segmentation boundaries, it greedily optimizes the NMI score between the two segments, and decides the boundary for the segmentation. The method and evaluation were done on CORD-19 datasets.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GUiLl57QeE", "reviewer": "Reviewer_Wkik", "summary": "", "questions": "A: In section 6.3, you mention \"the semantic metric ROUGE\". However, ROUGE is not explicitly semantic. Have you explored other measures that directly capture semantic similarity?\n\nB: Your proposed approach seems promising for abstract segmentation in scientific papers. Have you conducted any experiments or explored the adaptability of your method to other domains beyond scientific literature?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer Wkik,\n\nWe sincerely thank you for the valuable comments. Please kindly see our response to your questions as follows:\n\n**Q:  ROUGE is not explicitly semantic, have you explored other measures that directly capture semantic similarity?**\n\nA: Thank you for the precise comment. It's accurate that ROUGE doesn't directly gauge the semantic relevance between the original and segmented conclusions. We will revise the text for improved accuracy. \n\nWe opted against using semantic-based evaluation metrics because they often depend on the text encoder chosen. For instance, in our experience, Sent2vec (Matteo Pagliardini, Prakhar Gupta, Martin Jaggi, Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features NAACL 2018) embeddings for semantically different texts tend to nevertheless have high cosine similarity scores (>0.6). Cosine similarity is thus not discriminative enough for evaluating segmentation boundaries. \n\n**Q: Have you conducted any experiments or explored the adaptability of your method to other domains beyond scientific literature?**\n\nA: Thank you for your interest. Our focus in this paper was on scientific text, but GreedyCAS in principle is applicable to domains beyond scientific literature and we will explore its performance in future work.\n\nWe hope that we have addressed your concerns and we are happy to discuss further.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper introduces a novel approach. This provides a fresh perspective on the problem and offers a potential solution that is different from existing methods. \n2. The proposed approach is well-grounded in theory by utilizing NMI as a measure of semantic similarity between sentences. This theoretical foundation adds credibility to the method and makes it easier for researchers in the NLP community to understand and build upon the proposed technique.\n3. The authors have tested their approach using two variants, one based on nearest neighbor (NN) and the other standard (base). Additionally, the method has been extensively evaluated against baselines. ", "reasons_to_reject": "No major issues in the paper.", "questions_to_authors": "A: In section 6.3, you mention \"the semantic metric ROUGE\". However, ROUGE is not explicitly semantic. Have you explored other measures that directly capture semantic similarity?\n\nB: Your proposed approach seems promising for abstract segmentation in scientific papers. Have you conducted any experiments or explored the adaptability of your method to other domains beyond scientific literature?", "paper_topic_and_main_contributions": "This paper addresses the challenging task of automatic segmentation of conclusions from scientific abstracts, specifically focusing on non-structured abstracts. In non-structured abstracts, the concluding information is not explicitly marked.\n\nThe main contribution of this paper is the introduction of an unsupervised approach called \"GreedyCAS\" for abstract segmentation. The approach is based on Normalized Mutual Information (NMI). The paper makes some acceptable assumptions: (1) consider each abstract as a recurrent cycle of sentences and place two segmentation boundaries and (2) it assumes that conclusions are strongly semantically linked with preceding premises.\n\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an unsupervised approach for scientific abstract segmentation, and all the reviewers agree its soundness and give comprehensive reviews. The authors also response to the questions in detail. In addition, reviewers also have some concerns, e.g., efficiency. I strongly suggest that the authors make precise revisions based on the comments of the reviewers.
Decision: Accept-Main