M$^3$Seg: A Maximum-Minimum Mutual Information Paradigm for Unsupervised Topic Segmentation in ASR Transcripts
Authors: Ke Wang
Xiutian Zhao
Yanghui Li
Wei Peng
Venue: EMNLP 2023 Main (2023)
Keywords: Unsupervised topic segmentation
mutual information maximization/minimization
automatic-speech-recognition (ASR) transcripts structuring

Abstract: Topic segmentation aims to detect topic boundaries and split automatic speech recognition transcriptions (e.g., meeting transcripts) into segments that are bounded by thematic meanings. 
In this work, we propose M$^3$Seg, a novel Maximum-Minimum Mutual information paradigm for linear topic segmentation without using any parallel data.
Specifically, by employing sentence representations provided by pre-trained language models, M$^3$Seg first learns a region-based segment encoder based on the maximization of mutual information between the global segment representation and the local contextual sentence representation.
Secondly, an edge-based boundary detection module aims to segment the whole by topics based on minimizing the mutual information between different segments.
Experiment results on two public datasets demonstrate the effectiveness of M$^3$Seg, which outperform the state-of-the-art methods by a significant (18\%–37\% improvement) margin.

M'Se A Maxinun-Winimum Mutual Information Paradigm for Unsupervised Topic Segmentation in ASR Transcripts
Ke Wang Xiutian Zhao Yanghui Li Wei Peng"
Huawei IT Innovation and Research Center
(wangke215, zhaoxiutian, liyanghui, peng wei
wangg.wei1ehhaweiii Jehuawei..w..... com
Abstract
Edge-Based Boundary Delection
Topic Segments
Ntotual Intt rinaattoo Minimiztiion
Topic segmentation aims to detect topic bound- aries and split automatic speech recognition transcriptions (e.g., meeting transcripts) into segments that are bounded by thematic mean- ings. In this work, we propose Seg novel Maximum-Minimum Mutual informa- tion paradigm for linear topic segmentation without using any parallel data. Specifically, by employing sentence representations provided by pre-trained language models, M M'Seg first learns region-based segment encoder based the maximization of mutual information be- tween the global segment representation and the local contextual sentence representation. Secondly, an edge-based boundary detection module aims to segment the whole by topics based on minimizing the mutual information between different segments. Experiment re- sults on two public datasets demonstrate effectiveness of Seg. which outperform the state-or -ee--offthhhateeeeee art methods by significant (18%-. 37% improvement) margin.
Sonteencc in ANR Branver Spe
PLM
PLM
PLM
PLM
Region-Based Segment Modeling
Matual Information
Masimization
Sentence Representation
E
Segment Representation
Figure 1: Illustration of S which first learns seg- ment representations by mutual information maximiza- tion, and then segments the entire ASR transcript by mutual information minimization. PLM refers to pre- trained language models, and dotted red lines indicate topic changes.
1 Introduction
Automatic speech recognition (ASR, also known computer speech recognition speech-to-text) (Rabiner and Juang, 1993; Graves and Jaitly, 2014) has brought us great convenience by transcribing conversations into text anytime and anywhere to aid human understanding. However, the generated un- structured transcriptions are sometimes too lengthy for users to grasp high-level meaning quickly. Meeting transcripts usually long and contain multiple heterogeneous topies in various structures, such as opening sessions, different discussion sub- jects, and closing sections. As one or more topics usually drive conversations or discussions, topic segmentation (Labadié and Prince, 2008) can im- prove the readability of transcription and facilitate
downstream long-text tasks such as meeting sum- marization, passage retrieval, and automated article generation (Mohri et al.; Feng et al.).
However, acquiring annotated training data support topic segmentation is expensive. Unsu- pervised methods have attracted attention due to their less reliance on annotated data. Most exist- ing unsupervised works can be categorized into edge-based and region- region-based methods. Particularly, edge-based methods aim to detect discontinuities (i.c., topic changes) in sequence where semantic features change rapidly, based word frequency (Hearst, 1997). embeddings of pre-trained language models (Solbiati et al., 2021), etc (Choi, 2000). Ragion-based methods aggregate neighbouring sentences through the homogeneity criterion, such
'quual Contribution. "Corresponding author.

---

latent dirichlet allocation (Riedl and Biemann, 2012) or perplexity calculated by PLM (Feng et al.). Nevertheless, these methods rely on capturing lo- cal sentence information rather than global topic information to perform segmentation, which may be noise sensitive and have a gap in generating ground-truth semantic boundaries.
Note that we cannot access gold-standard segments as human annotations do not exist.
able.
Model Overview As aforementioned, given a topic segmenta- tion seeks to splits it into several topic segments, where the sentences in each segment belong the same topic (requirement but different seg- ments represents relatively independent topics (re- quirement ii). Inspired by this idea, we propose a maximum-minimun mutual information paradigm for topic segmentation (titled M in an un- supervised manner, which learns segment repre- sentations by maximizing mutual information and partitions different segments by minimizing mutual information.
transcript = (8), $2.'" 8n).
In contrast to prior works, which fail cap- ture the global semantic-level topie information of different segments, we propose novel Maximum- Minimum Mutual information paradigm for unsu- pervised topic segmentation (M&Seg). Our work is inspired by mutual information, which measures how much one random variable tells us about an- other. Intuitively, sentences in the same segment should depend on the same topic (with high mutual information), and different topic segments should be independent of each other (with low mutual in- formation). Based on this insight, M'Seg divides an ASR script into different topic segments based on a two-stage process: region-bel based segment mod- eling and edge-based boundary detection. We first learn segment encoder by maximizing the mu- tual information between the global segment rep- resentation and each local contextualized sentence representation (provided by pre-trained language Models). Stage 2 detects topic boundaries by mini- mizing mutual information between different seg- ments.
Mutual information (MI) can measure the de- pendence between two random variables (Shannon, 1948). Given two random variables a and 6, the MI between them is b(a = E..b P(a, b) log P(a) The intuitive interpretation of [(a; b) is a measure- ment of the degree reduces the uncertainty in b vice versa. For example, the MI [(; b) is equal to when a and are independent. Therefore, from the view of MI, different topic segments should be as independent of each other as possible (i.e., MI minimization), but sentences within the same topic segment should be as dependent the same topic as possible (i.e., MI maximization). Based this insight, 2 consists of two stages (as depicted in Figure 1):
P(a(b)
Experimental results on two widely-used bench- mark datasets show that M consistently sur- passes five existing methods by wide margin. We conduct ablation studies to demonstrate the effec- tiveness of the proposed model and show the great utility of using mutual information in this task.
(1) Segment Modeling Based on MI Maximiza- tion Intuitively, knowing a sentence reduces the uncertainty of its corresponding topic. Thus, we train segment encoder E to learn the global seg- ment representation Yey = (r(g))) of the segment Süg by maximizing the MI between and each of its local sentence representations Tk, E [ (requirement i):
2 Method
Formally, let s denote a meeting transcript pro- duced by an automatic speech recognition (ASR) system, which consists of a list of utterances s {$1, 82, ... )n)- Topic segmentation can be seen as problem of topic change detection, and aims to cut the transcript into consecutive segments {81:111 Siig. ... } based on the underlying topic structure. For each segment 8i:j. represents a segment in meeting transcript, from the i-th sequence to the j-th sequence St; = (..,.... , For example, an hour-long meeting transcript can be broken down into different topic segments (c.g., opening sessions, different discussion sub- jects, and closing remarks) to make it more read-
JSM = max
Σ [J..
M(r)) (1)
Specifically, given n utterances 8
{s1, {21, 82. s8p) in an ASR transcript, we first use the pre-trained language model (PLM) to obtain the contextualized representation r of each sentence (Peters et al., 2018): {ri.-.- .T.... "r.....,nn.rr
PLM(s)), Rd where d is the representation dimension of PLM. In our case, 7; is computed by applying a mean-over-time pooling layer on the token

---

representations of the last layer of RoBERTa-base (Liu et al.. 2019) model. We then use the segment encoder to get the segment representation Vigj of the text segment Si: - Eo(rey); Uus E RV where 0 denotes the parameters of E. The segment encoder E' is trained by maximizing the MI between the global segment representation and each of its local sentence representations. However, mutual information estimation is generally intractable for continuous and high dimensional random variables, so we maximize the InfoNCE (Logeswaran and Lee, 2018) lower bound estimator of Eq 1. Following (Kong et al., 2020), maximizing InfoNCE is analogous to maxi- mizing the standard cross-entropy loss: Lem&
gences which may offer favorable trade-offs. Following the InfoNCE estimator in Poole et al. (2019), we define wensen-Shannon mutual infor- mation estimator of two factorized latent variables and b: I (a; b)& = Le, Elolloo((1 + exp(a a b)) Ea,b-[log( + exp(a o b- b)))], where b- is sampled from the complement set of b. In order to achieve single-pass computation (low time com- plexity), we calculate the difference in MI between the proposed boundary i and its adjacent offsets of +/- I as the MI bound.
MIC(y)--130)) = 91--2 9111 VI--1 9 9h3
Disentangiement with offeet-1
Viu Vi+1i Vin-1 Vay
(2)
Disentanglemmnn with foct++
This can be easily extended to more complex bounds in the future. As a consequence, we pro- pose a metric based on MI gap (MIG) to quan- titatively assess the effectiveness of disentangle- ment between two neighboring regions 814-1 and Si:² MIG(y) 1: l:gg)) == Vi o Di 15j + V/. y/ 1:j 22 y1:i-1 Vi.j-
exp(315991,))
-E,je(Ln),sss [lo where w Tij is sampled from sentence representations in the segment Sty, and Vey is sampled from the rest. We use the dot product between embeddings to measure the distance (i.e., o) in the vector space. Note that we only need to train the segment encoder E, while PLM's parameters are fixed.
Finally, we derive the topic boundaries as pairs of regions 91:1-1 and Yaj where MIG(J1.-1i Way) scores are greater than certain threshold 5. The MIG measures how much the segment representa- tions change when we move the boundary by one position. A positive MIG means that the segments become more dissimilar when we shift the bound- ary. indicating a potential topic change. A negative MIG means that the segments become more similar when we shift the boundary, indicating a coherent topic. Therefore, can the MIG criterion for detecting topic boundaries.
Our assumption is that in meeting transcript, continuous sequence of utterances contains thematic information, which can be either coarse- grained or fine-grained. For example, the entire meeting transcript's utterances may be based on a certain motivation (coarse-graine theme) for con- vening, while if divided into fine-grained segments, each utterance can be associated with a certain issue (fine-grained theme). It is based on this as- sumption that construct data from any contin- uous sequence of utterances in the same meeting transcript and use the maximization of mutual in- formation to train a segment encoder to learn the global segment representation (Eq 1) (Wang and Wan, 2020, 2021). This also contributes to the effectiveness of our method.
Experiments
To demonstrate the effectiveness of our model, we evaluate S on two widely-used benchmark datasets, AMI Meeting Corpus (Carletta al., 2005) and the ICSI Meeting Corpus (the dataset details are shown in Appendix A.). According to the paper Ghosh et al. (2022), more likely that the dataset we used consists of Unstructured Chats, transcriptions of spoken language from a spoken scene, which result in often incoherent and incomplete utterances with some word errors.
(2) Boundary Detection Based on MI Minimiza- tion After obtaining the segment representation of any region, we propose an edge-based bound- ary detection module to detect topic changes by minimizing the MI between the segment represen- tations of different topic segments (requirement I(yn55 Hyak). Specif- k.j.ke[1.n],..pk
We compare So with several unsupervised topic segmentation methods, includ- ing region-basee based methods (i.c.. TopicTiling (Riedl and Biemann, 2012) and DialoGPT (Feng et al.) and edge- methods (i.e., TextTiling (Hearst, 1997), C99 (Choi, 2000), and TextTiling-BERT
ii): JBD = min Σ
ically, as we are primarily interested in maximizing the MI gap, and not concerned with its precise value, we can rely on non-Kulllen Kullback-Leiin diver-

---

(Solbiati al., 2021)). For analysis, we also show the Random and Even methods, which refer to placing topic boundaries randomly and every n-th utterance, respectively. We perform two standard evaluation metrics, Pk (Beeferman et al., 1999) and WinDiff (Wd) (Pevzner and Hearst, 2002) scores (details are shown in Appendix B), and the model implementation details are shown in Appendix C. Regarding the detection of multiple boundaries, we use certain threshold to control is, as long as the mutual information gap between pairs of regions is greater than 8, it will be con- sidered as topic boundaries (detailed Section 2). For the random baseline, following the settings of TextTiling-BEEA BERT, we set each utterance to have 0.30 probability of being placed with topic bound- aries (the reason is that this value is roughly consis- tent with the probability of boundaries occupying the entire utterance).
Model
AMI
ICSI
Variants
Pk
Wd
Pk
Wd
Seg
0.248
0.286
0.203
0.224
w/o segment modeling
0,363
0.379
0.358
0.363
w/o boundary detection
0,309
0.338
0.285
0.306
Table 2: Ablation study of different components. "w/o" means "without".
PLM
AMI
ICSI
Variants
Pk
Wd
Pk
Wd
BERT-hase
0.325
0.334
0.308
0,326
That
BERT-large
0.237
0.303
0.198
0.245
XLNet
0.307
0.319
0.3248
0.340
RoBERTa
0.248
0.286
0.203
0.224
Table 3: Results of M with different PL.Ms.
based boundary detection module. For "w/o seg- ment modeling", we dircctly apply max-over-time pooling operation to sentences instead of the seg- ment encoder E. For "w/o boundary detection", we apply TextTiling (Hearst, 1997) algorithm, which calculates the similarity based the segment repre- sentations provided by E. From Table 2, two com- ponents play role, yet the most significant drop when the region-based segment modeling module is removed, demonstrating the great effectiveness of using MI to model segments.
AMI
ICSI
Method
Pk
Wd
Pk
Wd
Random
0.570
0.747
0.640
0.981
Even
0.518
0.559
0.642
0.924
TextTiling
0.412
0.426
0,426
0.600
C99
0.442
0.457
0.514
0,550
TopicTiling
0.356
0.360
0,351
0.380
DialoGPT
0.355
0.356
0.319
0.333
TexiTiling- BERT
0.352
0.352
0.324
0.343
Seg (Ours)
0,248
0.286
0,203
0.224
Influence of Pre-trained Language Models (PLMs) To investigate the importance of the PLM's contextualized representations, we perform ablations by using different PLMs Table From the results, we can see that the larger the pre-trained model leads to more significant improvement, indi- cating that the contextual information provided by PLM with more data and parameters is more con- ducive to topic segmentation. But Mo can bring consistent and significant improvements under dif- ferent pre-treiiii tre-trained language models, reflecting the effectiveness of our method.
Table 1: Results of topic segmentation. The lower the values of Pk and Wd, the higher the segment perfor- mance. We mark the best results in bold.
Main Results Table compares the results of basclines and our method on two meeting transcrip- tion datasets. From the results, we can see that: 1) TopicTTiling has a significant improvement over TextTiling, which shows that considering the global region- begion-based topic information can facilitate topic segmentation. 2) DialoGPT and TextTiling-BENN achieve lower eиrоr rates than word-frequency- based approaches (i.c., TextTiling and C99), re- flecting that the representations provided by PLMs provide better contextual information. 3) Never- theless, our model outperforms all baseline models on two datasets by wide margin (about + +18% ~ +37% reduction in error rate), demonstrating the effectiveness of using MI for topic segmentation.
Sentence Repre-
AMI
ICSI
sentation Variants
Pk
Wd
Pk
Wd
CLS--Emhedding
0.347
0.365
0.342
0.358
Max- Mlx-Pooolig
0.264
0.297
0.316
0.339
Mean-Pooling
0.248
0.286
0.203
0.224
Sentence-BERe
0.267
0.303
0.197
0.221
Table 4: Results Seg with different contextualized sentence representations.
Module Effectiveness Analysis To investigate the importance of the model's individual com- ponents, we perform ablations by removing the region-basee based segment modeling module and edge-
Influence of Sentence Representations (r) To
analyze the influence of different sentence rep-
resentations of a PLM, we conduct the ablation experiments with different contextualized repre-

---

sentation calculation methods, including: 1) CLS- Embedding: The representation of the first token [CLS) used as the contextualized representation the entire sentence. 2) Max-Pooling: We apply a max-over-time pooling layer on the token represen- tation of the last layer of a PLM. 3) Mean-Pooling: Similarly, we apply mean-over-time pooling layer on the token representations. 4) Sentence-BEiI BERT: We Beitencce-ERTT (Reimers and Gurevych, 2019) embedding as our contextualized sentence representation. From the results in Table 4, CLS- Embedding performs the worst, indicating the im- portance considering global information in topic segmentation. Overall, all methods achieve compet- itive performances, especially Mean-Pooling and Sentence-BERG demonstrating the effectiveness of the contextualized sentence representation pro- vided by PLMs.
demonstrate that the proposed model outperforms competitive systems by reducing error rates by 18% to 37%. These results emphasize the effectiveness of using mutual information for topic segmentation and suggest future opportunities to develop more complex and controllable systems.
approach has limitations that should be acknowledged. Firstly, it has only been tested on the English language, and further ex- perimentation is required to evaluate its perfor- mance on low-resource languages. Additionally, our method relies on pre-trained language mod- cls, which may not always be available or suitable for certain applications. Nevertheless, we believe that our maximummmminimmm mutual information paradigm has potential to advance the development of unsupervised topic segmentation systems.
However,
Acknowledgement
0.50
0.45
#Seeeess
17.5
We would like to express our gratitude to Rongjun Li, Suqing Yan, Xupeng Meng and Wei Huang from Huawei for their invaluable assistance and detailed discussions on the ideas during the im- plementation process. Their expertise and insights have been instrumental in the success of our project. We also wish to extend our uppreciation to the anonymous reviewers whose constructive feedback greatly improved our work. Their comments and suggestions were of immense help throughout the course of our research.
Pk Scone on AMI 320248
0.40
15.0
0.30
1000
0.25
Averaa
0.20
5.0
0.15
2.5
0.10
The MIG threshold 6
Figure 2: Results of different MIG threshold 6.
Influence of the MIG Threshold (6) We also in- vestigate the impact of different MI gap threshold 8 on the results. Figure 2 presents the Pk score and the number of topic segments after segmentation of model on the AMI dataset under different MIG threshold 8. With the increase of 8, the number of segments decreases. indicating that the larger the threshold of the MI gap leads the rougher seg- mentation of topics. Interestingly, the Pk score first decreases to a minimum and then increases, poten- tially due to the presence of an optimal decision point wrt the selection of MIG threshold 6.
References
Doug Beeferman, Adam L. Berger, and John D. Lafferty. 1999. Statistical models for text segmentation. Mach. Learn,, 34(1-3):177-210.
Jean Carletta, Simone Ashby, Sebastien Bourban, Mike Flynn. Maël Guillemot, Thomas Hain, Jaroslav Kadlec, Vasilis Karaiskos, Wessel Kraaij. Melissa Kronenthal, Guillaume Lathoud, Mike Lincoln, Agnes Lisowska, lain McCowan, Wilfried Post, Den- nis Reidsma, and Pierre Wellner. 2005. The AMI meeting corpus: A pre-announcement. In MLMI 2005, volume 3869 of Lecture Notes in Computer Science, pages 28-39. Springer.
Limitations
Freddy Y. Choi. 2000. Advances in domain indepen- dent linear text segmentation. In ANLP 2000, pages 26-33. ACI.
In this study, we introduce the Seg framework, which is novel approach unsupervised topic segmentation. We transform the task into an op- timization problem that maximizes intra-segment mutual information and minimizes setemst mutual information. Our experiments and analysis
Xiachong Feng, Xiaocheng Feng, Libo Qin, Bing Qin, and Ting Liu. Language model as annotator: Exploring dialogpt for dialogue summarization. ACI/UICNLP 2021, pages 1479-1491.

---

Reshmi Ghosh, Harjeet Singh Kajal, Sharanya Kamath, Dhuri Shrivastava, Samyadeep Basu, and Soundarara- jan Srinivasan. 2022. Topic segmentation in the wild: Towards segmentation of semi-structured & unstruc- tured chats. CoRR, 00s/22111119555
Nils Reimers and Iryna Gurevych. 2019, Sentence-bert, Sentence embeddings using siamese bert-networks. In FMNN.P-IJCNIPP 2019, pages 3980-3990.
Martin Riedl and Chris Biemann. 2012. Topictiling: A text segmentation algorithm based on lda. In ACL
Alex Graves and Navdeep Jaitly. 2014. Towards end- to-end speech recognition with recurrent neural net- works. In JCML 2014, volume 32 of JMLR Work- shop and Conference Proceedings, pages 1764-1772. JMLR.org.
2012 (Student Research Workshop), pages 37-42, Jeju, Republic of Korea.
Claude Shannon. 1948. A mathematical theory of communication, Bell Syst. Tech. J., 27(3)337--233.
Alessandro Solbiati, Kevin Heffernan, Georgios Damaskinos, Shivani Poddar, Shubham Modi, and Jacques Cali. 2021. Unsupervised topic segmenta- tion of meetings with BERT embeddings. CoRR, abs/2106. 12978.
Marti A. Hearst. 1997. Texttiling: Segmenting text into mulli-paragraph subtopic passages. Comput. Linguis- tics, 23())333-64
Adam Janin, Don Baron, Jane Edwards, Dan Ellis, David Gelbart, Nelson Morgan, Barbara Peskin, Thilo Pfau, Elizabeth Shriberg, Andreas Stolcke, and Chuck Wooters. 2003. The ICSI meeting corpus. In ICASSP 2003, pages 364-367. IEEE.
Ashish Vaswani. Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez. Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurlPS 2017, pages 5998-6008.
Lingpeng Kong, Cyprien de Masson d' Autume, Lei Yu. Wang Ling, Zihang Dai, and Dani Yogatama. 2020. A mutual information maximization perspective of language representation learning. In ICLR 2020.
Ke Wang and Xiaojun Wan. 2020. Adversarial text generation via sequence contrast discrimination. Findings of the Association for Computational Lin- guisties: EMNLP 2020, Online Event, 16-20 Novem- ber 2020, volume EMNLP 2020 of Findings of ACL, pages 47-53. Association for Computational Linguis- tics.
Alexandre Labadié and Violaine Prince. 2008. Finding text boundaries and finding topic boundaries: Two different tasks? In GoTAL 2008, volume 5221 of Lecture Notes in Computer Science, pages 260-271. Springer.
Ke Wang and Xiaojun Wan. 2021. Transsum: Trans- lating aspect and sentiment embeddings for self- supervised opinion summarization. In Findings of the Association for Computational Linguistics: ACL/JJCNLP 2021, Online Event, August 1-6, 2021, volume ACL/IJCLP 2021 of Findings of ACL, pages 729-742, Association for Computational Lin- guistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11192..
Lajanugen Logeswaran and Honglak Lee. 2018. An efficient framework for learning sentence representa- tions. In ICLR 2018. OpenReview.net.
Appendix
A Datasets
Mehryar Mohri, Pedro J. Moreno, and Eugene Wein- stein. Discriminative topic segmentation of text and speech. In AISTATS 2010, volume 9, pages 533-540.
The AMI Meeting Corpus (Carletta et al., 2005) and the ICSI Meeting Corpus (Janin et al., 2003) include 139 and 75 transcribed meetings with topic segmentation annotations, respectively. The word error rates of ASR transcription for AMI and ICSI are 36% and 37%, respectively. We use the human- transcribed reference (gold) transcripts as same as in (Solbiati et al., 2021), which are corrected by humans based on the ASR transcript (which seem to be missing in AMI and ICSI), but still contain many errors, such as grammatical errors like "me and him have done this". We use all the data as the test set, and consider only the top-level meeting changes (i.e., linear topic segmentation). Noted that we used each script train the segment encoder E, but we did not use the annotations, which conforms to the unsupervised setting.
Matthew E. Peters, Mark Neumann, Mohit lyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word repre- sentations. In NAACL-HLT 2018, pages 2227-2231
Lev Pevzner and Marti A. Hearst. 2002. critique and improvement of an evaluation metric for text segmentation. Comput. Linguistics, 28(1):19-36
Ben Poole, Sherjil Ozair, Aäron van den Oord, Alexan- der A. Alemi, and George Tucker. 2019. On varia- tional bounds of mutual information. In ICML 2019, volume 97 Proceedings of Machine Learning Re- search, pages 5171-5188 PMLR.
Lawrence R. Rabiner and Büing- Juang. 1993. Fundamentals of speech recognition. Prentice Hall signal processing series. Prentice Hall.

---

B Metrics
Following previous works (Solbiati et al., 2021), we perform two standard evaluation metrics, Pk (Beeferman et al., 1999) and WinDiff (Wd) (Pevzner and Hearst, 2002) scores, to show the performance of segmented results. Both metrics fixed sliding window over the document, and calculate the segmentation error by comparing the number of boundaries in the ground truth with the number of boundaries predicted by the topic seg- mentation model.
C Implementation
Our segment encoder E is one-layer Transformer (Vaswani et al., 2017) with dimension of 768. The threshold of the mutual information gup score is set to 0.01. The pre-trained language model uses RoBERTa-base (Liu et al., 2019) and can be casily migrated to other PLM models. We implement our model based on PyTTrrc and use two Tesla V100 graphic cards for learning. In order the original features obtained from the PLM without additional scaling,
Dim(E) to the same dimension as the output layer of PLM.
We train separately for each meeting input, using only the input text list and random segment boundaries to maximize mutual information as the training goal. No gold segment boundaries are required, which is consistent with the test scenario, that is, it is not biased by other meeting data.


Reviews:
[{"review_id": "zINfcdo3mE", "reviewer": "Reviewer_mmRA", "summary": "", "questions": "Would M3Seg work with transcriptions where utterances are incoherent and incomplete (no fully formed sentences)?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive and constructive feedback! We are glad that you found our apporach new and effective. Below are our responses to specific comments.\n\n***\n\n> Comparing results from Table 2 and 3, the Pk and the WinDiff values are marginally but not notably different for M3Seg.\n\n|                        |     AMI-Pk    |     AMI-Wd    |    ICSI-Pk    |    ICSI-Wd    |\n|------------------------|:-------------:|:-------------:|:-------------:|:-------------:|\n| M3Seg                  |     0.248     |     0.286     |     0.202     |     0.224     |\n| w/o segment modeling   |  0.363(46.4%↓) | 0.379(32.5%↓) | 0.358(77.2%↓) | 0.363(62.1%↓) |\n| w/o boundary detection | 0.309(25.0%↓) | 0.338(18.2%↓) | 0.285(41.1%↓) | 0.306(36.6%↓) |\n \nTable 2 above demonstrates an ablation study removing different components, and the small difference in the results indicates that both parts are quite effective.\n\n\n|            |    AMI-Pk    |    AMI-Wd    |    ICSI-Pk   |    ICSI-Wd   |\n|------------|:------------:|:------------:|:------------:|:------------:|\n| BERT-base  |     0.325    |     0.334    |     0.308    |     0.326    |\n| BERT-large | 0.237(27.1%↓) |  0.303(9.3%↓) | 0.198(35.7%↓) | 0.245(24.8%↓) |\n| XLNet      |  0.307(5.5%↓) |  0.319(4.5%↓) |  0.325(5.5%↓) |  0.340(4.3%↓) |\n| RoBERTa    | 0.248(27.7%↓) | 0.286(14.4%↓) | 0.203(34.1%↓) | 0.224(31.3%↓) |\n \nTable 3 above presents the outcomes of an experiment wherein we substituted various pre-trained language models as the backbone. The negligible variance in the results shows that our method is effective across different PLMs, thereby advocating that the increases of performances are not attributable to the PLMs.\n\n\n|                 |     AMI-Pk    |     AMI-Wd     |    ICSI-Pk    |    ICSI-Wd   |\n|-----------------|:-------------:|:--------------:|:-------------:|:------------:|\n| TextTiling-BERT      |     0.352     |      0.352     |     0.324     |     0.343    |\n| TextTiling | 0.412 (17.0%↓) | 0.426 (21.0%↓) | 0.426 (31.5%↓) | 0.600 (74.9%↓) |\n\nHowever, as the table above demonstrated, compared to our strongest baseline results (cf. [1]), we believe that the improvements brought about by each component of our method are significant.\n\n[1] Solbiati et al. 2021. Unsupervised Topic Segmentation of Meetings with BERT Embeddings.  \n***\n\n> Topic segmentation for unstructured conversations\n\nWe thank the reviewer for pointing out this issue, and we have incorporated this suggestion throughout our experiment. According to the paper [2], it is more likely that the dataset we used consists of *Unstructured Chats*, transcriptions of spoken language from a spoken scene, which result in often incoherent and incomplete utterances with some word errors. In addition, we have conducted experiments on the Topical Chat, BOLT, and Wiki-727K datasets as proposed in [2], and the results are as follows:\n| Method  | Topical Chat |    BOLT   | Wiki-727K |\n|----------------------------|:------------:|:---------:|:---------:|\n| Cross Segment BERT|     0.764    |   0.569   |   0.604   |\n| Cross Segment RoBERTa-Base|     0.767    |   0.561   |   0.599   |\n| Hierarchical Bi-LSTM|   **0.951**  |   0.443   |   0.570   |\n| M3Seg|     0.950    | **0.602** | **0.612** |\n  \nIt is worth noting that our method is unsupervised yet still achieves equivalent or better performances than these supervised methods that use domain data for fine-tuning. Additionally, we will cite the paper [2] and add detailed analysis of topic segmentation on structured, semi-structured and unstructured conversations.\n\n\n[2] Ghosh et al. 2022. Topic Segmentation in the Wild: Towards Segmentation of Semi-structured & Unstructured Chats.  \n***\n\n> Typos and Writing Suggestions\n\nWe appreciate your suggestions and have carefully checked for grammar errors, rephrased some expressions and applied a consistent format in our revised version.\n\n***\n\nThank you for your valuable feedback! Please let us know if you have any remaining concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper introduces a novel approach to unsupervised topic segmentation, called M3Seg, which is based on a maximum-minimum mutual information paradigm. The paper's results demonstrate that the proposed approach outperforms competitive systems by reducing error rates by 18% to 37%. These results emphasize the effectiveness of using mutual information for topic segmentation and suggest future opportunities to develop more complex and controllable systems. This paper would benefit the NLP/ASR community by providing a new and effective approach to unsupervised topic segmentation. ", "reasons_to_reject": "Comparing results from Table 2 and 3, the Pk and the WinDiff values are marginally but not notably different for M3Seg. Along with the identified limitation of expanding beyond English transcriptions, especially to low-resource languages, authors should also look into defining whether the meeting transcript datasets are structured conversations or unstructured conversations. https://arxiv.org/abs/2211.14954 by Ghosh et. al found that topic segmentation methods for unstructured conversations (conversations that are ill-formed, short sentences, and incoherent, which is a possibility in meeting transcripts but could be more evident in casual conversations) require advanced modeling techniques from structured conversations.", "questions_to_authors": "Would M3Seg work with transcriptions where utterances are incoherent and incomplete (no fully formed sentences)?", "paper_topic_and_main_contributions": "The authors in this paper present a novel approach to unsupervised topic segmentation called M^3Seg. The paper addresses the problem of detecting topic boundaries and splitting automatic speech recognition (ASR) transcripts into meaningful segments. The main contribution of the authors of this paper is the development of a maximum-minimum mutual information paradigm for unsupervised topic segmentation. This approach transforms the task into an optimization problem that maximizes intra-segment mutual information and minimizes inter-segment mutual information. Experimental results demonstrate that the proposed model outperforms competitive systems by reducing error rates by 18% to 37%.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Authors, please cite https://arxiv.org/abs/2211.14954 - the paper shows work on topic segmentation methods for unstructured and semi-structured conversations. "}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "There are some grammatical errors, such as subject-verb agreement (e.g., 'two components play a role' instead of 'two components play roles'). The paper could benefit from more consistent formatting, such as consistent use of capitalization in headings and subheadings. Some sentences could be rephrased for clarity, such as 'The threshold θ of the mutual information gap score is set to 0.01' could be rephrased as 'The mutual information gap score threshold, denoted by θ, is set to 0.01'."}}, {"review_id": "h7MyxMAE4v", "reviewer": "Reviewer_ZX6U", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments and feedback on our paper. We appreciate your recognition of our motivation to maximize the mutual information within the same region while minimizing the mutual information across regions, and we are encouraged by your finding it interesting. We would like to address some potential misunderstandings in your comments.\n\n***\n\n> In maximization formula 1), i,j are iterated from 1 to n, which implies that they are just random boundaries and not the actual region boundary, then maximizing mutual information with this random region might not be very helpful.\n\nWe would like to clarify our hypothesis. Our assumption is that in a meeting transcript, a continuous sequence of utterances contains a thematic information, which can be either coarse-grained or fine-grained. For example, the entire meeting transcript’s utterances may be based on a certain motivation (coarse-grained theme) for convening, while if divided into fine-grained segments, each utterance can be associated with a certain issue (fine-grained theme). It is based on this assumption that we construct data from any continuous sequence of utterances in the same meeting transcript and use the maximization of mutual information to train a segment encoder E to learn the global segment representation (Formula 1). This also contributes to the effectiveness of our method.\n\n***\n\n> In the minimization formula 2, it is simply a gap between representations, it is difficult to claim they are based on mutual information as there are many skips from requirement 2.\n\nWe would like to clarify that Formula 2 describes a simple method for determining topic segmentation boundaries based on maximizing the mutual information gap between adjacent segments, given that we have trained a good segment encoder E that can capture the thematic annotations in any continuous sequence of sentences (i.e., Formula 1). As an initial study, we did indeed propose a simple method to analyze the effectiveness of our entire mutual information learning framework. This can be easily extended to more complex bounds in the future.\n\n***\n\n> The proposed methods of both maximization and minimization are ambiguous and not very well aligned with the original mutual information motivation.\n\nWe were inspired by the idea that mutual information (MI) can measure the dependence between two random variables, and proposed a general framework for unsupervised topic segmentation using the inherent characteristics of meeting transcript data. However，we would like to clarify that as an initial study on unsupervised topic boundary segmentation using mutual information, we used relatively simple settings for each part of our M3Seg framework in order to analyze its effectiveness (e.g., constructing random continuous sentence regions to train the topic encoder and using the method of maximum adjacent mutual information difference to determine topic boundaries). As you rightly pointed out, it is indeed possible to explore more complex data construction and mutual information gap-based boundary detection methods. However, this precisely demonstrates the robustness of our method and leaves room for improvement in future work.\n\n***\n\n> Many details are not covered in the main section. The segment encoder is one of the most important ideas in this work, but its implementation is totally omitted and only mentioned in the appendix. It is also not clear how to detect multiple boundaries in the text for the proposed approach. Details about other baselines are also not enough, For example, for the random baseline, what's the probability to place the boundary?\n\nAs you mentioned, while we introduced the core training method of the segment encoder (Equation 1) in the paper, we placed the specific implementation details of the segment encoder in the appendix.\n\nRegarding the detection of multiple boundaries, we use a certain threshold δ to control it. That is, as long as the mutual information gap between pairs of regions is greater than δ, it will be considered as topic boundaries (detailed in Section 2). For the random baseline, following the settings of TextTiling-BERT, we set each utterance to have a 0.30 probability of being placed with topic boundaries (the reason is that this value is roughly consistent with the probability of boundaries occupying the entire utterance).\n\nWe have rewritten our paper and placed more core information in the main section in line with your comments. We will certainly leverage the one more page in the final version to provide more details, elease all code and data, including our methods and the datasets used for reproducing the results, should we been given the opportunity.\n\n***\n\nThank you again for your valuable feedback! Hopefully this clarification addresses your questions. Please let us know if you have any remaining concerns,", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The main motivation is interesting, which tries to maximize the mutual information within the same region while minimizing the mutual information across regions.", "reasons_to_reject": "the proposed methods of both maximization and minimization are ambiguous and not very well aligned with the original mutual information motivation. For example, In maximization formula 1),  i,j are iterated from 1 to n, which implies that they are just random boundaries and not the actual region boundary, then maximizing mutual information with this random region might not be very helpful. In the minimization formula 2, it is simply a gap between representations, it is difficult to claim they are based on mutual information as there are many skips from requirement 2.\n\nMany details are not covered in the main section. For example, the segment encoder is one of the most important ideas in this work, but its implementation is totally omitted and only mentioned in the appendix. It is also not clear how to detect multiple boundaries in the text for the proposed approach. Details about other baselines are also not enough, For example, for the random baseline, what's the probability to place the boundary?", "questions_to_authors": "", "paper_topic_and_main_contributions": "This work proposes an approach to tackle the topic segmentation problem, which splits a long spoken transcript into several regions with similar topics, the proposed approach consists of two steps taking advantage of mutual information, in the first step, it learns segment encoder which encourages embedding of sentences within the same region to be similar, in the 2nd step, it uses those embedding to segment transcripts based on the \"mutual information gap\". The proposed approach outperforms a few benchmarks significantly.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "n3mXUnERf9", "reviewer": "Reviewer_t5HK", "summary": "", "questions": "How do you define the i:j segments in paragraph 1) of section 2?", "limitations": "", "rating": 3, "confidence": 1, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable and constructive comments on our paper. We appreciate your recognition of the effectiveness of our approach and the detailed analysis of the reasons for the improvements, including the robustness to different PLMs. Below are our responses to your specific comments.\n\n***\n\n> Confidence intervals are missing, and the paper is not easy to read for someone who is not expert of the topic.\n\nWe thank the reviewer for pointing out this issue. Regarding the missing confidence intervals, we would like to clarify that our mehtod is inspired by the mutual information theory yet does not apply a predetermined confidence interval when sampling. However, we will incorpate this in our future work and suppliement the experiemnt by testing the method under different confidence boundary settings.\n\nWe have reorganized the structure of our paper and include more introductory contents to ensure a more smooth reading experience in our revised version.\n\n***\n\n> How do you define the i:j segments in paragraph 1) of section 2?\n\nWe would like to clarify that the notation S_i:j represents a segment in a meeting transcript, from the i-th sequence to the j-th sequence. We apologize for any confusion and will add more explanation about S_i:j to our paper. We have also carefully revise our paper, including checking the bibliography, to make it more understandable.\n\n***\n\nWe hope that these clarifications address your questions. Please let us know if you have any remaining concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The results seems to demonstrate the effectiveness of the approach, and the reasons of the improvements are analyzed in details, including the robustness to different PLMs.", "reasons_to_reject": "Confidence intervals are missing, and the paper is not easy to read for someone who is not expert of the topic.", "questions_to_authors": "How do you define the i:j segments in paragraph 1) of section 2?", "paper_topic_and_main_contributions": "The paper introduces a new algorithm to perform unsupervised segmentation of sentences into different topics. With experiments on two datasets and comparisons with previous methods, the paper shows that the proposed methodology provides significant improvements.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "line 51: etc might be explained instead.\nline 110: representing -> represents.\nSome references are missing the year, so bibliography should be more careful checked.\n"}, "extra_text": {}}, {"review_id": "sRmoD1IjPF", "reviewer": "Reviewer_bouu", "summary": "", "questions": "1. During training, did you train the segment encoder over all possible segments? It seems so because you don't use the annotation data to know the start and end sentences of the segments.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful comments and suggestions. We are glad that you found our framework well-motivated and our focus on local and global semantics clearly explained.\n\n***\n\n> During training, did you train the segment encoder over all possible segments? It seems so because you don't use the annotation data to know the start and end sentences of the segments. The segment encoder would need to encode all possible segments during training as well as inference. There are number of segments. The quadratic effect is not explained, and it's not clear how it affects time to train.\n\n\nYes, you are correct. In our work, we adopted a simple quadratic construction of all possible segments to train the segment encoder for validating the effectiveness of our M3Seg topic segmentation without the annotation data.\n\nWe randomly sampled different proportions of data from the all possible segments data for training the segment encoder during inference, and the results are shown in the table below. We also provide the average time cost per case for analysis:\n\n| Scale of the Dataset | AMI-Pk | AMI-Wd | ICSI-Pk | ICSI-Wd | Time Cost/Case |\n|:--------------------:|:------:|:------:|:-------:|:-------:|:--------------:|\n|          0%          |  0.332 |  0.329 |  0.319  |  0.331  |      11ms      |\n|          25%         |  0.277 (+16.7%) |  0.297(+9.7%) |  0.262(+17.9%)  |  0.289(+12.7%)  |      49ms      |\n|          50%         |  0.259 (+6.4%)|  0.291(+2.0%) |  0.232(+11.5%)  |  0.247(+14.5%)  |      70ms      |\n|         100%         |  0.248 (+4.2%)|  0.286(+1.7%) |  0.203(+12.5%)  |  0.224(+9.3%)   |     124 ms     |\n\nThe first row of the table, with a scale of the dataset at `0%`, is a special cross-domain example, where we trained the segment encoder on ICSI data and tested it on AMI data, and vice versa. From the results, we can see that from `0%` to `25%`, the model performance increased significantly (average increase of `16.7%,9.7%,17.9%,12.7%`), indicating that using the inference samples themselves for training is very helpful.\n\nSecondly, as the proportion of data increases from `50%` to `100%`, the increase in performance is relatively low, indicating that efficient data selection strategies for training the segment encoder are worth further research. From the results in the table, we can see that time cost is proportional to both training data and performance, so this is a trade-off strategy that depends on specific scenarios.\n\n***\n\n> The framework was only applied to meeting transcripts, and the title aptly states that. However, it is not clear what property of meeting transcripts differentiate them from long form documents. Given this, it indicates the lack of generalizability of this approach.\n\nThank you for your suggestion. We have tested the performance of our framework on other types of documents, such as structured, semi-structured, and unstructured conversational text. Following the setting of the paper [1], we conducted experiments on the Topical Chat, BOLT, and Wiki-727K datasets proposed in the paper, and the experimental results are as follows:\n\n| Method  | Topical Chat |    BOLT   | Wiki-727K |\n|----------------------------|:------------:|:---------:|:---------:|\n| Cross Segment BERT|     0.764    |   0.569   |   0.604   |\n| Cross Segment RoBERTa-Base|     0.767    |   0.561   |   0.599   |\n| Hierarchical Bi-LSTM|   **0.951**  |   0.443   |   0.570   |\n| M3Seg|     0.950    | **0.602** | **0.612** |\n\nAs can be seen from the results, our method has good generalizability and achieves performance comparable to these supervised methods that use domain data for fine-tuning, even though our method is unsupervised. We will add this analysis to our paper in our revised version.\n\n[1] Ghosh et al. 2022. Topic Segmentation in the Wild: Towards Segmentation of Semi-structured & Unstructured Chats.\n***\n\nThank you for all the great questions and suggestions! Please let us know if you have any remaining concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The framework's region based and edge based approaches are well-motivated, and necessary references are provided for the reader to understand the two approaches.\n2. The focus on local and global semantics, and their individual effects via ablation study of segment modeling and boundary/edge detection respectively are explained and studies clearly.\n3. Since the framework can be used with any sentence representation models, an ablation study of the effect of choice of pretrained language model is done and the final choice of model is explained.", "reasons_to_reject": "1. The segment encoder would need to encode all possible segments during training as well as inference. There are $n(n-1)$ number of segments. The quadratic effect is not explained, and it's not clear how it affects time to train.\n2. The framework was only applied to meeting transcripts, and the title aptly states that. However, it is not clear what property of meeting transcripts differentiate them from long form documents. Given this, it indicates the lack of generalizability of this approach.", "questions_to_authors": "1. During training, did you train the segment encoder over all possible segments? It seems so because you don't use the annotation data to know the start and end sentences of the segments.", "paper_topic_and_main_contributions": "Main contributions of the paper are:\n\n1. Mutual information based framework that incorporates region-based and edge-based techniques for topic segmentation of a document. This framework captures both the local and global topic level semantic information.\n2. Experimental evidence of the effectiveness of the framework compared to other SoTA methods on two public meeting transcripts datasets.\n3. Ablation studies to study the effect of region based and edge based techniques' contribution to the overall performance of the framework, as well as, ablation studies of different sentence representation models.\n4. Clearly articulated ideas, and well-presented material.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This is a strong paper according to most reviewers. They value the effectiveness and novelty of the method, the proper referencing, clear explanations and motivations.  Several of the issues brought up by reviewers were properly addressed by the authors, for example, they conducted experiments on the datasets proposed in Gosh et al.’s work. All in all, the authors did a good job in addressing the reviewer’s concerns and building trust in being able to improve the final version of this paper, which resulted in some scores being raised. There are four reviewers for this paper as one of the three initial reviewers explained that the topic was outside their field of expertise. I am basing my meta-review on the three other reviewers. Both soundness and excitement scores are close to 4 on average.
Decision: Accept-Main