REGen: Multimodal Retrieval-Embedded Generation for Long-to-Short Video Editing
Authors: Weihan Xu
Yimeng Ma
Jingyue Huang
Yang Li
Wenye Ma
Taylor Berg-Kirkpatrick
Julian McAuley
Paul Pu Liang
Hao-Wen Dong
Venue: NeurIPS 2025 poster ()
Keywords: Multimodal Large Language Model; Retrieval Augmented Generation; Video Editing
Primary area: applications

Abstract: Short videos are an effective tool for promoting contents and improving knowledge accessibility. While existing extractive video summarization methods struggle to produce a coherent narrative, existing abstractive methods cannot `quote' from the input videos, i.e., inserting short video clips in their outputs. In this work, we explore novel video editing models for generating shorts that feature a coherent narrative with embedded video insertions extracted from a long input video. We propose a novel retrieval-embedded generation framework that allows a large language model to quote multimodal resources while maintaining a coherent narrative. Our proposed REGen system first generates the output story script with quote placeholders using a finetuned large language model, and then uses a novel retrieval model to replace the quote placeholders by selecting a video clip that best supports the narrative from a pool of candidate quotable video clips. We examine the proposed method on the task of documentary teaser generation, where short interview insertions are commonly used to support the narrative of a documentary. Our objective evaluations show that the proposed method can effectively insert short video clips while maintaining a coherent narrative. In a subjective survey, we show that our proposed method outperforms existing abstractive and extractive approaches in terms of coherence, alignment, and realism in teaser generation.

REGen: Multimodal Retrieval- Generation for Long-to-Short Video Editing
Weihan Xu
Yimeng Ma'
Yang Li' Julian McAuley² Paul Pu Liang*
Jingyue Huang
Wenye Ma² Hao-Wen Dong
Taylor KirgpKirkpatrick
Duke University
University of California San Diego MMBZUAI
*Massachusetts Institute of Technology UUniversity of Michigan
weihan xu@duke. edu
Abstract
Short videos are an effective tool for promoting contents and improving knowledge accessibility. While existing extractive video summarization methods struggle produce coherent narrative, existing abstractive methods cannot 'quote" from the input videos, i.e., inserting short video clips their outputs. In this work, we explore new video editing models for generating shorts that feature coherent narrative with embedded video insertions extracted from a long input video. We propose a novel retrieval-embeedee generation (REG) framework that allows a large language model to quote multimodal resources while maintaining a coherent narrative. Our proposed REGen system first generates the output story script with quote placeholders using a finetuned large language model, and then uses a multimodal retrieval model to replace the quote placeholders by selecting video clip that best supports the narrative from a pool of candidate quotable video clips. We examine the proposed method on the task of documentary teaser generation, where short interview insertions are commonly used to support the narrative documentary. Our objective evaluations show that the proposed method effectively insert short video clips while maintaining a coherent narrative. In subjective survey, we show that our proposed method outperforms existing abstractive and extractive approaches in terms of coherence, alignment, and realism in documentary teaser generation.
Introduction
Generating shorts from long videos allows audiences to digest information in more engaging way and helps content creators promote their original contents. Unlike text or visual-only summaries, short videos with visuals and audio are more engaging [1], accelerate comprehension [2], and improve recommendation and search [3]. Existing approaches for producing shorts from long videos can be categorized into extractive abstractive methods. Extractive methods stitch together video clips extracted from the input video [4-10], yet this may produce disjointed videos that do not together convey a coherent story. In contrast, abstractive approaches synthesize new narratives [11] and even new scenes [12], but these methods cannot insert extracted video clips from the input video to support the generated narrative. Moreover, while recent eetrieval-augmeee can augment a large language model (LLM) with additional knowledge at inference time, these methods cannot quote multimodal materials from external sources and embed the exact quotes into their outputs, and they sometimes fabricate or misattribute content when faced with extended contexts [13-15].
generation (RAG) methods
Crafting effective short videos requires both creating a coherent narrative and grounding with raw material extracts, especially for domains that necessitate strong factualness and reliability such
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Input long video
Extracted fromes
Extracted quotable video ciiss
Speech recognition
Transcribed script
S000 00
Harmation
Saba lamed' houre pataringee pigppeo mhst
Spe eo Monouu
canaec brable Mer or caloag net netyenee
3058-009225
Interwlon
These air rough simp cuskers,
Quate Retriever
2036-60
Nerratton
rocm desting then
wilcome
Output short vides
30x2-000
0233-00-22
Intewdoon
cesive dellored day before
Frce steis, uncerveir saits, shoes evrytling
Script generation
(0066-0030
Nerration
Hanssl sermes here ever day and thancdss for
Story script
Siba Hor ed liees in Couca, a Spanich punonorooi communtty by
tryingro thouuh Suoin from Morocoo -QUIIiE> it's risuiiig cros....
Text-to-speech synthesis
Perracion-visuat matching
Figure I: An overview of the proposed REGen system for long-to-short video editing. Given long input video, we first transcribe the narrations and dialogues using pretrained automatic speech recognition model, and then we finetuned large language model to generate the output story script with quote placeholders (i.e., the <QUOTE> token). For the generated narration, following [11], we first synthesize the narration into audio using text-to-speech synthesis model and apply a narration-visual matching algorithm to find accompanying visuals. For the generated quote placeholders, we propose an encoder-decoder based Quote Retriever select video clip that best supports the narrative from a pool of quotable video clips extracted from the input video. The proposed system represents a new hybrid video editing model that combines abstractive and extractive methods.
as journalism and education. Further, there exists no video dataset with annotation that identifies externalyy quoted footage from original narrative segments, making it hard to approach this task through a data-driven approach.
In this work, we introduce REGen, a novel multimodal retrieval-embeeddd generation framework for editing long videos into shorts (see Fig. 1). In the first script generation stage, we finetune an L.M to generate story scripts with quote placeholders that will be fulfilled later the second stage. In the second quotation retrieval stage, we then train multitask encoder-decoder language model to select a video clip extracted from the input video so that it can support the narrative. For other generated narration, we follow [11] synthesize the narration and accompanying it with visuals selected from frames extracted from the input video. To train the proposed system, we use the Documentary dataset [11] and construct training samples with transcribed, timestamped narrations and quotable interviews using an existing speech transcription and speaker diarization model [16].
We conduct extensive experiments to evaluate the effectiveness of our proposed models through objective evaluation metrics and a subjective survey. We show that the proposed REGen models can effectively edit long documentary into a short teaser that has coherent narrative and contain video quotations that support the narrative. Our experimental results show that our proposed method outperforms several abstractive and extractive baseline models in terms of coherence, audiovisual alignment, and realism. Video samples and all source code can be found on our website."
Our contributions can be summarized as follows:
-mbeededd generation (REG) framework that allows LLM to quote multimodal resources while maintaining a coherent narrative.
We propose
We propose novel long-to-short video editing model for generating shorts that feature coherent narrative with embedded video insertions extracted from a long input video.
2 Related Work
Generative modeling with factual grounding Previous work on generative modeling with factual grounding is primarily in the text domain and falls into two main categories; attribution-awar LLMs [17, 18] and reureeval-aummented generation (RAG) methods [19-21]. Attributin-awaa LLMs enhance verifiability by generating responses with in-text citations or via post-hoc attributions. In
'https://wx83, github. 1o/REGen/

---

Table I: Comparison of related methods in trailer generation and multimodal summarization
Type
Input modality
Output modality
Video clip insertions'
Model
Ext.
Abs.
Frames
Video
Narr.
Text
Video
Frames
CIIIPITT [24]
A2Summ [4]
V
/
LIVS [25]
X
V
TGT [26]
X
X
X
TaleSumm [27]
X
v
V
V
VTSUM-BLID [28]
TeaserGen [11]
X
REGen [11]
V
V
Whether its outputs include extracted video clips with original sounds
Achieved by dense video captioning
addition, they employ coarse attributions such URLs [22] or document ( RAG-based approaches first retrieve relevant documents or text chunks and then condition the generation on the retrieved passages [19-21]. ].]. In contrast, our work targets multimodal outputs and performs exact quoting from external multimodal resources, generating narratives that directly quote raw quotable footage as grounding evidence to support the generated narratives.
Long-to-short video editing Long-to-short video editing like video summarization or trailer generation addresses the challenge of condensing forg-form videos into informative short videos. Prior work can be broadly divided into extractive and abstractive approaches. Extractive methods identify and splice together key clips directly from the source footage. For example, A2Summ [4] produces extractive summaries with a unified multimodal transformer-bast sentences and their time-aligned video segments. LfVS [25] utilizes large language models (LLMs) to extract key sentences from transcribed text, which are then paired with time-aligned video segments summaries. TaleSumm [27] introduces two-level hierarchical model
model to predict key
create bseudo-grounde that identifies important sub-stories a TVV episode narrative. Although these techniques preserve the authenticity of the original clips, they often yield disjointed viewing experience due to abrupt transitions between extracted segments. Abstractive methods, by contrast, first generate cohesive narrative script and then retrieve synthesize matching visuals. For instance, TeaserGen [11] prompts a large language model to produce a teaser script and subsequently fetch corresponding video clips. VTSUM DISUMBLIIP [28] jointly train parallel video and text summarization decoders, enabling end-to-end video-to toe--t-vviide video summarization. Though abstractive approaches deliver smoother, more story-like short videos, they risk drifting from factual grounding. In this work, we propose hybrid framework that automatically generates coherent narrative and seamlessly inserts extractive quotable segments as grounding evidence. We compare related methods in Table
Method
To generate short videos that quote contents from long videos, adopt two-stage method: first, we generate scripts with explicit quotation encoding; then we retrieve the corresponding quotable segments from long videos fulfill each quotation coherently and support the surrounding narration.
3.1 Generating Script with Quotation via Fine-Tuned LLAMA
To train an L.M to identify quote insertion points, we leverage ASR with speaker diarization (see Appendix D) to generate data with quotable segments separated from narration. We explore two quote encoding methods for finetuning pretrained language model to enable quoting from long contexts:
REGen-DQ (direct quote): REGen-IDQ (indirect quote):
Tz, <SOQ>,,1......................,. ...... <EDQ>, II+1....
(1)
T3, <QUOTE>, T;+1....
(2)
To ensure comprehensive coverage of the souree material, we transcribe the documentary audio with WhisperX [16], split into ten chunks, and use GPT-4o [29] generate one-sentence summary for

---

Y1. Fa You (EOS), (SUM)
t
Encoder layer
Decoder layer
Encoder
Decoder
Encoder layer
Decoder layer
(S0S), X1
(QUOTE), of
pext 2. "g (BOS)
(SO5), y1, 1 1 ye. (EOS)
Extracted quotable video clips
Thise wander aimlessly ugh city, beach, Its bac.
"All here ware di lvered the day before shoes, ovarything."
Quote Retriever
the cip that maximiiess dlip ftiness OF = cos sim(h eq)
Optimal clip
Figure 2: An illustration of the proposed two-stage quote retriever REGen-IDQ. We finetune an encoder-decoder language model that learns 1) fulfill quotation placeholders (i.e., the <QUOTE> token) and 2) produce an embedding vector h that summarizes the quotation content. We then use the embedding vector h as the query to retrieve video clip from a pool of candidate quotable video clips extracted from the input video. The optimal clip is selected based our proposed clip filness measure (see Section 3.2 for its definition). In this work, we consider all non-narrator video clips as candidates for the quote retriever. Note that this framework can be generalized to support quoting materials in any modality such as audio and images as long as we can find a proper fitness measure.
each chunk as inputs to LLM. Specially, we finetune LLaMA [30] using instruction finetuning. The finetuning templates can be found in Appendices B.4 and B.5.
For the generated narrations, following TeaserGen [11], we accompany the synthesized narration with visuals using an interval-based matching approach, extracting content corresponding to each narration sentence. We use the pretrained UniVTG model [3 to identify highlights aligned with the narrations. For the quotable part, we search our quotable clip base for the sentence embedding closest to the text wrapped in <SOQ> and <EOQ> for direct quotes. We will refer to this model as REGen- DQ. For indirect quotes, we propose retriever module to retrieve quotable segments that blend with the surrounding narration (see Section 3.2), which we will refer to as REGen-IDQ.
3.2 Retrieving Quotable Segments Aligned with Narrative Context for RGEen-IDQ
As shown in Fig. retrieve a quotable segment from curated database that supports the surrounding narrative, we frame this task as a multitask learning problem: the proposed quote retriever is trained jointly with masked language modeling loss and retrieval loss.
To fulfill the content of the quotation placeholders generated by REGen- we finetune a pretrained mncoder-decoder model (specifically, BART [32] in this work) to perform masked infilling conditioned on surrounding narrations. Let previous sentence be gprev = [xxprev prev ,...,, prrrev and the following sentence be 8 neat (xxexx ab seett).
We have our encoder input and decoder output
Input:
<OSS>, prev
pree <QUOTE>
next #1
next <EOS>
Output:
<SOS>, 91 No: <EOS>, <SUM>
The proposed method decodes meaningful sequence supporting nearby narrations, while the added special <SUM> token expected to summarize the content of the quotation.
Inspired by retrieval-augmente generation (RAG) [19-21], we propose a retrieval module to find a suitable quotable segment from a candidate pool that best matches the decoded sentences. We use the hidden state of the final decoder token (i.e.. the special <SUM> token) from each generated sequence to retrieve quotable video segments from the candidate pool. For each <QUOTE> placeholder and Cas), }, we retrieve the optimal video clip c*
pool of candidate quotable video clips C' {ci,.... .ar that maximizes the clip fitness, defined as CF cos cos_ simms cm), where h is the last-layer hidden state at the final decoder layer for the last token (i.e., the <SUM> token) and em is an embedding vector that captures the semantic meaning a candidate clip Cm. We consider two variants of e..: First, we consider em = [cconcal(e text eimm)) 3), where ʃ is learnable mapping parameterized as a two-layer multi-layer perceptrons and feet is the sentence embedding of the whole narration of the quote video segments. To aggregate visual information, define e as the concatenated

---

frame embeddings of three randomly selected frames for the quotable segments. The proposed multimodal fusion module is expected learn combine visual and textual information efficiently to optimize the retrieval performance. We will refer to this retriever as QuoteRetriever-TV. The REGen-IDDO model using this retriever will be referred as REGen-IDQ-TV. consider em exxt i.e., a retrieval model that considers only textual information. We will refer to this retriever as QurteRetriever-T and the corresponding full system as GEGen-IDQ-1
addition, we
During training, we jointly train the fusion module and fine-tune the pretrained BART[32] model with multitask loss function: L = Lgen + Lret. where Lgen is the token level cross- eross-entropy loss for masked language modeling, and Liet is the retrieval loss defined as
aop(cos_sim(ha) e²)) e,)) Lioet Σ log Lejec,
where e" is the sentence embedding of the ground truth narration and Cj the set of negatives samples for quotation placeholder k. Furthermore, we train the retriever with in-batch negative sampling, and explore GroupSampler module construct hard negative samples, further separating ground-truth quotable video clips from the remaining quotable video clips in the video clip base (see Appendix E,4 for more details).
4 Experimental Setup
4.1 Dataset and Implementation Details
In this work, we use the DocumentaryNet [11] dataset in our experiments. DocumentaryNet contains 1,269 documentaries paired with their teasers from three reliable sources: DW Documentary, Public Broadcasting Service (PBS) and National Geographic. We perform speaker diarization generate scripts using WhisperX [16] on DocumentaryNet. We detect narrator segments by assuming that narrations correspond to the longest transcribed audio segments (we examine the robustness of our speaker diarization and narrator identification in Appendix C). In this work, we consider all non-narrator video clips quotable video clip candidates for the quote retriever. To validate data annotations, we recruited four people to validate the ASR and narrator-idettffcatioo results. We FPS (frame per second) of 1. We include details about the datasets and implementation in Appendices and E.
Script Generation
In our first experiment, we evaluate the performance of our proposed method in generating coherent scripts with quotations, as well as its ability to conduct exact quotations from input videos.
Baselines We compare our model with script existing multimodal video summarization model [4], teaser generation model [11], and three LLM-based method.
We randomly sample sentences from main documentary transcription. (ETS): We select two interviews whose content is closest to the video title and ask GPT-4o to connect the extracted interviews into a cohesive story. We include the prompts in Appendix B.7.
Random Extraction:
tetracivvetheenSSmoohhe
A2Summ [4]: This baseline model uses an extractive method to select joint textual and visual segments with temporal correspondence. Since A2Summ can only process videos shorter than 300 seconds, we divide each video into ten chunks, select key segments from each chunk separately, and concatenate them together.
TeaserGen [11]: This baseline model divides the audio transcription of long videos into ten chunks, generates a one-sentence summary for each, and instructs GPT-4o [29] weave these summaries into story-like teaser ending with compelling question.
GPT-4o-DQ: We transcribe each documentary with WhisperX [16], split the transcript into chunks, summarize each chunk, and concatenate the summaries into single documentary-level summary. Using this summary as context, GPT-4o [29] generates an introduction with in-text quotations. See Appendix B.1 for the prompt.
GPT-40-SP-DQ: We split the speaker-labeled script processed Section 4. into ten chunks and ask GPT-4o [29] to generate concise, engaging summary for each chunk. We then concatenate
5

---

these ten summaries to reconstruct complete, speaker-labeled script. We include our prompts in Appendix B.3.
LLaMA-DQ: We prompt LLaMM[30] to generate an introduction script given a summary of the main documentary content and include in-text quotations. We include the prompt in Appendix B.2.
Objective Evaluation Metries We evaluate our quotation insertion with three primary metrics. First, we measure quotation density index (QDI), i.e., the average number of quotes inserted per documentary. Second, we compute quote coverage rate (QCR), the proportion of test vidcos in which at least one quotation is correctly inserted. Third, we define overlap ratio (OR) to measure the overlap between direct quoted contents by large language model with the ground truth interviews as OR # (overlap words) # worrds in matched interviews).
4.3 Quote Retriever Evaluation
In this experiment, evaluate whether the proposed retriever can retrieve the correct video clip in our test set and assess its generalizability when applied to LLM generated scripts.
Baselines To evaluate the effectiveness of our proposed quote retriever, we compare it against two baselines: random selection and GPT-based infilling of <QUOTE> with surrounding narrations.
Random Selection:
Randomly choose interview segments for insertion.
GPT-4o Infilling: Given the preceding and succeeding narration chunks, we prompt GPT-4o [29] to generate content to fill the <QUOTE> position, and then retrieve the nearest neighbor the sentence embedding space [33] from our interview base. A detailed analysis for this baseline can be found in Appendix I.
Evaluation Metrics For objective evaluation, we evaluate our retrieval stage using recall, reporting Recall I. 5 and Recall @ 10 on teasers in the test set. Recall indicates that the correct segment appears among the top five retrieved interviews. To further assess the generalizability of our retriever when applied to LLM --Me-geeerteed scripts, we conduct a subjective evaluation. Twenty-one participants (1 evaluating version A and 10 evaluating version B) rate each inserted interview segment in the generated teasers on five- live-point& Likert scale, judging the effectiveness of the insertion based on how well it supports the surrounding claim and maintains natural flow. We include survey questions in Appendix K.
4.4 Documentary Teaser Generation
In this experiment, we measure our model performance in documentary teaser generation task.
Baselines We consider Random Extraction, Extractive-then-Se then-Smoothinng. A2Summ [4], and Teaser- Gen [11] described in Section 4.2 baselines. Additionally, for GPT-4o-DQ, extract quoted segments within quotation marks and use nearest-neighhor retrieval to fetch matching visual clips from the interview pool; for GPT-4o-SP-DQ.
retrieve interview segments from the interview pool via nearest-neighhor search on script segments labeled as non-narrator content.
Evaluation Metrics We first measure the quality of the final script, where each <QUOTE> marker generated by the seript-generation stage has been replaced by its retrieved interview segment in the retrieval stage. Specifically, we report ROUGE F1 [34], which measures n-gram overlap and sequence continuity between generated and reference teaser scripts. In addition, we assess narrative coherence using G-Eval [35] on the DeepEval platform [36]. In addition, following TeaserGen [11], we report five retrieval-bacc based metries: F1, scene change rate (SCR), repetitiveness (REP), CLIPScore [37], and VTGHLS [35, 11]. Fl measures retrieval accuracy against the ground-truth teaser, while SCR (the frequency of scene transitions) and REP (the degree of repetitive content) capture aspects of temporal continuity that affect the viewer experience. We report CLIPScore [37] (specifically, CLIPS-I and CLIP-N) to measure audiovisual alignment, and VTGHLS [35, 11] to measure the likelihood that each selected frame will be perceived as highlight relevant to the video title (see Appendix F for more details). Moreover, we define the interview ratio as the fraction interview time (in seconds) in video.

---

Table Objective evaluation results for documentary teaser script generation
Before fulfillment
After fulfillment
Model
Tokens
QCR (%)
QDI
Tokens
R-1
R-2
R-L
G-Eval
Random extraction
98
11.71
235
0.27
0.04
0.12
0.56 + 0.02
ETS
96
1.66
340
0.21
0.03
0.11
0.81 0.01
A2Summ [4]
96
3.98
172
0.27
0.03
0.13
0.42 0.01
TeaserGen [11]
304
0.21
0.03
0.11
0.85 1 0.01
GPT-4o-DQ
292
98
4.02
402
0.22
0.05
0.12
0.77 0.01
GPT-4o-SP-DQ
631
100
22.33
1372
0.13
0.03
0.07
0.75 0.01
REGen-DQ
153
76
2.31
210
0.28
0.05
0.13
0,43 1 0.02
REGen-IDQ-T
98
67
1.98
172
0.25
0.04
0.13
0.57 0.02
REGen-IDQ-TV
67
1.98
179
0.25
0.04
0.13
0.59 0.01
Ground truth
82
3.02
121
0.62 0.03
Table 3: Comparisons of quote retrieval methods
Similarity
Recall @ measure
Recall (%)
Recall @ 10 (%)
Insertion
Retriever
(%)
effectiveness
Random
0.00 0.00
0.28
0.48
7.22 5.54
3.08 0.25 2.48 + 0.31
GPT-4o infilling
Text only
2.78 = 0.48
13.89 + 1.27
22.50 + 1.44
QuoteRetriever-T
Text only
5.00
17.50
30.00
3.56
0.22
QuotcRetriever-TV
Text+ Visual
5.00
15.00
23.33
3.49 1 0.26
Following the subjective study Section 4.3, we randomly select ten documentaries from the test set, divide them into two groups of five, and ask participants to evaluate the generated teasers on coherence, alignment, realism, and interview effectiveness using a five-point Likert scale.
4.5 Ablation Study
We include additional ablation study in Appendix L. We evaluate the effects of the the max length of BART tokenizer [32] (Appendix L. 1), the alpha parameter for balancing losses (Appendix L.2), the use of GroupSampler during retriever training (Appendix L.3), the choice of loss function (Appendix L.4), and the position of the retrieval token <SUM> (Appendix L.5).
5
Results
5.1 Script Generation
In this experiment, we compare our model performance generating scripts with quotations against the following baselines: GPT-4o-DQQ GPT-40-SP-DDO random extraction, A2Summ [4], and Teaser- Gen [11]. First, we evaluate whether LLMs such as LLaMA [30] and GPT-4o [29] can directly quote from long contexts, and we report our results in Appendix G. We find that GPT-4o cannot quote exact content from long inputs when fed with the full transcript. In addition, while vanilla LLaMA cannot produce meaningful quotes using quotation marks, the proposed finetuning method with <sOQ> and <EOQ> increases the overlap ratio from 0.0 to 0.07. Second, we compare our model to GPT-4o-DQ. GPT-4o-SP-DO Random Extraction, A2Summ [4], and TeaserGen [11] using the quotation density index (QDI) and quote coverage rate (QCR). As shown in Table 2, we find that REGen-DD achieves QCR and QDI values closest those of the ground truth, indicating that REGen-DQ generates scripts with a similar quote distribution to the ground truth scripts.
5.2 Quote Retriever
In this experiment, we aim to compare the retrieval capability of our model against GPT-based infilling method. First, report in Table 3 recalls for retrieving the correct interview segments given
7

---

Cipp Fitasss
Finees
Fitress Distribution
01
66
2.6 64
02
64
08
08
344
Bod asstmancdn
Ica sic 048
11 de inees
12.10 0DC predd
68 -44
04 Simtenty
06
03
-26 -64
-02
0.2 6- Cosnne Simiemy
-65 -34
-02
3.
0.0
24
Costre Smile
(a) GPT-4o infilling
(b) QuoleRetriever-T
(c) QuoleRetriever-TV
Figure 3: Comparison of infilling methods. The dotted lines indicate the median values.
Table Objective evaluation results for documentary teaser generation
Dur (sec)
Interview ratio (%)
F1 (%)
SCR
REP
Model
(%)
(%)
VTGHILS
CLIPS-I
CLIPS-N
Random extraction
101
56 + 20
1.10
20.71
0.41
0.83
0.55
0.62
ETS
142
34 + 16
1.92
13.65
4.49
1.06
0.64
0,60
A2Summ [4]
42 25
1.70
14.20
1.73
0.89
0.56
0,63
TeaserGen [11]
1.64
22.61
21.38
0.80
0.67
GPT-4o-DQ
151
42 + 42
1.56
16.55
20.75
1.01
0.58
0.42
GPT-4o-SP-DQ
619
61 + 17
2.07
12.38
18.33
1.02
0.62
0.62
REGen-DO
95
37 + 26
1.45
19.13
10.35
1.05
0.48
0.57
REGen-IDQ-TT
77
35 + 31
1.89
19.79
10.02
1.03
0.41
0.57
REGen-IDD-TV
81
35
1.90
19.86
9.70
1.02
0.39
0.57
Ground truth
76
54 + 37
69.00"
27.60
7.886
<0.98
0.43
0.57
Following [1]], for each frame in the teaser, retrieve the 20 similar frames from main using CLJP embeddings. We then apply pixel-by-pixel comparison to the 20 candidates, however, this strict matching may fail to identify identical frames due to the low frame ratc.
their preceding and succeeding narration of each teaser in our test set. On average, each video in our test has 66 candidate interviews with standard deviation of 37. Our proposed QuoteRetriever-T and QuoteRetriever-T TV outperform infilling with GPT-4o and random selection in terms of retrieval recall. See Appendix for details about the objective evaluation on quote retrievers.
Second, to evaluate the discriminative capability of our retriever on interview segments, we plot in Fig. 3 the top-1 retrieval similarity ("Top-1 KDE"") alongside the all similarity distribution ("All-sim KDE"), where 'all' refers to the similarity between the embedding output by the fine-tuned BART [32] model and all interview segments. We observe more prominent separation between the top-1 KDE and All-sim KDE for QuoteRetriever--V than the GPT-4o infilling approach, which is beneficial for more accurate retrieval performance due to the increased contrast between probable candidates.
Third, we evaluate quote retriever performance by measuring how effectively retrieved segments can be inserted into generated narration scripts in subjective study. We report the mean score and 95% confidence interval in Table 3. Both QuoteRetriever-T T and QuoteRetriever-TT outperform the GPT-infilling baseline in terms of interview effectiveness, indicating that pretrained GPT-4o cannot produce meaningful text to fulfill the <QUOTE> placeholders and support accurate retrieval.
5.3 Documentary Teaser Generation
To evaluate the performance of the proposed method for the documentary teaser generation task, compare our proposed method against several baselines models: random extraction, A2Summ [4], TeaserGen [11], Extractive- Serrative-then-Sneee then-SSoothing (ETS), GPT-4o-DQ. and GPT-4o-SP-DQ. As reported in Table 2, we can see that REGen-Do achieves the highest ROUGE scores, indicating that REGen-Do DQ generates scripts closest the ground-truth teasers Appendix J. We also find that REGen-IDQ-TV achieves the closest G-Eval [35] score to that of the ground-truth and that 4PT-4-SP- the highest F1 score. Notably, the CLIPScore for the interview scenes of the ground truth is lower than that of the narration scenes. This suggests lower narration-visual correspondence for interview scenes, which is partly because interview scenes usually focus on the interviewees rather than having visuals to support the narration content. Our proposed REGen models also result in smaller CLIPS-I value than CLIPS-N, which consistent to the ground truth documentary teasers. As can be seen from Table 5, our subjective evaluation results show that the proposed LEGen-IDQ-In model achieves the
achieves

---

Table 5: Subjective evaluation results for documentary teaser generation
Model
Coherencef
Alignment?
Realness
Interview effectiveness?
A2Summ [4]
2.72 = 0.24
2.87 + 0.26
2.67 = 0.23
3.07 = 0.24
TeaserGen [11]
3.22 = 0.23
2.92 1 0.24
2.86 + 0.23
GPT-4o-SP-DQ
3,08 - 0.24
3,23 + 0.25
2,81 + 0.25
3.32 + 0.25
REGen-DQ
2.97 - 0.27
3.03 + 0.27
2.75 + 0.30
3.33 + 0.29
REGen-IDQ-TV
3.29 1 0.24
3.30 1 0.26
3.05 1 0.25
3.25 + 0.30
highest scores in terms of coherence, alignment, and realness, outperforming our another proposed REGen-DQ model. Meanwhile, our proposed REGen-Do method achieves the highest interview effectiveness score, but this does not reach significance difference against REGen-IDQ-TV.
For the prraative-then-smootiing (ETS) baseline, its higher VTGHLS score likely results from selecting interview chunks closest to the video title in sentence-embeddiig space [33]. but this value higher than the ground-truth VTGHLS. Additionally, the ETS method has a much lower scene change rate than the ground truth teaser. In Table 2. we observe that this method yields lower ROUGE scores than our proposed model, which indicates less overlap with the ground-truth teasers, while its higher G-Eval rating partly results from the enforced story-like prompt in Appendix B.7. Second, A2Summ [4] yields the lowest G-Eval score in Table 2, reflecting its low narrative cohesion, which is further verified by the lower coherence score in the subjective evaluation in Table 5. As shown in Table 4, our model produces scene change rate (SCR) and repetition rate (REP) closer to those of the ground truth than A2Summ, which aligns with the higher perceived realness of our model in the subjective test. Third, TeaserGen [11] achieves the highest G-Eval score of 0.85 in Table 2, indicating that it produces the most story-like and cohesive output: however, this value is higher than the ground truth value (0.62). Also, REGen- IDQ-TV achieves higher alignment than TeaserGen in the subjective test, suggesting that TeaserGen may select content that is less audio-visually aligned than naturally extracted interview segments. In addition, TeaserGen has the lowest VTGHLS score, suggesting that its retrieved frames are less likely to be considered as highlighted moments for its video title.
Finally, we compare our proposed model with GPT-based models, including GPT-4o-DO and GPT- 4o-SP-DQ. We can see that the proposed REGen models produce scripts with higher ROUGE scores, indicating that the generated scripts are closer to the ground-truth teaser scripts. In the documentary teaser generation task, although GPT-40-SP-DQ achieves the highest F1 score, it exhibits much lower scene change rate and significantly longer teaser length than the ground truth. Even though we cap teaser length at 500 tokens (approximately 2.5 minutes of speech assuming a normal speaking pace of 150 wpm), the generated teasers remain substantially longer than real ones. The lower scene change rate and higher repetitiveness suggest that GPT-40-SP-DO selects repetitive video clips, which can lead to negative viewer experience. This aligns with the lower perceived realness of GPT-4o-SP. DQ compared to REGen-IDQ-TV in Table 5. We also compare model with GPT-4o-SP TP--TTVV where we use the speaker- labeled script described in Section 4.1 and, with QuoteRetriever- TVV, retrieve only when the speaker is labeled non-narrator. We report results in Appendix H.
6 Limitations and Future Work
We want to point out several notable limitations of our work. First, while including video insertions may improve the factualness grounding of the output videos, the proposed method still has the risk of misplacing a quote in a wrong context. This may be alleviated by grounding the first-stage script generation model with information about all the quotable materials so that can better generate a more cohesive narrative. To examine this hypothesis, we include in Table 7 baseline model (GPT-40-DQ-NS) that supplies GPT-4o [29] with the full transcript and all candidate quotable interview segments for script generation. However, this is technically challenging for LLaMA- based models due to the their limited context- window [30], which prevents us from providing all the quote candidates as the context for script generation. Moreover, the proposed method relies on successful segmentation of the input video, which is possible in our case through speaker diarization that might not be applicable to other domains such as lecture recordings. For future work, we note that the proposed framework can be generalized to support quoting materials in other modalities as long as can find proper fitness measure. We plan to investigate quoting audio and images towards
9

---

more capable video editing model. We would also like to extend our proposed retrieval embedded augmentation (REG) framework other application domains such as virtual assistants, chatbots, and recommender systems.
7 Broader Impacts
We envision our proposed method to be applied to other fields such as education technology. natural language processing and information retrieval. For education technology. believe our proposed model can be adapted to generate short review videos from lecture recordings to enhance learning experience. The proposed multimodal retrieval-embedded generation method can also be applied to current LLMs so that they can quote external sources embedded in their outputs. From the information retrieval perspective,
proposed method brings the power of LLMs to extractive methods where can generate a coherent narrative to connect multiple extracted materials. We believe our proposed framework will contribute improving knowledge accessibility by generating engaging short videos for long videos that may be more approachable to certain groups.
8 Conclusion
We have proposed novel to include multimodal quotations in its outputs. We have examined the proposed method on the task of documentary teaser generation. We have shown through objective and subjective evaluations the effectiveness of our proposed REGen models on quoting short interview clips within a coherent narrative. The subjective evaluations show that our proposed hybrid approach outperforms several abstractive and extractive baseline models in terms of coherence, alignment, and realism. Our work contributes towards next generation Al-assisted video editing tools that can assist video creators in selecting, cutting, and arranging raw video materials into cohesive video.
generation (REG) framework that allows an LLM
9
Acknowledgment
This work is supported by the NVIDIA Academic Grant Program under the project titled "Teaser Generation for Long Documentaries and Educational Videos."
References
[1] L. Zhang, Yu, S. Zhang. L. Li, Y. Zhong, G. Liang. Y. Yan, Ma, F. Weng. F. Pan, J. Li. R. Xu, and Z. Lan, "Unveiling the impact of multi-modal interactions on user engagement: A comprehensive evaluation in ai-driven conversations," 2024. [Online]. Available: https://axxiv.or//bb2240... 15000
[2] A. Gatcho, J. Manuel, and R. Sarasua, "Eye tracking research on readers' interactions with multimodal texts: a mini-review," Frontiers in Communication, vol. 9, 12 2024.
Y. Liu, Y. Wang, L. Sun, and S. Yu, "Reo-gpt4v: Multimodal recommendation with large languae models," 2024. [Online], Available:
[4] B. Hc, J. Wang, Qiu, T. Bui, Shrivastava, and Z. Wang, "Align and attend: Multimodal summarization with dual contrastive losses," 2023. [Online]. Available: httpss//arxiv.or//bb///:2
[5] B. Gaikwad, A. Sontakke, M. Patwardhan, N. Pedanekar, and Karande, "Plots previews: Towards automatic movie preview retrieval using publicly available meta-data," Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 3205-3214.
[6] Hesham, B. Hani, Fouad, and Amer, "Smart trailer: Automatic generation of movie trailer using only subtitles." " in 2018 First International Workshop on Deep and Representation Learning (IWDRL). IEEE 2018, pp. 26-30.
[7] Y. Kawai, Н. Sumiyoshi, and N. Yagi, "Automated production of tv program trailer using electronic program guide." in Proceedings of the 6th ACM International Conference on Image and Video Retrieval, 2007, pp. 49-56.
[8] A. King. E. Zavesky, and M. J. Gonzales, "User preferences for automated curation of snackable content." in 26th International Conference on Intelligent User Interfaces, 2021, pp. 270-270 274.
10

---

[9] X. Liu and J. Jiang, "Semi-supprrisse learning towards computerized generation of movie trailers." in 2015 IEEE International Conference on Systems, Man, and Cybernetics. IEEE, 2015, pp. 2990-2995.
[10] J. R. Smith, D. Joshi, B. Huet, W. Hsu, and J. Cota, "Harnessing ai for augmenting creativity: Application to movie trailer creation." in Proceedings of the 25th ACM International Conference Multimedia, 2017,
pp. 1799-1808.
[11] W. Xu, P. P. Liang. H. Kim, J. McAuley, T. Berg- Kirkpatrick, and H.- Dong, "Teasergen: Generating teasers for long documentaries." 2024. [Online]. Available: https//aaiiioorg/ab//411...0
[12] K. Dalal, D. Koceja, G. Husscin, J. Xu, Y. Zhao, Y. Song, S. Han, K. C. Cheung, J. Kautz, C. Guestrin, T. Hashimoto, Koyejo, Y. Choi, Y. Sun, and X. Wang. "One-minute video generation with test-time training." 2025. [Online]. Available: https//arriv..rr/ab//55....4
[13] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, F. Ishii, Y. Bang, A. Madotto, and P. Fung. "Survey of hallucination in natural language generation," ACM Comput. Surv., vol. 55, no, 12, Mar. 2023, [Online]. Available: https://doi.org wow.ooooor/////////// 10.1 145/3571730
[14] Z. Bai, P. Wang. T. Xiao, T. He. Z. Han, Z. Zhang. and M. Z. Shou, "Hallucination of multimodal large language models: A survey," 2025. [Online]. Available: http:://ardiv.or//ab////44 18930
[15] Z. Xu, S. Jain, and M. Kankanhalli, "Hallucination inevitable: An innate limitation of large language models," 2025. [Online]. Available: hutps:/aaxiiv.orr/ab///0.......
[16] Bain, Huh, T. Han, and A. Zisserman, "Whisperx: Time- accurate speech transcription of long-form audio," 2023. [Online]. Available: https://arxi..og//bb/233......
[17] B. Bohnet, Q. Tran, Verga, R. Aharoni, D. Andor, B. Soares, M. Ciaramita, Eisenstein, K. Ganchev, J. Herzig, K. Hui, T. Kwiatkowski, J. Ma. J. Ni, L. S. Saralegui, T. Schuster, W. W. Cohen, M. Collins, D. Das, D. Metzler, S. Petrov, and K. Webster, "Attributed question answering: Evaluation and modeling for attributed large language models," 2023. [Online]. Available: htts://arri..rrgaaa222222220
[18] L. Huang, X. Feng, W. Ma, Y. Gu, W. Zhong. X. Feng, W. Yu, W. Peng. D. Tang, D. Tu, and B. Qin, "Learning fine-grained grounded citations for autributed large language models," in Findings the Association for Computational Linguistics: ACL 2024, L.-W. Ku, A. Martins, and V. Srikumar, Eds. Bangkok, Thailand: Association for Computational Linguistics, Aug, 2024, pp. 14095-14 113. [Online]. Available: https//aclanthologyccooo wwwp://oooooooooo22224/... www.://24..niiinaaa.......
[19] V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-L Yih, "Dense passage retrieval for open-domain question answering," in Proceedings of the 2020 Conference on Empiricul Methods in Natural Language Processing (EMNLP). B. Webber, T. Cohn, Y. He, and Y. Liu, Eds. Online: Association for Computational Linguistics, Nov. 2020, pp. 6769 6781. [Online], Available: https://acanbboooggrg///22000mo wwwp:/00.mmiimmmi.55551 m5in.550/
[20] P. Lewis, F. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t Yih, T. Rocktäschel, S. Riedel, and D. Kiela, "Retreeval-uugmeented generation for knowledge-intensivee nlp tasks," in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc.. 2020. pp. 9459 9474. [Online]. Available: https: /proceedings www.iis..c/pappeeeee.
fieelpppeer220000//6433300000000 (e266l547799444441.... Papper..
[21] Z. Feng. W. Ma, W. Yu, L. Huang, H. Wang, Q. Chen, W. Peng. X. Feng. B. Qin, and T. liu, "Trends in integration of knowledge and large language models: A survey and taxonomy of methods, benchmarks, and applications." " 2024. [Online]. Available: https://arxiv.orr/abs//11
[22] R. Thoppilan, D. D. Freitas, J. Hall, N. Shazeer, A. Kulshreshtha. H.-T. Cheng, A. Jin, T. Bos, L. Baker, Y. Du, Y. Li. Lee, H. Zheng, A. Ghafouri, M. Menegali, Huang, M. Krikun, D. Lepikhin, J. Qin, D. Chen, Y. Xu, Z. Chen, A. Roberts, M. Bosma, V. Zhao, Y. Zhou, C.-C. Chang. I. Krivokon, W. Rusch, M. Pickett, P. Srinivasan, I.. Man, K. Meier-Hellstern M. R. Morris, T. Doshi, R. D. Santos, T. Duke, J. Soraker, B. Zevenhergen, V. Prabhakaran, M. Diaz, B. Hutchinson, K. Olson, A. Molina, E. Hoffman-John,
Lee, I. Aroyo, Rajakumar, A. Butryna, M. Lamm, V. Kuzmina, J. Fenton, A. Cohen, R. Bernstein, R. Kurzweil. B. Aguera-Areas, C. Cui, M. Croak, E. Chi, and Q. Le, "Lamda: Language models for dialog applications." 2022. [Online]. Available: https://axxivorr//a///// 222000029922299
[23] N. Liu, Zhang, and P. Liang, "Evaluating verifiability in generative search engines," in Findings of the Association for Computational Linguistics: EMNLP 2023, H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 7001-7025, [Online]. Available: https:/accanthooooy.o22222 findings-emmlp.4677
11

---

[24] M. Narasimhan, A. Rohrbach, and T. Darrell, "Clip-it! language-guided video summarization." Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer. Y. Dauphin, P. Liang. and J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 13 9800 [Online]. Available: htpps:
20550000000000000000000000002222 Papppr.
[25] D. M. Argaw, S. Yoon, F. C. Heilbron, H. Deilamsalchy, T. Bui, Z. Wang, F. Demoncoutt, and J. S. Chung. "Scaling up video summarization pretraining with large language models," " 2024. [Online]. Available: htpps//arri..rrgaass2444..00
[26] D. M. Argaw, M. Soldan, A. Pardo, C. Zhao, F. C. Heilbron, J. S. Chung, and B. Ghanem, "Towards automated movie trailer generation." 2024. [Online]. Available: http://aaxii.orrr/a//4444..2
[27] A. K. Singh, D. Srivastava, and M. Tapaswi. " ""previously on ..." from recaps to story summarization." 2024. [Onlinc]. Available: http::/aariivorr/aas/240....... 11487
[28] J. Lin, H. Hua, M. Chen, Y. J. Hsiao, C. Ho, and Luo, "Videoxum: Cross-modal visual and textural summarization of videos," IEEE Transactions on Multimedia, 2023.
[29] OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, Avila, I. Babuschkin, S. Balaji, V. Balcom, P. Baltescu, H. Bao, Bavarian, J. Belgum, I. Bello, Berdine, Bernadett Bernadett-Shapiro,
Shapiro, C. Berner, L. Bogdonoff, M. Boyd, A.-L. Brakman, G. Brockman, T. Brooks, M. Brundage, K. Button, T. Cai, R. Campbell, A. Cann, B. Carey, C. Carlson, R. Carmichael, B. Chan, C. Chang, F. Chantzis, D. Chen, S. Chen, R. Chen, J. Chen, M. Chen, B. Chess, C. Cho, C. Chu, H. W. Chung, D. Cummings, J. Currier, Y. Dai, C. Decareaux, T. Degry, N. Deutsch, D. Deville, A. Dhar, D. Dohan, S. Dowling, S. Dunning, A. Ecoffet, A. Eleti, T. Floundou, D. Farhi, I.. Fedus, N. Felix, S. P. Fishman, J. Forte, I. Fulford, I.. Gao, E. Georges, C. Gibson, V. Goel, T. Gogineni. G. Goh, R. Gontijo-Lopes, J. Gordon, M. Grafstein, S. Gray. R. Greene, J. Gross, S. S. Gu, Y. Guo, C. Hallacy. J. Han, J. Harris, Y. He, M. Heaton. J. Heidecke. C. Hesse. A. Hickey. W. Hickey, P. Hoeschele. B. Houghton, K. Hsu. S. Hu, X. Hu. J. Huizinga, S. Jain, S. Jain, Jang, A. Jiang, R. Jiang, H. Jin, Jin, Jomoto, Jonn, H. Jun, T. Kaftan, Lukasz Kaiser, Kamali, I. Kanitscheider, N. S. Keskar, T. Khan, L. Kilpatrick, J. W. Kim, C. Kim, Y. Kim, J. H. Kirchner, J. Kiros, M. Knight, D. Kokotajlo, Lukasz Kondraciuk, A. Kondrich, A. Konstantinidis, K. Kosic, G. Krueger, V. Kuo, M. Lampe, I. Lan, T. Lee, J. Leike, J. Leung, D. Levy, C. M. Li, R. Lim, M. Lin, S. Lin, M. Litwin, T. Lopez, R. Lowe, P. Lue, A. Makanju, K. Malfacini, S. Manning. T. Markov, Y. Markovski, B. Martin, K. Mayer, A. Mayne, B. McGrew, S. M. McKinney. C. McLeavey, P. McMillan, J. McNeil, D. Medina, A. Mehta, J. Menick, L. Metz, A. Mishchenko, P. Mishkin, V. Monaco, E. Morikawa, D. Mossing, T. Mu, M. Murati, O. Murk, D. Mély, A. Nair, R. Nakano. R. Nayak, A. Neelakantan. R. Ngo. H. Noh, L. Ouyang. C. OeKeefe. J. Pachocki, A. Paino, J. Palermo, A. Pantuliano, G. Parascandolo, J. Parish, E. Parparita. A. Passos, M. Pavlov, A. Peng, A. Perelman, F. de Avila Belbute Peres, M. Petrov, H. P. de Oliveira Pinto, Michael, Pokorny, M. Pokrass, V. H. Pong, T. Powell, A. Power, B. Power, F. Prochl, R. Puri, A. Radford, J. Rae, A. Ramesh, C. Raymond, F. Real, K. Rimbach, C. Ross, B. Rotsted, H. Roussez, N. Ryder, M. Saltarelli, Т. Sanders, S. Santurkar, G. Sastry. H. Schmidt, D. Schnurr. J. Schulman, D. Selsam, K. Sheppard, T. Sherbakov, J. Shich. S. Shoker, P. Shyam. S. Sidor, E. Sigler, M. Simens. J. Sitkin, K. Slama, I. Sohl, B. Sokolowsky, Y. Song, N. Staudacher. F. Such, N. Summers, I. Sutskever, Tang, N. Tezak, M. B. Thompson, P. Tillet, A. Tootoonchian, E. Tseng, P. Tuggle, N. Turley, J. Tworek, J. C. Uribe, A. Vallone, A. Vijayvergiya, C. Voss, C. Wainwright, J. J. Wang. A. Wang. B. Wang, J. Ward, J. Wei, C. Weinmann, A. Welihinda, P. Welinder, J. Weng, L. Weng. M. Wiethoff, D. Willner, C. Winter, S. Wolrich, H. Wong, L. Workman, S. Wu, J. Wu, M. Wu, K. Xiao, T. Xu, Yoo, Yu, Q. Yuan, W. Zaremba, R. Zellers, C. Zhang, M. Zhang, S. Zhao, Zheng, J. Zhuang, W. Zhuk, and B. Zoph, "Gpt-4 technical report," " 2024. [Online]. Available: https//aarivv.or/aas//223..
Boiko,
[30] H. Touvron, T. Lavril. G. Izacard, X. Martinet. M.-.. Lachaux. T. Lacroix, B. Rozière, N. Goyal, E. Hambro, Azhar, A. Rodriguez, A. Joulin. E. Grave, and Lample. "Llama: Open and efficient foundation language models," 2023. [Online]. Available: https//aarivvorr//bb///U
[31] K. Q. Lin, P. Zhang, J. Chen, S. Pramanick, D. Gao, J. Wang, R. Yan, and M. Z. Shou, "Univtg: Towards unified video-language temporal grounding," 2023.
[32] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemayer, "BART: Denoising sequence-to-sequencce pre-training for natural language generation, translation, and comprehension." " in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, July 2020. [Online]. Available:
[33] Sentence Transformers, sentence- transformers/alll-leeoooooa..t mww.lll--m----iiie
CeentenceTrrnsfoomeee mapalll--nnettb-e2222 httpss/huuggiggfacecoo
mpnet-base- base v2, 2022, apache-2.0 220 license; accessed on 2025-04-28.
12

---

[34] C.-Y. Lin, "ROUGE: A package for automatic evaluation of summaries," in Text Summarization Branches
Out. Barcelona, Spain: Association for Computational Linguisties, Jul. 2004, pp. 74-81. [Online]. Available: https//ccaatthoooooo////040
[35] Y. Liu, D. Iter, Y. Xu, S. Wang. Xu, and C. Zhu, "G-eval: Nlg evaluation using gpt-4 with better human alignment," 2023. [Online]. Available: https//arrii.orgaab//333....
[36] Confident AI Inc., "DeepEval: The open-source Ilm evaluation framework," http://www.deepevll.com 2025, accessed: 2025-04-29.
[37] J. Hessel, A. Holtzman, M. Forbes, R. L. Bras, and Y. Choi, "Clipscore: A reference-fiee evaluation metric image captioning." " 2022. [Oaline]. Available: https://arii..orr/bbb//00......
[38] L. Mahon and M. Lapata, "Screenwriter: Automatic screenplay generation and movie summarisation," 2024. [Online]. Available: hmtp://aariv.orr/abs244000 19809
[39] D. W. Lee, C. Ahuja, P. Liang. Natu, and L.-P. Morency, "Multimodal lecture
presentations dataset: Understanding multimodality in educational slides," " 2022. [Online]. Available: https//aaxv.orrgaah/228..00000
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have conducted extensive experiments to show the effectiveness of our proposed model. Please refer to Section 5 for more details.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contribu- tions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings. It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of our propsoed metod in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations these assumptions (c.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA].
Justification:
We do not have theoretical results that require proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
14

---

All assumptions should be clearly stated or referenced in the statement of any theorems. The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We include implementation details in Appendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., large language model), then there should either be a way to access this model for reproducing the results or way to reproduce the model (c.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but should be possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We include code, pretrained models, and data processing script on our demo page supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https: : ctt.ippscccppbbicc////cc
guides/ Sowpe//oeeSebmeeeeeeee sai onPolicn looollyyy liyy for more details.
15

---

While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ //nips.c. ccs.cc/puuu wtcc/pubic//aee wttgg/dee///eeueaigeeeeeeeeeeee swiiisssioplcyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why. At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparame- ters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We include implementation details in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report 95% confidence interval for our subjective test.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main
claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to
library function, bootstrap, etc.)
The assumptions made should be given (c.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error the mean.
It OK to report 1-sigma error bars, but one should state The authors should preferably report 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
16

---

Answer: [Yes],
Justification:
We include the information on the GPU resources Appendix E
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or
cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experi- mental runs as well as estimate total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (c.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:////eerrrp...pccbb//////eae cctppppcttc//ttiicccccceeeeeeee
Question:
Answer:
[Yes]
Justification: We will not release raw copyright materials but release data preprocessing scripts processed data instead.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there a special consideration due laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss broader impacts of our work in Section 7.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploy- ment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications. the authors should point out. For example. it legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses addition to attacks, mecha- nisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
17

---

Answer: [Yes]
Justification: We discuss the risks in Section We will restrict the usage of the pretrained models and datasets for research purpose only.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer:
[Yes]
Justification: The data publicly available YouTube vidcos. We will follow the common practice in the machine learning community and release only the YouTube IDs for these videos.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi www.swwweeeee....eee com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: As shown in our introduction, we will release our code and pretrained model on our demo page.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [Yes]
Justification: We include the details of our survey in Appendix K.
18

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Including this information the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should paid at least the minimum wage in the country the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements your country or institution) were obtained?
Answer: [NA]
Justification: Our research does not require IRB approvals per institutional policies.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non- standard component of the core methods this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: Our proposed method include finetuning an open-source LLM (LLaMA) to generate the script. A pretrained, commercial LLMs (GPT-4o) is used as baseline model in our experiments.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non- non-standard components.
Please refer to our LLM policy what should or should not be described.
19

---

A Comprehensive View of REGen System
Figure 4: Comprehensive view of our REGen System
B
GPT-4o Prompts
B.1
GPT-4o-DQ
You are helpful assistant. Generate an engaging introduction of the content with quotation based on the following input
Input:
f"[Ten sentences chunk summary)"
Output:
B.2 LLaMA-DQ
You helpful assistant. Generate engaging introduction of the content with quotation based the following input
Input:
f"(Ten sentences chunk summary)"
Output:
B.3 GPT-40-SP-DQ
A. You are helpful assistant. Generate an engaging introduction of the content with quotation based on the following input.
"Generate a concise version of the following screenplay segment by preserving its plain text format "where each line starts with either 'Narration:' or 'SpeakerlI:: [Speaker]:'. "Limmt the summary to approximately tokens."
B.4 LLaMA Finetuning Template for REGen-DQ
Instruction:
"You are helpful assistant. Generate engaging introduction of the content with quotation based on the following input."
Input:
""TTen sentences chunk summary)"
Output:
Tkk <SDQ>, VI..... Un. <EDQ>, oktl....
(6)
20

---

B.5
LLaMA Finetuning Template for REGen-IDQ
Instruction:
"You are helpful assistant. Generate an engaging introduction of the content with quotation based on the following input."
Input:
f" (Ten sentences chunk summary)"
Output:
TK, SUUTEE>,*+1,,
(7)
B.6 G-Eval
"Assess how naturally the text flows in a story-like manner, evaluating grammatical correctness, syntactic variety, and seamless transitions that enhance narrative coherence."
B.7
Extraction-thhe--mmothin
System:
You are creative storyteller and writing coach.
User: Process the following input. It contains interview chunks, each prefixed with "Interview:".
Treat each "Interview: segment as a single, indivisible unit and do not split or merge them.
Input: f [Selected Interviews)"
Your task: 1. Generate a concise, engaging story that preserves the exact wording of each interview chunk. 2. Clearly indicate where the narrative each interview chunk is inserted by using [Interview]. Now, please craft the story with these guidelines. .*.
C Robustness of Speaker Diarization
Our system demonstrates robust speaker diarization performance on our documentary dataset, achiev- ing a narrator-prediction Fl score 71.6%, start-time correctness 93.5%, end-time correctness of 94.9% and transcription accuracy of 94.9%. We thank the reviewer for pointing out the dependency of our framework on diarization quality. Although we acknowledge that alternative methods may further improve segment extraction other datasets, injecting controlled noise: we introduced varying proportions of segments labeled as "narrator" into the set of quotable interview segments. Using fixed random seed, then measured recall under different noise levels and reported in the table below; as expected, recall declines slightly as more incorrect segments are added.
evaluated framework's 's robustness by
Table 6: Robustness of Speaker Diarization on Downstream Task
Model
Level of Accuracy
Recall @ (%)
Recall@5 (%)
Recall 10 (%)
Ours
Build upon pretrained model Build upon pretrained model, intentionally add 10% narration Build upon pretrained model, intentionally add 20% narration
3.33
17.50
31.67
10% extra error
1.67
14.17
29.17
20% extra
0.83
12.50
27.50
D Dataset
Generating Script with Speaker Annotation In order generate scripts with timestamp and speaker labeled, following [38], we use WhisperX [16] for speaker diarization, which generates script with start, end timestamps, speaker IDs well as the transcribed text. We assume that narration typically dominates and corresponds to the longest audio track; therefore, we label the speaker with the longest transcript as the narrator. An example of our processed data is available on our demo page.' We constructed 941 paired teaser-main documentary screenplays from DocumentaryNet [1 1]. and we include an additional 157 teaser-only samples. We estimate that in our training and validation sets, 766 out of 1,098 teaser part contain inserted vidcos, with an average of 1.8 inserts per example.
21

---

Validating Automatic Speech Recognition and Narrator Identification Results We recruited four people evaluate the annotation quality of our dataset. We asked them assess if the automatically detected narrator was correct, if the start and end times of an interview segment were accurate, and if the transcription achieved over 95% accuracy. We provide an example of our annotation process on our demo page.' In the teaser part of our test set, we report the narrator prediction Fl score, audio-track start-time correctness, end-time correctness, and transcription accuracy. The narrator prediction Fl score is 71.6%. Start-time correctness is 93.5%. End-time correctness is 94.9%. Transcription accuracy is 94.9%. We present 128 interview segments with correct start times, end times, and transcriptions. In addition, 40 of 49 teasers in test set include second speaker, and on average each test video has 3.02 inserted clips. In our main documentary part of our test set, we also report the narrator prediction Fl score, audio-track start-time conrectness, end- time correctness, and transcription accuracy. The narrator prediction F1 score is 88.7%. Start-time correctness is 90.8%. End-time correctness is 91.7%. Transcription accuracy is 96.3%. We present 2,472 interview segments with human-validated start times, end times, and transcriptions.
E
Implementation Details
All experiments conducted single NVIDIA A100 GPU with batch size of 64. We reserve 5 of the dataset (49 documentaries) for final testing and allocate 10 of the remaining samples for validation. The learning rate is set to x 10 5, and training terminates when either the generation or the retrieval validation loss does not decrease within 30 consecutive epochs. We use Adam optimizer for training.
E.1 Script Generation
In script generation stage, we remove any teaser outputs containing non-English tokens. Consequently, we fine-tune LLaMA [30] on 839 paired examples.
E.2 Quote Retriever
In the quote retriever stage of pipeline, we construct 47,883 training samples to jointly fine- tune the all-mpnet-base maximum input length of 256 tokens. This extended context window enables the retriever to capture long-range dependencies-such as narrative shifts and cross-segment entity references in long main documentaries- that often exceed 128 tokens. For the documentary teaser-generation task (Section 4.4) and the accompanying subjective evaluation (Section 4.3), we cap BART's input window at 128 tokens. This choice is informed by our analysis Table 2, which shows that the typical teaser length is 121 tokens; thus, a 128-token limit tightly encompasses almost all world examples. Moreover, reducing the context window cuts inference time and memory usage by nearly half during large-scale A/B studies, while ensuring that all generated outputs are compared under identical input constraints. our GroupSampler module, if the number of distinct documentaries in batch is less than the batch size, additional negatives are sampled from other documentaries to encourage fine-grained discrimination. When multiple interview segments occur between two consecutive narration chunks, we select the nearest preceding and succeeding narration scripts to construct each training sample.
sentence embedder [33] and the Facebook BART-base model [32] with
E.3 Documentary Teaser Generation
When constructing teasers with our proposed method, prevent repetition, we maintain sliding window over the last three selected clips and disallow any duplicate segment within that window.
E.4 GroupSampler
If the number of distinct documentaries in batch is less than the batch size, additional negatives are sampled from other documentaries to encourage fine- fine-grained discrimination.
22

---

Objective Evaluation Details
In Table 4, for each narration we may select multiple intervals (frames) to accompany it. We compute the CLIPScore[37] between the narration script and each frame, then the highest CLIPScore among them as the score for that narration. For each interview segment, we similarly consider the highest CLIPScore within its interval. We report CLIPS-I as the CLIPScore measuring audiovisual alignment of interview segments and CLIPS-N as the CLIPScore measuring audiovisual alignment of narration segments.
We observe that certain interview segments in the main documentary are post- post-processed (e.g., trimmed reshoot) before being incorporated into teasers. We treat these edited segments distinct items in our retriever stage. To ensure a fair comparison with fully extractive baselines, we remove those interviews that cannot be exactly found in main documentary when compute Fl, Repetitivenss, CLIPScore and VTGHLS when evaluating teaser generation task in Table 4. There is no difference in SCR because interview is usually considered
In Table 3, we also notice that some of the teaser are fully narrations and those are removed for retriever stage evaluation. Moreover, conduct the experiments with 3 random seeds and report the standard deviation for random selection and GPT-4o infilling.
In Table 2, we run G-Eval with three random seeds and report the mean score and standard deviation.
G Script Generation Evaluation on Direct Quote from Long Contents
Table 7: Comparison in the capability direct quotation
Summarized input script
Semantic similarity with the nearest neighhor
Model
Quotation encoding
Overlap ratio
GPT-4o-DD DQ
Quotation marks
0.52
0.07
GPT-4o-DQ-N
Quotation marks Screenplay-Iie
0.50
0.13
GPT-4o-SP-DQ
0.69
0.17
I.LMA-DQ
Quotation marks
0.00
REGen-DQ
<SDQ> <EOQ>
0.45
0.07
In Table 7, we report the semantic similarity between segments predicted quotations-that is. LLM outputs enclosed by quotation markers-and and their nearest neighbors in our interview database, well as the Overlap Ratio defined in Section 4.2. When we provide GPT-4o [29] with the full documentary transcript, the overlap ratio is only 0.13. To accommodate extremely long inputs, we feed GPT-4o a summarized version of the main documentary: this further lowers the overlap to 0.07. The screenplay-lice quotation encoding with GPT-4o raises the token overlap ratio to 0.17, but this remains inadequate. While vanilla LLaMA [30] cannot produce meaningful quotation markers, fine-tuning it with <SOQ> and <EOQ> increases the overlap ratio from 0 to 0.07. However, we note that this overlap ratio still lower than that of the GPT-4o-bases model. We expect better performance scale up the dataset.
23

---

H
Additional Teaser Generation Evaluation
Table 8: Objective Evaluation for Teaser Generation Task
Dur (sec)
Interview Ratio (%)
F1 (%)
SCR
REP
Model
(%)
(%)
VTGHLS
CLIPS-1
CLIPS-N
Random extraction
101
56 20
1.10
20.71
0.41
0.83
0.55
0.62
ETS
142
34 + 16
1.92
13.65
4.49
1.06
0.64
0.60
A2Summ [4]
73
42 - 25
1.70
14,20
1.73
0.89
0,56
0.63
TeaserGen [11]
155
1.64
22.61
21.38
0.80
0.67
GPT-4o-DQ
151
42 42
1.56
16.55
20.75
0.58
0.42
GPT-4o-SP-DQ
619
61 17
2.07
12.38
18.33
1.02
0.62
0.62
GPT-4o-SP-TV
673
64 + 17
1.61
11.29
41.46
1,02
0.64
0,62
REGen-IDQ (random)
82
32 + 33
1.34
20.40
7.50
1.03
0.41
0.57
REGen-DQ
95
37 1: 26
1.45
19.13
10.35
1,05
0.48
0.57
REGen-IDQ-T
77
35
+ 31
1.89
19.79
10.02
1,03
0,41
0.57
REGen-IDQ-TT IDQ-TV
81
35
1.90
19.86
9.70
1.02
0.39
0.57
Ground truth
76
54 37
69.00
27.60
7.86
<0.98
0.43
0.57
Following each the upply pixel-by-pixee comparison the 20 candidates; however, this strict matching may fail to identily identical frames due to the low Irume ratl..
top similar frames from main content using CLIP embeddings.
When comparing GPT-4o-SP-DO with GPT-4o-SP-TV, and GPT-4o-SP-TV with REGen-IDQ-TV, we observe a significant drop in reaser-generaticn performance for GPT-4o-SP-TV in Table 8, indicating that the surrounding narration produced by our fine-tuned script aids the our retrieval stage.
I GPTT-o Infilling Analysis
Mediaemmee was ooce sejer crosonddu bout the undeent worl. Today. has beoome berrer aryting dureet sokay's distas cultares their droms Mediemooo
seperxing Earope
Croon Trath
Tasisaas
allodd food for
antraid? God Join
The Medtemmen was ooce rejer crosenad hort ancient world Thday, tas becrme herneer separning Europe
GPT-4
anytiig left ahoreef sokay' cultures the caimald? drnere Molizmmonn
Now
The toc that danald replace "iuade' expliclll provided be query Hloweerr lona on the contee, some like the "mrae"" plackelder axe mast to filal with planao or acsicsoo that comnect sarmaive. in axgentin for coeld replace "ouaake" plachelles
Cupp-
Predicuel
their oct take antrule
journsy pouilty famar
"ausic" sefore this ouidd somuthing like "Thyy meet local fermer who stune his challenges.
The toe the dooid replace "craado" is "Thetr first dop Merocze.
Predicted: Yeu, mother shom Casibiinoo, and my Saher foon teen Tiightr.
Secoad
Figure 5: GPT-4o Infilling Analysis
In Fig. 5, we observe GPT-4o can only produce generalized insertions, which are not ideal as query for retrieving a quotable segment. In contrast, our proposed model is trained specifically to learn to generate insertions from the context that includes detailed information that can be used as effective queries. Moreover, as the quotable segments (oftentimes an interview in our case) might be spoken by a different person other than the narrator, our proposed method can better capture the nuanced style of these quotable segments. We show an example our demo page. GPT-4o's output is sometimes unreliable, making postprocessing challenging. While we believe GPT-4o could be improved with more contextual guidance in future work, second-stage retriever remains effective.

---

J Quote Distribution
Table Effect of the position of the <SUM> token
Model
QCR (%)
QDI(mean + std) (%)
Mean Absolute Difference (%)
Random Extraction
98
11.71 + 4.09
9.27 = 3.66
ETS!
96
1.96 0.49
1,71 = 1.28
A2Summ[4] [4]
96
3.98 + 2.41
2.02 + 1.77
TeascrGen[1] [1]]
GPT-4o-DQ
98
4.02 1: 4.09
3.49 + 2.98
GPT-4o-SP-DQ
100
22.33 + 8.18
19.88 7.46
REGen-DQ
76
2.31 + 2.12
2.18 + 1.85
REGen-IDQ
67
1.98 + 2.19
2.02 = 1.93
Oracle
76
2.45 2.12
Ground Truth
82
3.02 + 2.54
ETS(cxtractive- sewtiiv-then-mmoooeee wwwpeiivveeeee--mmotninnng first extracts two interview segments, then uses GPT-4o to smooth them into coherent output. To ensure fair comparison in downstream teaser generation. ETS is given prior knowledge that teasers typically contain two interview segments (based on the oracle average). Its closeness to the ground truth effectively biasing it toward the oracle.
from this privileged information,
Oracle: In our setting, the outputs generated by our dataset preprocessing pipeline, which includes pretrained models, are treated as oracle annotations representing an upper bound.
O suruuu ond ahe Axrte Decumensr
Disiitaiion of Sthecece Concared Craud
do
(a) The number of quotes per documentary, where the x-axis represents the number of quotes and the y-axis shows the count of documentaries
(b) Mean difference(can be negative) in quote count per documentary between the REGen system and the ground truth, with the x-axis the mean difference and y-axis as number of documentaries
Figure 6: Detailed Distribution of Quotations in Our Generated Scripts
We present detailed distribution of quotations in generated scripts in Table 9 andFig. From the table, we can see that REGen-IDO is the closest to the ground truth distribution and is comparable A2Summ[11 1]. In addition, we would like point out that the position of the quotation marks and whether the retrieved content fits the context are also important in addition to matching the overall distribution.
K Survey Questions
K.1 Demographic Questions
How often do you watch documentary?
Do you have any video editing experience?
On scale of 1-5, where = Beginner and 5 Native/Bilingual, how would you rate your English proficiency?
25

---

K.2 Interview Insertion Evaluation
Please indicate your level of agreement with the following statement: "The inserted interview integrates with the narration, effectively supports the surrounding claim, and maintains a natural flow."
K.3
Documentary Teaser Generation
To what extent do you feel that the sample maintains coherence and smooth flow, ensuring that each segment transitions logically and the overall experience feels seamless?
Coherence:
Alignment: To what extent do you feel that the narration and video match and work well together, making the overall presentation clear and easy to follow?
Realness: How well do you feel this sample meets your expectations as a teaser for documentary in general?
Interview Effectiveness: Please indicate your level of agreement with the following statement:
"The inserted interview integrates with the narration, effectively supports the surrounding claim, and maintains a natural flow."
K.4
Generalizability Evaluation Prompts
Lecture Videos: Which sample would you prefer as a teaser for lecture videos? News Videos: Which sample would you prefer as a teaser for news videos?
K.5 Generalizability Evaluation Results
To assess the generalizability of framework, we randomly select 10 lecture videos from the Multimodal Lecture Presentations Dataset [39], covering subjects such as psychology, machine learning, dentistry, and biology, and also include full-episode news broadcasts from NBC News. We then conduct an A/B study comparing our method against TeaserGen [11], asking participants which teaser they would prefer for each lecture video or news broadcast. We perform side- by-side evaluation against TeaserGen [11] on both lecture and news videos. In the lecture domain, participants prefer TeaserGen 62% of the time versus 38% for model, indicating clear preference for abstractive summaries there. For news videos, the split is 55% in favor of TeaserGen and 45 % for our approach- smaller gap that does reach significance.
Table 10: Subjective evaluation results of Generated Teaser
Dataset
Model
Preference (%)
News News
REGen-IDQ-TV
TeaserGen
Lecture videos
REGen-IDQ-TT
Lecture videos
TeaserGen
62
Ablation Study
L.1 Effects of Max Context Window Length
compare different context window length in Table 11. We find no significant difference when we set different context window the teasers usually short.
26

---

Table 11; Effects of maximum context window length
Model
Max context tokens
Recall (%)
Recall (%)
Recall @ 10 (%)
GPT-4o infilling
128
2.78
0.48
13.89
1.27
22.50 = 1.44
GPT-4o infilling
256
3.33
10.83
13.33 + 0.83
22.78 + 1.27
QuoteRetriever-I
128
5.00
15.00
23.33
T
256
4.17
16.67
22.50
QuoteRetriever-I TV
128
5.00
17.50
30.00
QuoteRetriever-IV
256
5.00
17.50
27.50
L.2 Effects of Alpha in Balancing Loss
We compare in Table 12 the effects of the weights of the generation loss and retrieval loss in the loss function Section 3.2. We find that when a = 1, meaning that generation loss and retrieval loss equally weighted, our model achieves its highest performance. In the following table, we set include 30% interviews a batch as hard negative samples. Here we set max length of tokens for each sample being 256.
Table 12: Effects of a in the loss function Section 3.2
Recall @1 (%)
Recall @5 (%)
Recall 10 (%)
0
0.00
6.67
14.17
0.5
2.50
15.83
29.17
1
2.50
20.00
32.50
2 5.00
18.33
30.83
L.3 Effects of Group Sampler
To examine whether treating interviews from the same documentary hard negatives improves retriever training, we conduct two experiments. the first, we treat all interviews within the same documentary as hard negative samples. In the second, negatives and omit the group sampler. We find that the 30 % group-sampler configuration yields the highest recall @5 and recall @ 10, while omitting the group sampler achieves the highest recall @ 1.
treat only 30 % of those interviews hard
Table 13: Effect of negative sampler construction and loss function
Model
Loss
GroupSampler
Recall@ (%)
Recall@ (%)
Recall (%)
Model-Visual
Contrastive
5,00
17.50
27.50
Model-Visuna
L2
1.67
6.67
13.33
Model-Visual
Contrastive
30%
2.50
20.00
32.50
Model-Visunl
Contrastive
7.50
17.50
31.67
L.4 Effeet of Loss Function
We also use L2 loss to find the closest embedding during the retrieval stage (see Table 13). We find that leveraging contrastive loss increases our model's performance, yielding higher recall. This is likely because contrastive loss can better differentiate embeddings that are close in the embedding space. Here we set max length of tokens for each sample being 256.
L.5 Efffec of Position of Retrieval token
As shown Table 14, appending the special token <SUM> the end of the decoder output before retrieval improves recall. Here set max length of tokens for each sample being 256.
27

---

Table 14: Effect of the position of the <SUM> token
Position
@ecall (%)
Recall@5 (%)
Recall 10 (%)
Start 1.67
10.00
25.83
End 2.50
20.00
32.50
M Comparison within REGen System
We compare the models in REGen system with different variants. We find that REGen-DQ achieves the highest ROUGE score and the most realistic quote distribution, as indicated by the quote coverage rate and quotation density index, both closest to the ground truth. The G-Eval scores for REGen- IQQ-T and REGen-IDQ-TV are closer to the ground truth than REGen-DQ, indicating that they produce more coherent, story-like scripts under automatic LLM evaluation. In Table 5, our subjective evaluation further indicates that our proposed models, REGen-IDQ-TT and REGen-IDQ-TY receive higher ratings for interview effectiveness compared with REGen-IDQ (random), indicating the effectiveness of our proposed retriever. In Table 4, we present the effects of different retriever methods in the documentary saaser-generation task, we find that REGen-IDQ-TV achieves the highest Fl score among models in REGen system. In Table our subjective evaluation of teaser generation shows that teasers generated by REGen-DQ yield higher mnterview-efeectiveesss scores than those by REGen-IDQ-TT indicating that fine-tuning to enable direct quotations can increase the coherence and supportiveness of inserted interviews.
28


Reviews:
[{"review_id": "tOcbG9tDzE", "reviewer": "Reviewer_yMCw", "summary": "This paper introduces REGen, a novel framework for generating short promotional videos (e.g., teasers) from long-form videos like documentaries. The core problem it addresses is the inability of existing methods to create a coherent abstractive narrative while also embedding (\"quoting\") verbatim clips from the original source to ground the story in facts.\n\nREGen tackles this with a two-stage, hybrid approach with Script Generation followed by Quotation Retrieval.\n\nThe final output is a short video that combines a synthesized narrative with directly extracted clips. The authors demonstrate the effectiveness of this system on the DocumentaryNet dataset, showing through comprehensive objective and subjective evaluations that REGen outperforms purely extractive and abstractive baselines in generating teasers that are coherent, factually grounded, and realistic.", "questions": "I have a few questions on the empirical analysis below:\n\n1. The *QuoteRetriever-TV* model fuses text and visual features. However, the objective results in Table 3 show that it does not significantly outperform the text-only *QuoteRetriever-T* on recall metrics. Could you elaborate on the role of the visual modality? Is it more important for qualitative aspects not captured by recall, or does its benefit depend on the specific type of content being retrieved?\n\n2. The results show that prompting a powerful LLM like GPT-4o to fill in the quote content and then using that text for retrieval performs poorly. This is an interesting finding. Why do you think this approach fails? Does the LLM generate text that is too generic or semantically divergent from the actual interview clips, leading to poor retrieval?\n\n3. The proposed pipeline involves multiple models, including calls to GPT-4o for some baselines and processing steps. Could you comment on the computational cost and scalability of the REGen system? Is it feasible for processing very large video archives, or is it better suited for single-document editing?", "limitations": "I have covered potential empirical limitations in questions and weaknesses section above.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "*Strengths*\n\n1. The paper introduces a novel and highly relevant task: enabling generative models to \"quote\" multimodal content. This hybrid abstractive-extractive approach is a significant conceptual advance over existing video summarization techniques. It directly addresses a key weakness of purely abstractive methods (lack of factual grounding) and extractive methods (lack of narrative coherence), proposing an elegant and practical solution. The problem formulation is a strong contribution in itself\n\n2. The REGen system is technically sound and well-engineered. The two-stage pipeline is a logical and effective way to decompose this complex task. The design of the Quote Retriever, which is jointly trained to perform masked infilling and retrieval, is particularly clever. The paper is supported by a thorough set of experiments with strong baselines.\n\n3. The paper is exceptionally well-written and easy to follow. The motivation is clear, the methodology is explained in detail, and the figures (especially the system overview) are highly effective at illustrating the proposed framework.\n\n*Weakness*\n\n1. The system's performance is dependent on the quality of the initial speaker diarization used to identify quotable interview segments. The authors are transparent about the accuracy of this step, but errors here could negatively impact the entire pipeline by creating a flawed pool of candidate quotes. A brief analysis of the system's sensitivity to these upstream errors would be beneficial.\n\n2. The method is developed and primarily evaluated on documentaries, a genre where the distinction between a narrator and an interviewee is often clear. The appendix includes a generalizability study on news and lecture videos, which honestly reports that users preferred a purely abstractive method for lectures. This suggests the \"quoting\" paradigm, as currently framed, may be best suited for specific content types.\n\n3. The end-to-end system is a multi-stage pipeline involving several distinct models (ASR, diarization, LLM for scripts, retriever model). While this modularity is effective, it introduces complexity. Future work could explore more integrated, end-to-end models, although this would be a significant research challenge.", "was_revised": true, "final_justification": "My concerns are addressed and I shall maintain the score.", "rebuttal": "We thank the reviewer for their thoughtful feedback and are encouraged by their recognition of our task’s novelty, the elegance and rigor of our hybrid abstractive‑extractive framework, and the clarity of our presentation. We hope this work meaningfully advances grounded and coherent long-to-short video editing. We answer the reviewer’s comments below.\n\n**[Weakness 1]**:\n\nOur system demonstrates robust speaker diarization performance on our documentary dataset, achieving a narrator‑prediction F1 score of 71.6%, start‑time correctness of 93.5%, end‑time correctness of 94.9%, and transcription accuracy of 94.9%. We thank the reviewer for pointing out the dependency of our framework on diarization quality. Although we acknowledge that alternative methods may further improve segment extraction on other datasets, **we evaluated our framework’s robustness** by injecting controlled noise: we introduced varying proportions of segments labeled as “narrator” into the set of quotable interview segments. **Using a fixed random seed, we then measured recall under different noise levels and reported in the table below; as expected, recall declines slightly as more incorrect segments are added.**\n\n| Model                              | Level of Accuracy                                                   | Recall @1 (%) | Recall @5 (%) | Recall @10 (%) |\n|----------------------------|--------------------------------------------------------|--------------:|--------------:|---------------:|\n| Ours                               | Build upon pretrained model                                         |          3.33 |         17.50 |          31.67 |\n| 10 % extra error   | Build upon pretrained model, intentionally add 10 % narration       |          1.67 |         14.17 |          29.17 |\n| 20 % extra error      | Build upon pretrained model, intentionally add 20 % narration       |          0.83 |         12.50 |          27.50 |\n\n**[Weakness 2]**:\n\nOne possible reason is the preprocessing step used for lecture videos. As we mentioned in the limitations section, the current preprocessing method results in overly fragmented outputs in lecture videos. **We anticipate better performance by segmenting lecture videos based on slide transitions—for example, using OCR to detect transition points or leveraging built-in slide metadata that may accompany lecture recordings.**\n\n**[Weakness 3]**:\n\nOur pipeline begins by **inputing raw videos**: we first apply ASR and speaker diarization to convert audio into structured transcripts. Drawing on our video‑editing experience, we then use an LLM to infill and draft the script—essentially outlining where interview clips or other footage should be inserted—and follow up with a learned retriever to find the most appropriate video segments for each insertion placeholder. **This modular design not only mirrors real‑world editing workflows but also maximizes data efficiency by allowing each component to be trained with specialized and smaller datasets.**\n\n**[Question 1]**:\n\nIn quotable interview segments, **the visual modality often provides background context**. For instance, an interview filmed outdoors differs from one conducted indoors. We observe that in current quotable segments—primarily interviews—the text modality carries more information than the visual modality. However, **our framework is designed to generalize to broader cases** where the visual modality contributes more substantially to retrieval. For example, when retrieving natural view videos without narration, visual information becomes important for accurate retrieval.\n\n**[Question 2]**:\n\n**We observe GPT-4o can only produce generalized insertions, which are not ideal as a query for retrieving a quotable segment**. In contrast, our proposed model is trained specifically to learn to generate insertions from the context that includes detailed information that can be used as effective queries. Moreover, as the quotable segments (oftentimes an interview in our case) might be spoken by a different person other than the narrator, our proposed method can better capture the nuanced style of these quotable segments. We show an example on our demo page. GPT-4o’s output is sometimes unreliable, making postprocessing challenging. While we believe GPT-4o could be improved with more contextual guidance in future work, our second-stage retriever remains effective.\n\n**[Question 3]**:\n\n**We run speaker diarization on a single A6000 GPU, and fine‑tune LLaMA and train the retriever (batch size being  64) on a single A100 GPU.  The entire system was trained in under one day**. To build the candidate pool, we first separate each video’s audio into music, dialogue, and sound effects—providing a clean dialogue track that can enhance WhisperX’s accuracy. WhisperX then processes this track to identify speaker turns and determine start‑end timestamps for each quotable segment (the most time‑consuming step, yet still 50× faster than real time). Next, we extract textual and visual features for every interview segment using our feature extractor.\n\nOnce the candidate pool is assembled, there’s no need to rerun diarization or feature extraction. Retrieval over L quotes and N candidates incurs O(L · N) time, so computation scales linearly with archive size and can become challenging for very large video collections. In our human‑annotated test set, each video contains on average three quotes—equating to roughly one second of retrieval time per video when inserting three video segments."}, {"review_id": "WYrZyQRDXN", "reviewer": "Reviewer_QQGE", "summary": "This work presents a new multimodal retrieval-embedded generation framework, named REGen, for editing long videos into shorts. It combines abstractive and extractive methods via a two-stage pipeline: 1) generating a story script with quote placeholders using fine-tuned LLaMA, and 2) retrieving supporting video clips from the input to fill these placeholders. Experiments on documentary teaser generation show REGen effectively inserts video quotes while maintaining narrative coherence, outperforming baselines.", "questions": "Please refer to the weaknesses for detailed concerns.\n\n**Additional Questions:**\n1. For videos without clear speaker segmentation (e.g., lectures), could REGen be adapted to detect \"quotable moments\" based on visual/audio cues (e.g., slide transitions, emphasis in speech) rather than relying solely on diarization?\n2. How does the system ensure temporal coherence when inserting retrieved clips? For instance, if a selected interview clip references an event that has not yet been introduced in the narrative, could this lead to logical inconsistencies in the final output?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. Effective framework for creating documentary teasers and has promising potential for applications in education and other domains.\n2. The combination of extractive and abstractive summarization methods is inspiring.\n\n\n**Weaknesses:**\n1. The description of the methodology is overly dense and lacks emphasis on the novel innovations.\n2. The choice of using fixed 10 chunks is not empirically justified or supported by ablation studies, leaving the rationale unclear.\n3. Although the paper emphasizes maintaining narrative coherence while integrating factual quotes, the architecture relies on a sequential pipeline rather than joint optimization. This limitation is reflected in Table 6, where the overlap ratio results fail to fully support the claim of narrative coherence.", "was_revised": true, "final_justification": "The authors' response has successfully addressed my concerns. I maintain my initial rating with higher confidence.", "rebuttal": "We thank the reviewer for their constructive feedback. We’re encouraged that the review find our documentary‑teaser framework effective and our hybrid extractive–abstractive approach inspiring. We answer the reviewer’s comments below.\n\n**[Weakness 1]**:\n\nWe propose a new retrieval-embedded generation framework that enables an LLM to quote multimodal resources while maintaining a coherent narrative. Our framework supports long-to-short video editing by generating short videos that feature a coherent narrative with embedded video insertions extracted from a long input video.\n\n**[Weakness 2]**:\n\nThank you for pointing that out. Following TeaserGen [1], we split the transcribed text into smaller segments to generalize more effectively to extremely long videos.\n\n**[Weakness 3]**: \n\nWe would like to clarify that the overlap ratio in Table 6 is used to measure the overlap between the content quoted by the large language model and the ground truth interviews. It is defined as {#overlap words} / {#words in matched interviews}. This metric is not used to evaluate narrative coherence. When training our retriever, we use joint optimization with a multitask loss:\n L = $L_{gen}$ + α · $L_{ret}$. Here,  $L_{gen}$ encourages the model to attend to the context window around the masked segment, while  $L_{ret}$  helps differentiate relevant quotable segments from other candidates.\n\n**[Question 1]**:\n\nSpeaker diarization is effective for content such as news and documentaries, where multiple speakers are involved. However, in lecture videos, a key constraint is that there is usually only one speaker—the instructor—making speaker diarization less applicable. To address it, we detect slide transitions using OCR to compare frames and identify the start and end timestamps of each slide. This allows us to segment long lectures into smaller chunks. Importantly, these adjustments only affect the data preprocessing step. Once quotable video segments are extracted, our two-stage method remains effective.\n\n**[Question 2]**:\n\nYes, it could. We intend to alleviate this issue in our script generation stage and retrieval stage. In our script‑generation stage, we leverage an LLM to preserve temporal information. In our retrieval stage, the query embedding is generated from the surrounding context, thereby retaining temporal cues. In the future, we can incorporate these contextual embeddings into the representations of quotable interview segments in our candidate pool to ensure temporal coherence and maintain logical consistency.\n\n**Reference**:\n\n[1] Xu et al., TeaserGen: Generating Teasers for Long Documentaries, ICLR 2025."}, {"review_id": "xxwTTqQedz", "reviewer": "Reviewer_tTNr", "summary": "This paper proposes a novel approach for generating short videos that combine a coherent narrative with embedded video excerpts from longer videos. While extractive summarization methods often fail to produce cohesive stories, existing abstractive methods cannot directly incorporate video clips from the source material. To address this, the authors introduce a retrieval-embedded generation framework (REGen). REGen first uses a fine-tuned large language model to generate a narrative script containing placeholders for video quotes, and then employs a retrieval model to select suitable video clips that replace these placeholders, ensuring alignment with the narrative. The method is evaluated on the task of documentary teaser generation, where short interview clips are commonly used to enhance storytelling. Results show that REGen effectively integrates video excerpts while maintaining narrative coherence, and outperforms existing extractive and abstractive approaches in terms of coherence, alignment, and realism, as demonstrated by both objective metrics and subjective user studies.", "questions": "1 Is the computation required by the proposed method expensive?\n\n2 Have you ever had some ideas to improve the limitations you list?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Stengths: \n\n**1**  The motivation of the paper is clear, that existing abstractive methods cannot directly incorporate video clips from the source material.\n\n**2** The authors conducted extensive experiments, including thorough ablation studies and comparisons with baselines, which convincingly demonstrate the effectiveness of the proposed REGen framework.\n\nWeakness:\n\n**1 ** Although video insertions help improve factual grounding, the method can still mistakenly place quotes in incorrect contexts. Incorporating all quotable materials into the initial script generation could reduce this issue, but this is technically challenging for LLaMA-based models due to their limited context window.\n\n**2 ** The approach relies on accurate video segmentation, such as speaker diarization, which may not be applicable in other domains like lecture recordings.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer and sincerely appreciate your acknowledgement of the strong motivation and significance of our work—also highlighted by reviewer yMCw—as well as the thorough experiments demonstrating our framework’s effectiveness. We address your comments point by point below.\n\n**[Question 1]**:\n\n**We run speaker diarization on a single A6000 GPU, and fine‑tune LLaMA and train the retriever (batch size 64) on a single A100 GPU.  The entire system was trained in under one day**. To build the candidate pool, we first separate each video’s audio into music, dialogue, and sound effects—providing a clean dialogue track that can enhance WhisperX’s accuracy. WhisperX then processes this track to identify speaker turns and determine start‑end timestamps for each quotable segment (the most time‑consuming step, yet still 50× faster than real time). Next, we extract textual and visual features for every interview segment using our feature extractor.\n\nOnce the candidate pool is assembled, there’s no need to rerun diarization or feature extraction. Retrieval over L quotes and N candidates incurs O(L · N) time, so computation scales linearly with archive size and can become challenging for very large video collections. In our human‑annotated test set, each video contains on average three quotes—equating to roughly one second of retrieval time per video when inserting three video segments.\n\n**[Question 2, Weakness 1 and Weakness 2]**:\n\nAs discussed in section 6, we identify two primary constraints. First, the limited context window can be addressed **by adopting a more powerful LLM with a larger context length**; we chose LLaMA for its strong open‑source performance, and our architecture readily accommodates such upgrades. Second, in lecture settings where the instructor’s voice is present (as discussed in the section 6), **we can use OCR‑based frame comparison to detect slide transitions and segment long lectures into smaller chunks**. This variation affects only the preprocessing stage, and our two‑stage method remains effective once quotable segments are extracted. **We hope this work will serve as a foundation for hybrid abstractive–extractive summarization that supports both factual grounding and narrative coherence, with significant potential in educational domains.**"}, {"review_id": "xb8CO3WsxA", "reviewer": "Reviewer_sppt", "summary": "This paper proposed REGen, a multimodal retrieval-embedded generation method for long-to-short video editing, which utilizes a large language model for multimodal resources quoting while maintaining a coherent narrative. The proposed method is evaluated on the task of documentary teaser generation and outperforms existing abstractive and extractive approaches in many evaluation metrics on teaser generation.", "questions": "See weaknesses. I believe that weakness 1 is especially crucial to address for theoretically supporting the evaluated performance. I would increase the rating if the reply is convincing.", "limitations": "The authors adequately addressed the limitations and potential negative societal impact of their work.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The overall idea is interesting, and the task is important for real applications.\n2. The experiment result is solid, both objective and subjective evaluations are provided on quoting short interview clips within a coherent narrative.\n3. Convincing visualization examples are provided to prove the effectiveness of the proposed method.\n\nWeaknesses:\n1. There is some controversy over the rationality of some evaluation metrics, for example, QCR and QDI may be close to ground truth due to the methods used that generate scripts with a similar quote distribution to the ground truth scripts. But it does not affirmatively mean that the closer QCR and QDI are to the ground truth, the better the method would be for generating a similar quote distribution to the ground truth. On the contrary, the paper should use the average absolute difference (of number of quotes inserted per documentary, or proportion of test videos in which at least one quotation is correctly inserted) between the method and the ground truth, to demonstrate the performance difference for generating scripts with a similar quote distribution to the ground truth scripts.\n2. Recall of quote retrieval methods seems relatively low, but this should be considered as a core component in the framework of this article and deserves further analysis and improvement.\n3. The complete framework of Figure 1 lacks some innovative method designs, which the author should incorporate with the method section of the article.", "was_revised": true, "final_justification": "The additional experiment solving weakness 1 is encouraged to have a deeper analysis in the final paper, such as adding a comparison on the newly provided Quote Distribution Plot between baselines and the proposed method.\n\nMeanwhile, the main issues from the weaknesses of my concern have been solved. I will raise my initial rating to 4.", "rebuttal": "We thank the reviewer for appreciating the significance of our proposed task, as also noted by reviewers yMCw and QQGE. We also appreciate your encouraging feedback on our extensive experiments, which reviewer tTNr similarly highlighted.\n\n**[Weakness 1]**:\n\n| Model             | QCR (%) | QDI (mean ± std)  | Mean Absolute Difference |\n|-------------------|---------|-------------------|--------------------------|\n| Random Extraction | 98      | 11.71 ± 4.09      | 9.27 ± 3.66              |\n| ETS \\*           | 96      | 1.96 ± 0.49       | 1.71 ± 1.28              |\n| A2Summ            | 96      | 3.98 ± 2.41       | 2.02 ± 1.77              |\n| TeaserGen         | –       | –                 | –                        |\n| GPT-4o-DQ         | 98      | 4.02 ± 4.09       | 3.49 ± 2.98              |\n| GPT-4o-SP-DQ      | 100     | 22.33 ± 8.18      | 19.88 ± 7.46             |\n| REGen-DQ          | 76      | 2.31 ± 2.12       | 2.18 ± 1.85              |\n| REGen-IDQ         | 67      | 1.98 ± 2.19       | 2.02 ± 1.93              |\n| Oracle\\**          | 76      | 2.45 ± 2.13       | –                        |\n| Ground Truth      | 82      | 3.02 ± 2.54       | –                        |\n\nNote: \n\n\\* ETS (extractive-then-smoothing) first extracts two interview segments, then uses GPT-4o to smooth them into a coherent output. To ensure fair comparison in downstream teaser generation, ETS is given prior knowledge that teasers typically contain two interview segments (based on the oracle average). Its closeness to the ground truth stems from this privileged information, effectively biasing it toward the oracle.\n\n\\** Orcale: In our setting, the outputs generated by our dataset preprocessing pipeline, which includes pretrained models, are treated as oracle annotations representing an upper bound.\n\nFrom the table, we can see that REGen-IDQ is the closest to the ground truth distribution and is comparable to A2Summ[1]. In addition, we would like to point out that the position of the quotation marks and whether the retrieved content fits the context are also important in addition to matching the overall distribution. We verify our performance through subjective evaluations (Please see table 3 and table 5) and present qualitative examples on our demo page (Please see section 3).\n\nIn addition, we provide two comparison plots on our demo page (Section 0): (1) the number of quotes per documentary, where the x-axis represents the number of quotes and the y-axis shows the count of documentaries; (2) the mean difference(can be negative) in quote count per documentary between the REGen system and the ground truth, with the x-axis as the mean difference and the y-axis as the number of documentaries. \n\n**[Weakness 2]**:\n\nThank you for pointing that out! Even though the recall is relatively low, our model is still able to retrieve quotable interviews that support surrounding claims while maintaining a cohesive narrative flow. This is further supported by our subjective evaluation (see Tables 3 and 5), where the effectiveness of interview insertion is significantly higher than the baseline. We also provide qualitative examples on our demo page (see Section 3). We hypothesize that this is because some interviews convey similar ideas—for example, multiple interviewees may be included to support the same claim. As a result, their representations in the latent embedding space are very similar, making it challenging to retrieve the exact original interview segment. However, this does not undermine the role of quotable interviews, which is to support the claims while preserving the overall narrative flow.\n\n**[Weakness 3]**:\n\nWe will expand the quote retrieval and narration-matching sections in our camera-ready version. Specifically, we will elaborate on the quote retriever using the architecture shown in Figure 2. We will also expand the explanation of how narration-flow matching works. Please see our demo page for Figure 1.\n\n**Reference**:\n\n[1]He et al., Align and Attend: Multimodal Summarization with Dual Contrastive Losses, CVPR, 2023"}]

Author rebuttal:

Meta-review: All four reviews [yMCw,QQGE,tTNr,sppt] leaned towards acceptance and ultimately had the same borderline accept rating.

The reviewers appreciated several aspects of the work:

+ The motivation was considered clear [tTNr]
+ The idea was considered interesting [sppt]
+ The task was considered novel [yMCw] and important [yMCw,sppt]
+ The approach was considered a significant advance over existing video summarisation addressing a key weakness [yMCw]; similarly, the framework was considered effective [QQGE] and to have potential application in e.g. education [QQGE]
+ The combination of extractive and abstractive summarisation was appreciated [QQGE]
+ The proposed REGen system was considered technically sound and well engineered [yMCw]
+ The experiments and ablation studies were considered extensive [tTNr]
+ The experiment results were considered solid [sppt] and convincing [tTNr]
+ The visualisation examples were appreciated [sppt]
+ The paper was considered very well written and easy to follow by one reviewer [yMCw]

However, several weaknesses were pointed out, and authors responded to them in the rebuttal stage:

- The dependence of system performance on the quality of the initial speaker diarization was criticised [yMCw] and analysis of sensitivity to such upstream errors was desired [yMCw]; authors provided an experiment involving noise injection.
- Limitation primarily to documentaries was criticised [yMCw], as in appendix results on lectures a purely abstractive method was better; authors provided brief speculation that preprocessing of the lectures may be a cause of the difference.
- The complexity of the multi-stage pipeline was criticised [yMCw], and discussion of the cost and scalability was desired [yMCw,tTNr]; authors provided a brief argument about the pipeline and some details and discussion of the time consumption of the steps.
- Rationality of some evaluation metrics was criticised [sppt]; authors provided some discussion.
- Further analysis of low recall in quote retrieval was desired [sppt]; authors provided some discussion.
- Clarification on the role of the visual modality was desired [yMCw]; authors provided brief discussion.
- The claim of narrative coherence was considered not well supported by the sequential pipeline versus joint optimisation and the overlap ratio results [QQGE]; authors argued the overlap ration did not evaluate narrative coherence, and argued their multitask loss involves joint optimisation.
- Clarification of ensuring temporal coherence was desired with a concern about potential logical inconsistencies [QQGE]; authors replied they "intend to alleviate the issue" in the future via contextual embeddings.
- Analysis of the poor performance of LLM (GPT-4o) based quote filling-in was desired [yMCw]; authors analyse GPT-4o yielded generalised insertions.
- The use of fixed 10 chunks was criticised [QQGE]; authors replied they follow TeaserGen but did not seem to justify the amount of chunks.
- It was criticised the method could still place quotes in incorrect contexts [tTNr]
- Reliance on accurate video segmentation was criticised [tTNr] and its suitability to domains like lecture recordings was questions [tTNr]
- It was questioned whether the method could work by detecting moments instead of relying on diarization; authors provide some discussion on how in lecture videos they can detect slide transitions via OCR.
- Discussion on avoiding the limitations was desired [tTNr]; reviewers provided brief discussion of e.g. using OCR-based slide transition detection
- Discussion of some 'innovative method designs' were desired for the methods section [sppt]; authors stated they would expand some descriptions.
- One reviewer considered the methodological description dense and lacking emphasis on novelty [QQGE]; authors restated their main goal.

Ultimately after the rebuttals reviewers [yMCw,QQGE] considered their questions answered, [tTNr] considered most of the concerns addressed, and [sppt] considered their main issues to have been solved.

Overall, despite some concerns reviewers appreciated the work overall, and although some concerns could be further explored, it seems if the authors incorporate the material from their rebuttals the paper could be in a sufficiently good state to be presented at NeurIPS.
Decision: Accept (poster)