ScreenWriter: Automatic Screenplay Generation and Movie Summarisation

## Abstract

Abstract This paper presents a method for analysing and summarising feature-length movies from the video only. Our method includes a novel, and entirely parameter-free, algorithm for segmenting the input video into scenes based the sequence of visual vectors and the minimum description length principle. It also includes a novel method for solving the challenging problem of determining character names, by using a database of actors‘ faces to cast this as a linear optimization problem in terms of the distance between the facial feature vectors of actors and detected characters. We then describe a modular system for movie summarisation that includes both of these algorithms as components, as well as a speech-to-text model and an image-to-text model. We test this method empirically on the recent Moviesumm dataset, and show that our summaries are superior to a number of comparison models, including prompting only and speech-to-text plus prompting, in terms of Rouge Score, and the recent P rsima for factual evaluation of generated summaries. Given a video as input, our model can produce a pseudo-screenplay, including scene breaks, character names and action descriptions, as well as a plot synopsis/summary. To our knowledge, no previous method is capable of this from a video-only input.

## 1 Results

**Table 1: Caption**
| \toprule | ours | most common | random |
| --- | --- | --- | --- |
| \midruleacc movie-wise | 61.12 | 19.35 | 2.97 |
| acc line-wise | 65.72 | 19.62 | 2.61 |
| \bottomrule |  |  |  |

### 1.1 Scene Detection Results

**Table 2: Caption**
| \toprule | ours | un60 | un75 | un90 | unorc |
| --- | --- | --- | --- | --- | --- |
| \midruleacc | 0.563793 | 0.461071 | 0.441385 | 0.429662 | 0.436109 |
| ari | 0.375179 | 0.277590 | 0.243719 | 0.216477 | 0.227847 |
| nmi | 0.745689 | 0.720267 | 0.711684 | 0.704285 | 0.710762 |
| \bottomrule |  |  |  |  |  |

### 1.2 Name Accuracy

Multiple factors contribute to the errors in name assignment: some incorrect faces being retrieved from the database (though this is low due to our clique-based filtering procedure), inaccuracies in the face feature vectors, such that the same person can sometimes receive dissimilar vectors in the different contexts while different people can receive sometimes similar vectors, and the speaker diarization performed by WhisperX, which sometimes gives the same character a different speaker ID, or gives the same speaker ID to two different characters. This last error is especially problematic because it makes it impossible for the assignment algorithm to find a solution with zero mistakes. We expect that future improvements in speaker diarization and face verification will reduce the prevalence of these errors. Indeed, this is one of the advantages of a modular framework: improvements in specific areas can be incorporated into the framework without needing to change the other modules.

## 2 Submission of conference papers to ICLR 2025

ICLR requires electronic submissions, processed by
[https://openreview.net/](https://openreview.net/). See ICLR’s website for more instructions.

If your paper is ultimately accepted, the statement \iclrfinalcopy should be inserted to adjust the
format to the camera ready requirements.

The format for the submissions is a variant of the NeurIPS format.
Please read carefully the instructions below, and follow them
faithfully.

### 2.1 Style

Papers to be submitted to ICLR 2025 must be prepared according to the
instructions presented here.

Authors are required to use the ICLR LaTeX style files obtainable at the
ICLR website. Please make sure you use the current files and
not previous versions. Tweaking the style files may be grounds for rejection.

### 2.2 Retrieval of style files

The style files for ICLR and other conference information are available online at:

[http://www.iclr.cc/](http://www.iclr.cc/)

The file iclr2025_conference.pdf contains these
instructions and illustrates the
various formatting requirements your ICLR paper must satisfy.
Submissions must be made using LaTeX and the style files
iclr2025_conference.sty and iclr2025_conference.bst (to be used with LaTeX2e). The file
iclr2025_conference.tex may be used as a “shell” for writing your paper. All you
have to do is replace the author, title, abstract, and text of the paper with
your own.

The formatting instructions contained in these style files are summarized in
sections 3, 4, and 5 below.

## 3 General formatting instructions

The text must be confined within a rectangle 5.5 inches (33 picas) wide and
9 inches (54 picas) long. The left margin is 1.5 inch (9 picas).
Use 10 point type with a vertical spacing of 11 points. Times New Roman is the
preferred typeface throughout. Paragraphs are separated by 1/2 line space,
with no indentation.

Paper title is 17 point, in small caps and left-aligned.
All pages should start at 1 inch (6 picas) from the top of the page.

Authors’ names are
set in boldface, and each name is placed above its corresponding
address. The lead author’s name is to be listed first, and
the co-authors’ names are set to follow. Authors sharing the
same address can be on the same line.

Please pay special attention to the instructions in section 5
regarding figures, tables, acknowledgments, and references.

There will be a strict upper limit of 10 pages for the main text of the initial submission, with unlimited additional pages for citations.

## 4 Headings: first level

First level headings are in small caps,
flush left and in point size 12. One line space before the first level
heading and 1/2 line space after the first level heading.

### 4.1 Headings: second level

Second level headings are in small caps,
flush left and in point size 10. One line space before the second level
heading and 1/2 line space after the second level heading.

#### 4.1.1 Headings: third level

Third level headings are in small caps,
flush left and in point size 10. One line space before the third level
heading and 1/2 line space after the third level heading.

## 5 Citations, figures, tables, references

These instructions apply to everyone, regardless of the formatter being used.

### 5.1 Citations within the text

Citations within the text should be based on the natbib package
and include the authors’ last names and year (with the “et al.” construct
for more than two authors). When the authors or the publication are
included in the sentence, the citation should not be in parenthesis using \citet{} (as
in “See Hinton06 for more information.”). Otherwise, the citation
should be in parenthesis using \citep{} (as in “Deep learning shows promise to make progress
towards AI (Bengio+chapter2007).”).

The corresponding references are to be listed in alphabetical order of
authors, in the References section. As to the format of the
references themselves, any style is acceptable as long as it is used
consistently.

### 5.2 Footnotes

Indicate footnotes with a number(^1^11Sample of the first footnote) in the
text. Place the footnotes at the bottom of the page on which they appear.
Precede the footnote with a horizontal rule of 2 inches
(12 picas).(^2^22Sample of the second footnote)

### 5.3 Figures

All artwork must be neat, clean, and legible. Lines should be dark
enough for purposes of reproduction; art work should not be
hand-drawn. The figure number and caption always appear after the
figure. Place one line space before the figure caption, and one line
space after the figure. The figure caption is lower case (except for
first word and proper nouns); figures are numbered consecutively.

Make sure the figure caption does not get separated from the figure.
Leave sufficient space to avoid splitting the figure and figure caption.

You may use color figures.
However, it is best for the
figure captions and the paper body to make sense if the paper is printed
either in black/white or in color.

Figure: Figure 1: Sample figure caption.

### 5.4 Tables

All tables must be centered, neat, clean and legible. Do not use hand-drawn
tables. The table number and title always appear before the table. See
Table 3.

Place one line space before the table title, one line space after the table
title, and one line space after the table. The table title must be lower case
(except for first word and proper nouns); tables are numbered consecutively.

**Table 3: Sample table title**
| PART | DESCRIPTION |
| --- | --- |
| Dendrite | Input terminal |
| Axon | Output terminal |
| Soma | Cell body (contains cell nucleus) |

## 6 Default Notation

In an attempt to encourage standardized notation, we have included the
notation file from the textbook, Deep Learning
goodfellow2016deep available at
[https://github.com/goodfeli/dlbook_notation/](https://github.com/goodfeli/dlbook_notation/). Use of this style
is not required and can be disabled by commenting out
math_commands.tex.

Numbers and Arrays

| $\displaystyle a$ | A scalar (integer or real) |
| --- | --- |
| $\displaystyle{\bm{a}}$ | A vector |
| $\displaystyle{\bm{A}}$ | A matrix |
| $\displaystyle{\bm{\mathsfit{A}}}$ | A tensor |
| $\displaystyle{\bm{I}}_{n}$ | Identity matrix with $n$ rows and $n$ columns |
| $\displaystyle{\bm{I}}$ | Identity matrix with dimensionality implied by context |
| $\displaystyle{\bm{e}}^{(i)}$ | Standard basis vector $[0,\dots,0,1,0,\dots,0]$ with a 1 at position $i$ |
| $\displaystyle\text{diag}({\bm{a}})$ | A square, diagonal matrix with diagonal entries given by ${\bm{a}}$ |
| a | A scalar random variable |
| $\displaystyle{\mathbf{a}}$ | A vector-valued random variable |
| $\displaystyle{\mathbf{A}}$ | A matrix-valued random variable |

Sets and Graphs

| $\displaystyle{\mathbb{A}}$ | A set |
| --- | --- |
| $\displaystyle\mathbb{R}$ | The set of real numbers |
| $\displaystyle\{0,1\}$ | The set containing 0 and 1 |
| $\displaystyle\{0,1,\dots,n\}$ | The set of all integers between 0 0 and $n$ |
| $\displaystyle[a,b]$ | The real interval including $a$ and $b$ |
| $\displaystyle(a,b]$ | The real interval excluding $a$ but including $b$ |
| $\displaystyle{\mathbb{A}}\backslash{\mathbb{B}}$ | Set subtraction, i.e., the set containing the elements of ${\mathbb{A}}$ that are not in ${\mathbb{B}}$ |
| $\displaystyle{\mathcal{G}}$ | A graph |
| $\displaystyle Pa_{\mathcal{G}}({\textnormal{x}}_{i})$ | The parents of ${\textnormal{x}}_{i}$ in ${\mathcal{G}}$ |

Indexing

| $\displaystyle{a}_{i}$ | Element $i$ of vector ${\bm{a}}$, with indexing starting at 1 |
| --- | --- |
| $\displaystyle{a}_{-i}$ | All elements of vector ${\bm{a}}$ except for element $i$ |
| $\displaystyle{A}_{i,j}$ | Element $i,j$ of matrix ${\bm{A}}$ |
| $\displaystyle{\bm{A}}_{i,:}$ | Row $i$ of matrix ${\bm{A}}$ |
| $\displaystyle{\bm{A}}_{:,i}$ | Column $i$ of matrix ${\bm{A}}$ |
| $\displaystyle{\mathsfit{A}}_{i,j,k}$ | Element $(i,j,k)$ of a 3-D tensor ${\bm{\mathsfit{A}}}$ |
| $\displaystyle{\bm{\mathsfit{A}}}_{:,:,i}$ | 2-D slice of a 3-D tensor |
| $\displaystyle{\textnormal{a}}_{i}$ | Element $i$ of the random vector ${\mathbf{a}}$ |

Calculus

| $\displaystyle\frac{dy}{dx}$ | Derivative of $y$ with respect to $x$ |
| --- | --- |
| $\displaystyle\frac{\partial y}{\partial x}$ | Partial derivative of $y$ with respect to $x$ |
| $\displaystyle\nabla_{\bm{x}}y$ | Gradient of $y$ with respect to ${\bm{x}}$ |
| $\displaystyle\nabla_{\bm{X}}y$ | Matrix derivatives of $y$ with respect to ${\bm{X}}$ |
| $\displaystyle\nabla_{\bm{\mathsfit{X}}}y$ | Tensor containing derivatives of $y$ with respect to ${\bm{\mathsfit{X}}}$ |
| $\displaystyle\frac{\partial f}{\partial{\bm{x}}}$ | Jacobian matrix ${\bm{J}}\in\mathbb{R}^{m\times n}$ of $f:\mathbb{R}^{n}\rightarrow\mathbb{R}^{m}$ |
| $\displaystyle\nabla_{\bm{x}}^{2}f({\bm{x}})\text{ or }{\bm{H}}(f)({\bm{x}})$ | The Hessian matrix of $f$ at input point ${\bm{x}}$ |
| $\displaystyle\int f({\bm{x}})d{\bm{x}}$ | Definite integral over the entire domain of ${\bm{x}}$ |
| $\displaystyle\int_{\mathbb{S}}f({\bm{x}})d{\bm{x}}$ | Definite integral with respect to ${\bm{x}}$ over the set ${\mathbb{S}}$ |

Probability and Information Theory

| $\displaystyle P({\textnormal{a}})$ | A probability distribution over a discrete variable |
| --- | --- |
| $\displaystyle p({\textnormal{a}})$ | A probability distribution over a continuous variable, or over<br>a variable whose type has not been specified |
| $\displaystyle{\textnormal{a}}\sim P$ | Random variable a has distribution $P$ |
| $\displaystyle\mathbb{E}_{{\textnormal{x}}\sim P}[f(x)]\text{ or }\mathbb{E}f(x)$ | Expectation of $f(x)$ with respect to $P({\textnormal{x}})$ |
| $\displaystyle\mathrm{Var}(f(x))$ | Variance of $f(x)$ under $P({\textnormal{x}})$ |
| $\displaystyle\mathrm{Cov}(f(x),g(x))$ | Covariance of $f(x)$ and $g(x)$ under $P({\textnormal{x}})$ |
| $\displaystyle H({\textnormal{x}})$ | Shannon entropy of the random variable x |
| $\displaystyle D_{\mathrm{KL}}(P\|Q)$ | Kullback-Leibler divergence of P and Q |
| $\displaystyle\mathcal{N}({\bm{x}};{\bm{\mu}},{\bm{\Sigma}})$ | Gaussian distribution over ${\bm{x}}$ with mean ${\bm{\mu}}$ and covariance ${\bm{\Sigma}}$ |

Functions

| $\displaystyle f:{\mathbb{A}}\rightarrow{\mathbb{B}}$ | The function $f$ with domain ${\mathbb{A}}$ and range ${\mathbb{B}}$ |
| --- | --- |
| $\displaystyle f\circ g$ | Composition of the functions $f$ and $g$ |
| $\displaystyle f({\bm{x}};{\bm{\theta}})$ | A function of ${\bm{x}}$ parametrized by ${\bm{\theta}}$.<br>(Sometimes we write $f({\bm{x}})$ and omit the argument ${\bm{\theta}}$ to lighten notation) |
| $\displaystyle\log x$ | Natural logarithm of $x$ |
| $\displaystyle\sigma(x)$ | Logistic sigmoid, $\displaystyle\frac{1}{1+\exp(-x)}$ |
| $\displaystyle\zeta(x)$ | Softplus, $\log(1+\exp(x))$ |
| $\displaystyle||{\bm{x}}||_{p}$ | $L^{p}$ norm of ${\bm{x}}$ |
| $\displaystyle||{\bm{x}}||$ | $L^{2}$ norm of ${\bm{x}}$ |
| $\displaystyle x^{+}$ | Positive part of $x$, i.e., $\max(0,x)$ |
| $\displaystyle\bm{1}_{\mathrm{condition}}$ | is 1 if the condition is true, 0 otherwise |

## 7 Final instructions

Do not change any aspects of the formatting parameters in the style files.
In particular, do not modify the width or length of the rectangle the text
should fit into, and do not change font sizes (except perhaps in the
References section; see below). Please note that pages should be
numbered.

## 8 Preparing PostScript or PDF files

Please prepare PostScript or PDF files with paper size “US Letter”, and
not, for example, “A4”. The -t
letter option on dvips will produce US Letter files.

Consider directly generating PDF files using pdflatex
(especially if you are a MiKTeX user).
PDF figures must be substituted for EPS figures, however.

Otherwise, please generate your PostScript and PDF files with the following commands:

### 8.1 Margins in LaTeX

Most of the margin problems come from figures positioned by hand using
\special or other commands. We suggest using the command
\includegraphics
from the graphicx package. Always specify the figure width as a multiple of
the line width as in the example below using .eps graphics

or

for .pdf graphics.
See section 4.4 in the graphics bundle documentation ([http://www.ctan.org/tex-archive/macros/latex/required/graphics/grfguide.ps](http://www.ctan.org/tex-archive/macros/latex/required/graphics/grfguide.ps))

A number of width problems arise when LaTeX cannot properly hyphenate a
line. Please give LaTeX hyphenation hints using the \- command.

#### Author Contributions

If you’d like to, you may include a section for author contributions as is done
in many journals. This is optional and at the discretion of the authors.

#### Acknowledgments

Use unnumbered third level headings for the acknowledgments. All
acknowledgments, including those to funding agencies, go at the end of the paper.

## Appendix A Appendix

You may include other additional sections here.

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 3
presentation: 3
questions: Questions:
- What makes the two algorithms innovative if there is already so much out there?

- The objective: the movie application is exactly where these kind of techniques for automated objective textual summarization seem less interesting as you can find a movie summary for every movie at the IMDB website and if not, likely the actors can also not be found there. Personalized summaries of movies are in my opinion more interesting (although even more difficult to evaluate, you should probably work with real users for that). Another issue with the objective is that a typical summary should not contain any spoilers where the current system doesn't take that into account. 

Remarks:

Scoping:  The paper says that summarization is usually done from the text perspective but that is maybe true for the NLP community but in Computer Vision there is (and has been from the 90's) a lot of work on visual video summarization. Many of those are referring to the datasets you are also mentioning (like MovieSum). So the authors should remove this claim (or substantiate it with evidence based on citations) and consider the computer vision driven video summarization literature and make more clear how the current work is different. 

Ref [Abhimanyu Dubey] is not proper. The CLIP website actually gives you a proper way to refer to it.
soundness: 3
strengths: - a complete system addressing all modalities
- well written, easy to read
- potential for interesting applications
summary: This paper proposes a complete system to take an entire movie with associated audio track and automatically creates a textual summary of the movie. For the system a set of mostly existing methods are integrated into a full solution. The two elements that are (according to the authors) new are the scene segmentation and character identification method, for the text generation a standard LLM is employed.
weaknesses: - The tasks for the given "new" algorithms go back to the 90's of the previous century. Yet the methods are compared to non-algorithmic baselines like just doing random or uniform assignment. For example the Scene Transition Graph by IBM from 1997 (https://www.spiedigitallibrary.org/conference-proceedings-of-spie/3312/1/Classification-simplification-and-dynamic-visualization-of-scene-transition-graphs-for/10.1117/12.298470.short) and the NameIT system by Satoh (Name-it: Naming and detecting faces in news videos
S Satoh, Y Nakamura, T Kanade - Ieee Multimedia, 1999). Both have a multitude of citations so hence many follow-up systems. The authors should do a thorough review of the field and compare their methods to the best performing ones. 
- Overall the system seems very much to be extensions of the work of Lapata which are the only ones mentioned in long-form summarization. Authors should do a far more in-depth analysis of the field. I googled "automatic summarization of complete movies" and already got a number of papers which are related. Authors should indicate how their method is different from those. 
- This is not a typical ICLR paper where the main contribution of the paper is on the algorithmic side. As indicated above the contribution in terms of algorithms is limited so the main novelty is the integration of all of these into one system. Other venues like ACM Multimedia, SIGIR, or a similar conference are far more suited for that topic, so my suggestion would be to improve the work and submit the work there.

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 3
questions: (1) Please discuss the quality of the screenplay output of the method.
(2) What would be the quality of the summary generated by the name-only prompt of the current leaders of the ChatBot Arena (say ChatGPT-4o, Gemini 1.5 and Claude 3.5)? if it is substantially better than the proposed method, then the case for the proposed research will be weakened.
(3) In Table 3, the name-only prompt results are actually quite impressive. It is not clear why the quality degrades when more information is provided i.e. full script and whisperx script. That appears to be a bit counter-intuitive.
soundness: 3
strengths: The strengths of this paper are:

(1) The authors propose a new task for movie content processing i.e. generating the screenplay of the movie using only the video and audio content of that movie.

(2) The authors have proposed a solution framework, as captured in Figure 1, which is both plausible and also provides scope for improvement for follow-up work.

(3) The quality of the summary generated, as shown in the example, is very good even when compared to the gold standard.
summary: The paper presents a new problem of automatically generating the screenplay of a full-length movie (which is typically of at least 90 minutes length) utilizing only the video and audio information. The primary motivation is for movie summarization (as inferred from the title of the paper). This assumes that the actual original screenplay, based on which the movie had been originally shot, is unavailable. The method has four main components - text transcript generation from audio which essentially performs speaker diarization, video scene detection, identification of the names of the characters in the movie and associating them with the transcript using the face images, and finally screenplay generation using caption generation of three key-frames per scene. The contributions are (a) the new task (b) scene boundary detection algorithm (c) detection of character names. Evaluation results are provided for scene detection, character name assignment and movie summarization in text using the generated screenplay.
weaknesses: (1) The motivation of the paper is somewhat weak and can hope fully be strengthened. It is assumed that the screenplay of a movie is not available. Given that screenplays exist for all movie created (including the nascent attempts at creating movies using diffusion models), why not just purchase them? They are apparently available at www.scriptly.com (for example the screenplay for the movie Oppenheimer is available for $24.95 on that site), it does not seem worthwhile to work on this problem. Other than just taking it up as an intellectual challenge. It would be good to strengthen the motivation. A possible way to strengthen it is by thinking of situations where the screenplay might not be available. Or if there is a divergence between the original screenplay and the actual movie which could be captured via the generated screenplay.

(2) In general, it appears that the authors are not familiar with the literature in the multimedia content processing and computer vision areas. Scene boundary detection has been researched there for decades and while the proposed method appears to be novel, a proper literature survey needs to be done. Even NetFlix and Amazon Prime have published methods: 

https://netflixtechblog.com/detecting-scene-changes-in-audiovisual-content-77a61d3eaad6 
https://www.amazon.science/blog/automatically-identifying-scene-boundaries-in-movies-and-tv-shows

A comparison with the state of the art methods such as the above ones and also in the literature, will help establish the novelty as well as the superiority of the proposed approach.

(3) The name assignment method is simple and effective. However, it appears that there could be misses and also incorrect assignments. The frequency of its occurrence seems to be roughly 34% from Table 2. It is not clear what is the impact of these errors on the quality of the screenplay generation and also on the summarization?

(4) While the paper provides evaluation results for the summarization task, no evaluation is provided for the main work of the paper which is the screenplay generation. Why is there no quality evaluation of the screenplay generated by the proposed method? Only the indirect measure via the quality of summary generated is provided. It would strengthen the paper if an evaluation is done by comparing the generated screenplay with the original screenplay, perhaps using some of the text processing metrics used for the summary comparisons.

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 3
questions: - It seems this paper only uses text and audio as input, and does not consider image and visual input. Is my understanding correct? [Updated: from author responses, this submission uses visual input]. 

- From the paper it is not clear what are the details of Otter AI. I am not sure whether the comparison is done by using the product, or reimplementing the paper published in 2023.

- Consider expanding the evaluation by including additional metrics, especially end2end user evaluation.
soundness: 3
strengths: - Good Application: This system has many potential everyday applications. 
- Scene Detection: Introduces a novel, parameter-free scene boundary detection method based on the minimum description length principle.   
- Character Naming: Proposes a new algorithm for identifying and assigning character names based on a database of actors' faces.   
- Hierarchical Summarization: Employs a hierarchical approach for summarizing movies, first condensing individual scenes and then fusing them into a final synopsis.   
- Zero-Shot Learning: Utilizes zero-shot prompting for summarization, demonstrating the potential of leveraging large language models without task-specific fine-tuning.
summary: This paper presents a system for generating automatic screenplays and summaries from movies. The system is a combination of several well-established techniques, including keyframe extraction, speaker diarization, face recognition, and scene segmentation.

After carefully reviewing the authors' responses, I believe this research would be highly valuable if the authors shared their code with the community. I would be willing to change my rating to 'above threshold' if the authors decide to make their code open source.
weaknesses: Novelty: The system primarily combines existing techniques like keyframe extraction, speaker diarization, and face recognition into a single pipeline. While the integration of these techniques is non-trivial, it may be more convincing if the paper can clarify the contribution of each component and identify their novelty. 

More Comparison: The paper lacks a comparative analysis with current powerful LLMs, both publicly available models like GPT-4 and Gemini. In addition, it is not clear how the commercial system (Otter AI) is used in comparison. It will be crucial to dig deep to assess the  potential advantages and disadvantages of the proposed system.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: see weakness
soundness: 2
strengths: 1. The paper proposes a scene detection algorithm to divide long video sequences into chunks. It segments a sequence of visual feature vectors into scenes by minimizing a cost function based on the minimum description length principle, encoding each scene’s vectors relative to their mean, and using dynamic programming to efficiently find the optimal partition.

2. The paper proposes a character face assignment algorithm to associate specific character names with the screenplay, incorporating face similarity between a character bank and scene information.
summary: This paper introduces ScreenWriter, a framework that generates screenplays and movie summaries directly from video and audio without relying on text-based transcripts. Designed to address the growing demand for video summarization, ScreenWriter overcomes limitations of existing methods that require screenplay or transcript data by operating solely on video content.  The framework includes several modules: 1. Scene Detection algorithm 2. Character Name Assignment algorithm 3. Screenplay and Summary Generation.
weaknesses: 1. Most components of the proposed framework are adopted from existing works, lacking academic and technological contributions. There is almost no insight provided on summarizing long-context videos using a “screenplay.”

2. The paper lacks an ablation study on the effect of different model sizes/types on the summarization results.

3. The screenplay generation process seems somewhat unnecessary. If the transcript is summarized and then processed by the summarizer alongside visual information in parallel, why is a screenplay still needed?

4. The baseline selection in the experiment lacks justification. For face assignment, randomly assigning or commonly assigned methods are not ideal choices. For scene detection, there is no comparison with classical tools like SceneDetection.

5. There are existing video summarization frameworks, such as MM-VID[1], MM-Narrator[2] and MM-ScreenPlayer[3], that show high similarity in their pipelines. A discussion of these works should be included.

6. Some typo:
    - 'sctructure' in line 67 should be 'structure'
    - 'movide' in line 317 should be 'movie'


Reference:

[1]Lin, Kevin, et al. "Mm-vid: Advancing video understanding with gpt-4v (ision)." arXiv preprint arXiv:2310.19773 (2023).

[2]Zhang, Chaoyi, et al. "Mm-narrator: Narrating long-form videos with multimodal in-context learning." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

[3]Wu, Yongliang, et al. "Zero-Shot Long-Form Video Understanding through Screenplay." arXiv preprint arXiv:2406.17309 (2024).

Meta-review: The paper proposes a movie understanding problem and solution. The reviewers appreciate the problem, writing, and aspects of the solution. However they raise concerns about motivation (given availability of screenplays), novelty (limited algorithmic contribution, limited discussion of broader prior work), and experiments (baseline selection, lack of comparison to commercial LLMs). There is some support for acceptance but stronger opposition (three negative and one positive score).

Decision: Reject