A Descriptive Basketball Highlight Dataset for Automatic Commentary Generation

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 05:24:27

Abstract: The emergence of video captioning makes it possible to automatically generate natural language description for a given video. However, generating detailed video descriptions that incorporate domain-specific information remains an unsolved challenge, holding significant research and application value, particularly in domains such as sports commentary generation. Moreover, sports event commentary goes beyond being a mere game report, it involves entertaining, metaphorical, and emotional descriptions. To promote the field of sports commentary automatic generation, in this paper, we introduce a novel dataset, the Basketball Highlight Commentary (BH-Commentary), comprising approximately 4K basketball highlight videos with ground-truth commentaries from professional commentators. In addition, we propose an end-to-end framework as a benchmark for basketball highlight commentary generation task, in which a lightweight and effective prompt strategy is designed to enhance alignment fusion among visual and textual features. Extensive experiments on the BH-Commentary dataset demonstrate the validity of the dataset and the effectiveness of the proposed benchmark for sports highlight commentary generation. (The dataset is available at https://anonymous.4open.science/r/dataset-DC8E)

Main paper: 
 introduction: 
 Video captioning [48, 63] stands as a challenging and essential task in both the computer vision and natural language processing communities. Aimed at automatically generate the description about the visual content of a given video in natural language, this task has gained significant attention in recent years due to its importance across various applications. One good example is sports commentary generation (especially for team sports such as football, basketball, and volleyball etc). Figure 1 illustrates the distinction between the conventional video captioning task and our sports commentary generation. We can notice that the conventional video captioning can solely provide a macroscopic perspective description of the video (e.g., a scene featuring players are playing basketball). In contrast, the commentary generation is capable of offering more vivid description of individual technical movements and the coordination between team members (e.g., Player A launches a long pass from the backcourt, setting up Player B for a one-handed slam dunk, showcasing flawless teamwork.). For a sports highlight video that showcases exquisite individual move and teamwork, the key of commentary generation lies in capturing the the visual characteristics of athletes' technical movements and map them into statements that contain technical terms and descriptive words. However, for basketball highlight video, this presents several challenges. Firstly, the basketball highlights usually contain players' gorgeous technical movements and exquisite teamwork, which provides a visual basis for downstream generative model. Accurately and effectively capturing and representing these visual features from video can offer more informative cues for commentary generation. Several recent studies [50, 64, 70, 73] focused to leverage the action information contained in videos to enhance the downstream task. However, such studies usually employ multiple feature extractors that trained for visual understanding tasks to extract 2D and 3D visual features. Although these approaches have shown promising results, there raises problems about the extent to which these extracted features from off-line extractors can be effectively adapted to suit the requirements of the captioning task. Secondly, the generated commentaries should highlight the player's gorgeous moves and coordination, that is the model should be able to generate commentaries containing the player's technical movements based on visual information. Most of the existing research [3, 19, 47, 55] tends to emphasize the effective exploitation of visual features, with few taking into account the significance of cross-modal interactive fusion and explicitly leveraging such interaction to enhance the downstream generation. Moreover, as for basketball highlight video, the commentary on it must not be a simple description of the player's actions. While it is difficult to train a model that can generate commentary that is descriptive and professional in content on the basis of existing video captioning datasets. In this context, there is a significant need for a readily accessible sports commentary dataset, annotated by professional sports analysts, to facilitate research in this area.To explicitly tackle these challenges and develop a practical sports highlight commentary generation system, particularly in the context of basketball highlight video, in this paper, we propose an end-to-end framework for basketball highlight commentary generation. Specifically, for visual feature extraction, taking inspiration from the recent works of transformer-based models in video understanding [2, 13, 14] , we utilize a video transformer to extract features with the original video as input. In contrast to employing separate offline 2D and 3D visual extractors, our model integrates visual extraction within a unified framework, along with subsequent multi-modal feature encoding and commentary generation modules. This integration enhances the suitability of the extracted visual features for downstream tasks. As for multi-modal feature encoding, a lightweight but effective prompt strategy is designed to promote the interaction fusion between visual and textual features, which prompts the model to focus on the visual representations that are most relevant to the text. It is worth noting that in our proposed model, each component is integrated into the unified framework, which makes the components of the model compatible and mutually reinforcing. Moreover, to initiate shareable research in this emerging field, we are introducing a new dataset, called Basketball Highlight Commentary (BH-Commentary). This dataset comprises 4,396 high-definition NBA basketball highlight videos from the Tencent Video website, each of which is annotated with detailed descriptive commentary.In summary, the main contributions of this work are summarized as follows:• We collect a novel high-quality dataset for sports highlight commentary generation, which contains 4K basketball highlight videos from websites and corresponding commentaries from professional commentators.  
related works 2.1 video captioning: 
 Video captioning aims to generate a condensed natural linguistic sentence that describes the main event of a video. Early researches adopt the template-based strategy to generate video captions [24, 60] , this sort of methods usually align the sentence components to the detected visual content, and generate the description based on the pre-defined templates, which are typically limited by the fixed templates. Recent works usually adopt encoder-decoder structure for this task [17, 48, 68] , where the encoder translates the input video to visual features, and the decoder integrates the encoded visual features and generate a natural sentence. Since without bounded by the pre-defined template, such methods can generate captions with more flexible sentence patterns. Specifically, based on the extracted visual or visual-linguistic feature, [47, 66, 69] utilize the LSTM/GRU-based architecture for caption generation task, [50, 63, 72] use transformer-based model for video captioning generation. Unlike the above models that adopt offline feature extraction, we take an end-to-end approach to integrating feature extraction with downstream task.Transformer [53] , adopting an attention-based encoder-decoder structure, has demonstrated promising performance on the NLP tasks. Inspired by the outstanding ability on sequence modeling, some recent researches explore transformer-based structure in the field of computer vision, achieving remarkable results on basic CV tasks [13, 18, 58, 65] . Since the competitive modeling capabilities, the visual transformers have achieved impressive performance improvement compared with the traditional methods. The application of visual transformer to video field is also gaining increasing attention. In order to cope with the characteristics of videos with long sequences, Neimark 𝑒𝑡 𝑎𝑙 . [40] adopt temporal attention-based encoder, which could attend to all tokens in the input sequence, making the model capable of handling long sequences. Arnab 𝑒𝑡 𝑎𝑙 .[2] introduce a transformer-based model to employ spatial-temporal attention for better video representation. Zhang 𝑒𝑡 𝑎𝑙 . [67] propose stacked attention to aggregate spatio-temporal information contained in the video for improving representation learning. Moreover, inspired by the success of Swin Transformer in image domain [35] , Liu 𝑒𝑡 𝑎𝑙 . [36] further propose Video Swin Transformer, which introduces an inductive bias of locality in spatiotemporal domain into transformer structure, obtaining promising video representation.Joint vision language understanding associates the computer vision and natural language processing together, and has attracted increasing attention from the two fields. Recent researches [27, 50] have shown the success in the field of multi-modal representation learning for vision-language understanding and generation, including downstream tasks like video-language retrieval [6, 16] , video question answering [26, 32] , and video captioning [56, 62] .In order to get better performance, most language models tend to adopt large scale training data, causing the loss of computation and memory. With the success of the language pre-training and video-text pre-training strategy, recent works attempt to employ the pre-trained language models to the vision language task. For example, by freezing the weights of a pre-trained language model, [1, 52, 57] show promising results in vision language tasks. Moreover, masked language models also show success in language works [12, 25, 31] , which pre-trains a transformer-based structure to learn language representations, achieving competitive performance in downstream tasks after being fine-tuned. The success of masked language models also drives the exploration the works of applying it to the multi-modal representation model with paired visual-textual data [15, 29, 37] , which show competitive performance on vision language tasks.  
basketball highlight commentary dataset: 
 Basketball Highlight Commentary (BH-Commentary) is a basketball highlight video commentary generation dataset. Each highlight clip is annotated with an description of its content. Unlike previous video captioning datasets that describe motions from a macro perspective, this dataset focuses on providing a lively language of commentary on the technical movements of players in basketball videos, where each comment corresponds to one event highlight.In the following, we introduce the dataset collection process and provide a comprehensive statistical analysis on this dataset.We collect 4,800 highlight videos from the NBA's 2020-2023 season from websites. And we filter out videos that are too short and had poor visual quality, ultimately choosing 4,396 videos with diverse and detailed motions for the final annotation process. Basketball highlights videos in our dataset involve six categories of basketball actions, including pass, dunk, block, shot, steal and layup, as shown in Figure 3 Our dataset includes 4,396 videos, each of which corresponds to one annotated statements from professional commentators. Each video has an average of 15.3 words. On average, each word describes 0.5s in video and 4.8% of the entire video, which demonstrates that our annotations are informative and detailed, comprehensively encompassing the contents in the video. Table 1 provides a comparison of major statistics between our dataset and other existing popular video captioning datasets. Unlike other datasets collect videos from common domain or generate them virtually that stand out with longer total video duration, our dataset mainly focuses on highlight from real basketball game scenes. Based on a limited number of basketball game highlight and the corresponding commentaries from professional commentators, our dataset contains 4,396 highlight videos with a total of 10.1 hours and the same number of annotations. As shown in Figure 2 (a), in our dataset, the most frequently occurring words are the names of the players, followed by words that are semantically related to basketball and associated elements. In addition, the distribution of video duration of our dataset is shown in Figure 2 the shortest one lasts 3.1s, and the average video duration is 10.5s. And the distribution of commentary length of our dataset is shown in Figure 2 (c), the length of the commentaries in our dataset varies, with the longest being 36 words and the shortest being 6 words.球 员 上 演 对 手 暴 扣 完 成 防 守 三 分 扣 篮 单 臂 篮 下 劈 扣 对 方 空 接 隔 扣 上 篮 命 中 传 球 突 破 暴 力 轻 松As for the Chinese version of our dataset, Figure 3(b-c ) illustrate the distribution of frequent words and commentary length in Chinese version. As shown in Figure 3(b) , the length of the Chinese commentaries in our dataset varies, with the longest being 43 words and the shortest being 7 words. In addition, similar to the English version, in Chinese version of our dataset, the most frequently occurring words are the names of the players, followed by terms that are semantically related to basketball and associated elements, as shown in Figure 3(c) .Moreover, we conduct a parts of speech analysis on our dataset in comparison with MSR-VTT [59] . As depicted in Figure 4 , our dataset exhibits a higher proportion of nouns, adverbs, and adjectives, which underscores our dataset's increased focus on players, their technical movements, and the accompanying descriptive elements. In Table 2 , the comparison of the average number of verbs, nouns, adjectives, and adverbs per sentence further demonstrate the descriptive advantage of our annotations. In each highlights video, our annotations feature a higher count of descriptive words per sentence, this is in line with our objective: delivering vivid commentaries for sports highlights. And for dataset splitting, we take the same settings as MSR-VTT dataset that we randomly divided the dataset into training, validation, and testing sets with proportions of 65%, 5%, and 30%, respectively.Committed to advancing the researches about video captioning/ description generation, multiple datasets covering various domains have been introduced. In general, video captioning tasks can be primarily categorized into two families: single event caption generation [9, 59] and multiple events caption generation [39, 64, 71] . As shown in Table 1 , due to the objective reasons such as the difficulty of video collection and annotation, previous studies rarely focus on sports video description generation. Some studies, such as [39] , utilize virtual methods such as games to create sports game videos for building dataset. [44] and [64] built datasets of video descriptions generation based on volleyball and basketball games. However, all the previous works on sports video description generation focus on the relaying of players' movements during a game. In this work, based on the basketball games, we propose the fist dataset focusing on sports highlight commentary generation, which provides more descriptive language than mere statements of action. Compared with the existing basketball video description dataset FSN [71] , our dataset tends to provide emotional and descriptive commentary for basketball highlights, which makes it easier for the audience to empathize with the excitement of the game. 
commentary generation model: 
 The goal of sports highlight commentary generation is to automatically generate eloquent and descriptive sentence to paint a vivid picture of the technical movements executed by the players in the video. This challenge raises the question of how to enable the Player A waved his hand and Player B understood without words, with both hands hanging on the rim, he executed a thunderous dunk with the grace of flowing water.Textual Encoder Visual Encoder Prompt MHA GateFigure 5 : The overall architecture of the proposed model. We formulate the commentary generation as a sequence-to-sequence task, the raw video frames and the text are first encoded by the visual and textual encoder, respectively. The prompt embedding that utilized to facilitate the multi-modal feature fusion is aggregated with the input and the previous state, and is concatenated with the multi-modal embeddings, which are further input to the multi-modal encoder. Then the language decoder head autoregressively generates the output commentary based on the multi-modal representaions.efficient mapping of visual input to commentary output. Furthermore, the well-extracted visual features could serve as crucial visual cues for commentary generation. To tackle these problems, we first introduce a unified end-to-end multi-modal encoding framework, treating the automatic generation of sports highlights commentary as a sequence-to-sequence task, as explained in Section 4.1. Then, an effective prompt strategy is devised to enhance the alignment of multi-modal representations in Section 4.2. And the strategy of training and inference are introduced in Section 4.3.We wish to design an architecture that can effectively map the sports highlight video to corresponding descriptive commentary.To achieve this goal, we introduce an end-to-end framework that takes raw sports highlight video frames as input and generates natural language commentary for input content description. Figure 5 shows the overview of our proposed benchmark model. In detail, given a pair of video {𝑓 𝑡 }| 𝑇 𝑡 =1 and text sequence {𝑠 𝑛 }| 𝐿 𝑛=1 , where 𝑇 represents the number of sampled frames from the input video, and 𝐿 denotes the length of the sentence. We first separately encode them using individual encoders to obtain unimodal features, the visual encoder extracts visual features from the raw sports highlight video frames, while the text encoder embeds the textual representation. Subsequently, the multi-modal encoder further encodes the multi-modal representation based on both the visual and textual features. And the commentary is generated in an auto-regressive manner. The detailed description of each module are given as follows.Visual Encoder. Drawing inspiration from the success of various transformer-based model for video representation learning in long-range temporal relationship modeling [5, 36, 61] , recent advancements in video-language research [31, 51] have begun to leverage the success of video transformers, showcasing improved performance in downstream tasks. In this paper, we employ the Video Swin Transformer [36] (VST) as visual backbone for visual feature extraction, based on the frames from raw input video.The visual encoder takes a sequence of 𝑇 frames 𝑓 ∈ R 𝐻 ×𝑊 ×3 sampled from raw video as input, where 𝐻 and 𝑊 refer to the height and width of each frame. Then the grid features are extracted from the last encoder block of VST, resulting in grid features with dimension of 𝑇 2 × 𝐻 32 × 𝑊 32 × 8𝐶, where 𝐶 represents the channel dimension. These grid features are then tokenized along the channel dimension, yielding a total of 𝑇 2 × 𝐻 32 × 𝑊 32 video tokens, with each token being an 8𝐶-dimensional feature vector. For more in-depth information, please refer to [36] . These extracted visual features are then utilized as input for the multi-modal fusion encoder to facilitate the learning of cross-modal representation. Textual Encoder. For text encoding, the input text sentence is first tokenized into a sequence of 𝑁 tokens {𝑡 𝑛 }| 𝑁 𝑛=1 . And two special tokens, [CLS] and [SEP] , are inserted at the start and the end of the token sequence. Then, like previous works [27, 37] , we utilize a lightweight word embedding layer [21] to obtain textual embeddings, which are concatenated with the visual features and then input to the multi-modal encoder. Multi-modal Encoder. We utilize a transformer-based multi-modal encoder for multi-modal features encoding. Specifically, the multimodal encoder takes two modal inputs, which correspond to the visual and textual features extracted from the two unimodal encoders. Denoting the encoded visual and textual embeddings as 𝐸 𝑣 ∈ R 𝑇 ×𝑘 and 𝐸 𝑡 ∈ R 𝑁 ×𝑘 , we then concatenate these two embeddings as input to the multi-modal encoder, denoted as 𝐸 𝑚 = [𝐸 𝑣 ; 𝐸 𝑡 ] ∈ R (𝑇 +𝑁 ) ×𝑘 , where [; ] denotes concatenation and 𝑘 denotes the dimension of hidden state. To obtain the cross-modal representations, the visual and textual embeddings are combined through cross-attention operations. Then we conduct sequence to sequence generation process to implement commentary generation, where we employ a causal self-attention mask, ensuring that a caption token can only attend to the previously generated output tokens.Through our generic design, we enable end-to-end training for commentary generation using the raw video frames. Furthermore, by leveraging the versatility of the transformer architecture, our model can handle video sequences with variable lengths. The illustration of prompt embedding updating scheme. The prompt embedding is selectively updated according to the input using the multi-head attention with a residual connection.Recently, a line of works show promising performance in obtaining the desired output through prompt designing [20, 28, 30, 41] . Instead of designing manually, soft prompt is proposed as a series of continuous embeddings that are prepended to the input and updated throughout training. In this work, we propose to utilize a lightweight soft prompt strategy using the attention network with a residual connection for promoting multi-modal features interaction and fusion. Instead of using the original prompt setting [28] , we pass the soft prompt embedding and multi-modal embeddings through the attention network with a residual connection. Subsequently, we reparameterize the prompt and prepend it to the multi-modal embeddings before feeding it into the multi-modal encoder. In specific, as shown in Figure 6 , we set a sequence of soft prompt embedding 𝑃 = [𝑝 1 , . . . , 𝑝 𝑛 ] ∈ R 𝑛×𝑘 , here 𝑛 and 𝑘 denote the number of and the dimension of prompt vectors respectively.With multi-head attention, we can aggregate the feature from both the multi-modal embeddings 𝐸 𝑚 and the prompt embedding 𝑃, and the prompt embedding can then be selectively updated according to both the current input and the previous state with residual connection. Then the prompt-concatenated multi-modal representations are further encoded through the multi-modal encoder. The entire process above is denoted as below:EQUATIONwhere FFN denotes feed-forward network, 𝑀𝐻𝐴𝑡𝑡 denotes multihead attention in transformer network [53] , tanh and sigmoid denote activation functions, ⊙ denotes Hadamard product, 𝑊 𝑠𝑝 , 𝑊 𝑠𝑎 , 𝑊 𝑧𝑝 and 𝑊 𝑧𝑎 are trainable weights, 𝑏 𝑠 and 𝑏 𝑧 are trainable bias.Train Setting. The visual encoder is pre-trained on the Kinetics action recognition task [8] . During training, the model takes video and text input, which are further input to the visual and textual encoder for feature extraction. The prompt embedding is jointly updated with the model during training. Furthermore, all textual tokens have complete attention not only to the visual tokens but also to the prompt, ensuring that the prompt can enhance the comprehensive interaction of multi-modal features, which allows the model to effectively utilize both visual and textual modalities to generation accurate and descriptive commentary. Inference. During inference, the model solely takes the video as input, and the commentary is generated in an auto-regressive manner. The model generates one textual token at a time, using the tokens generated thus far as inputs for the multi-modal transformer encoder. And the prompt is no longer updated, instead, it serves the purpose of facilitating the commentary generation. 
experiment: 
 In this section, we demonstrate the effectiveness of our benchmark model on its ability of generating sports highlight commentary. We conduct experiments on BH-Commentary dataset, which is specifically built for this task, and we compare our model to the state of the art. We first introduce the experimental setting in Section 5.1, and the ablation study is conducted in Section 5.2. Finally, we present the experimental results and analysis in Section 5.3.Metrics. We adopt several widely-used evaluation metrics, including BLEU@4 [42] , METEOR [4] , Rouge-L [33] , and CIDEr [54] to measure the performance of the benchmark model. We calculate these metrics using the standard COCO evaluation tools1 [10] . Implementation Details. Our model is implemented in PyTorch [43] , the visual encoder is initialized with Kinetics-600 pre-trained weights, the textual encoder is initialized from pre-trained BERT-Base [12] , and the multi-modal encoder is initialized randomly. The number of prompt vectors is set to 8, which is equal to 1% of the The whole model is trained in end-to-end manner. In addition, we adopt the AdamW [22] optimizer with an initial learning rate of 3e-5 and use a learning rate warm-up during the early 10% training steps followed by linear decay.To verify the effectiveness of the designed prompt, we show the performance changes in the last block of Table 3 by removing the prompt and simply input the concatenated multi-modal embeddings to the multi-modal encoder for the following commentary generation, which obviously results in a decline in model performance across all evaluation metrics. The results suggest that the performance of commentary generation can be greatly lifted by using prompt for promoting multi-modal feature fusion. Moreover, the model without prompt setting is selected as the baseline model for comparison in experiment analysis, which is discussed in the following section.Compare to the State of the Art. We consider four up-to-date baselines for comparison. Table 3 lists the main results on the commentary generation task. According to Table 3 , our benchmark model demonstrates the capability to generate more accurate and higher-quality commentaries when compared with the baselines. We attribute the superior performance of our benchmark model to two main factors. Firstly, the end-to-end setting enables the modules to be iteratively updated within a unified framework, enhancing the compatibility between each module. Additionally, the well-designed prompt plays a crucial role in facilitating the fusion of multi-modal features, thereby promoting the downstream commentary generation task. However, from an intuitive perspective, the models' performance metrics on our dataset seem comparatively lower than on other datasets. This can be attributed to the inherent complexity of generating commentary for sports highlight videos. The intricate sentence structures and highly descriptive content in our dataset pose significant challenges to the learning process of the model. Despite this, our model serves as an inspiration for future efforts to address the challenges in sports highlight commentary generation.Qualitative Analysis. To further qualitatively assess the performance of our benchmark model in the commentary generation task, Figure 7 shows generated commentaries can cover the key technical moves of the players. In specific, for the first example, the video showcases the clever pass and powerful dunk executed by players. As shown in Figure 7 (a), our model can accurately generate commentary that encompasses detailed actions and matches the groundtruth. In contrast, the baseline model is not able to generate the appropriate vocabulary to link the two actions, thereby failing to depict the coordination between the players. For the second example shown in Figure 7 (b), we can observe that our model accurately generates the basketball skill action "two-handed dunk," which aligns with the groundtruth. In contrast, the baseline model fails to do so. As shown in Figure 7 (c), in the third example, the commentary produced by our model accurately describes the actions "dribbles past the defense" and "one-handed slam dunk, " which correspond to "drives straight in" and "sky-high slam dunk" in the groundtruth, respectively. While the baseline model, though correctly generating the term "one-handed slam dunk, " provides inaccurate information with "clever pass." In the fourth example, as shown in Figure 7 (d), our model delivers precise commentary, stating that the player successfully nails a long-range three-pointer, followed by a concise summary description. While the baseline model just repeats the given information.Chinese Version. We also conducted a qualitative analysis on the Chinese version of the dataset, as illustrated in Figure 8 . Our model accurately provides descriptions in the commentary, enriched with the suitable idioms. Specifically, in the first example, the highlight video showcases the player's dynamic and powerful dunk as he breaks through the defense. As shown in Figure 8 (a), our model excels in generating accurate commentary that matches with the groundtruth and provides a summary description. In the second example, shown in Figure 8 (b), we can observe that our model accurately generates the basketball skill actions based on precise pass and dunk, which is consistent with the groundtruth. In contrast, the baseline model produces inaccurate information with"threepointer." As shown in Figure 8 (c), in the third example, the commentary generated by our model accurately provides the term "three-pointer" and corresponding description. While the baseline model just repeats the given information. In the fourth example shown in Figure 8 (d), our model outperforms the baseline by offering additional detailed information about Player A before executing the pass, which is not mentioned but actually true in the video. Shortcoming. During experiments, we also find some shortcomings in our benchmark model. Our model may sometimes fail to recognize similar movements, such as layups and dunks. Moreover, as shown in Figure 7 and Figure 8 , compared with the groundtruth, out generated commentary may lack some background information description like "fast break", and does not have the ability to generate such statements like "showcasing what "unstoppable" means with a sky-high slam dunk". Our benchmark model serves as inspiration here. Addressing the challenge of enabling the model to understand these statements and generate them in the appropriate context remains a task that needs further exploration in subsequent research. 
conclusion: 
 In this work, we create a descriptive basketball highlight video dataset for sports highlight commentary generation task. We propose a benchmark model for this task and outperform the state of the art models. Extensive experiments demonstrate the effectiveness of the proposed benchmark model and validity of the collected dataset. Due to the rich content of descriptive commentary, it is apparent that there is room for improvement in the performance of the model for the sports highlight commentary generation task, which remains research direction for subsequent studies. Our benchmark model serves as an inspiration here for the further research.  
 
 Appendix: https://github.com/tylin/coco-caption2024-04-13 01:41. Page 6 of 1-10. 


Decision: The reviewers tend to recommend acceptance and the authors addressed some of the raised questions. I think this would be a valuable contribution as poster given also the dataset resources proposed by the paper.

Reviews: review: paper_summary: paper_summary: The paper discusses the challenge of generating detailed video descriptions that incorporate domain-specific information, specifically in the context of sports commentary generation. The authors introduce a novel dataset called BH-Commentary, consisting of basketball highlight videos with ground-truth commentaries from professional commentators. They also propose an end-to-end framework as a benchmark for basketball highlight commentary generation, which includes a prompt strategy to enhance alignment fusion among visual and textual features. Experimental results on the BH-Commentary dataset validate its usefulness and demonstrate the effectiveness of the proposed benchmark for sports highlight commentary generation.
strength_weakness: strength_weakness: The strengths of the paper include:

Novel dataset: The introduction of the Basketball Highlight Commentary (BH-Commentary) dataset is a significant strength. This dataset provides a valuable resource for researchers and practitioners in the field of sports commentary generation, as it consists of approximately 4K basketball highlight videos with ground-truth commentaries from professional commentators. The availability of such a dataset allows for more accurate evaluation and comparison of different approaches in this domain.

End-to-end framework: The proposed end-to-end framework for basketball highlight commentary generation is another strength of the paper. This framework offers a comprehensive solution to the challenge of generating detailed and domain-specific video descriptions. By designing a prompt strategy to enhance alignment fusion between visual and textual features, the framework aims to improve the overall effectiveness of the commentary generation process.

Experimental validation: The paper provides extensive experimental results on the BH-Commentary dataset, which demonstrate the validity of the dataset and the effectiveness of the proposed benchmark for sports highlight commentary generation. This empirical evidence adds credibility to the research findings and highlights the practical value of the proposed approach.
limitations: limitations: 1. Anonymous websites load very slowly, which seriously affects the review effect.
2. The pipeline figure can be further optimized. The current layout of the entire image is not very beautiful.
3. In the introduction, authors need to spend more energy introducing their strengths and advantages. This allows the work to be highlighted more.
paper_summary: paper_summary: The paper proposes the Basketball Highlight Commentary (BH-Commentary) dataset to faciliate research of automatic commentary generation. For building the dataset, the authors have collected approximately 4K basketball highlight videos from websites and corresponding ground-truth commentaries from professional commentators. In addition, the paper propose a benchmark model for  commentary generation from sport videos. Extensive experiments on the BH-Commentary dataset show the effectiveness of the proposed benchmark model, and reveal the challenges of sports highlight commentary generation.

The proposed dataset brings new challenges comparing to existing Video Caption datasets. From both academic and industrial perspective, this work potentialy opens interesting research directions for sports highlight commentary generation. However, the paper lacks details of some components of the benchmark model. The missing details reduce the validity and reproducibility of the experiments performed. Therefore, the paper is rated as borderline accept. See also the details in the following Strengths and Limitations sections.
strength_weakness: strength_weakness: - The paper has constructed a novel dataset for automatic commentary generation. The dataset brings new challenges that need to generate emotional and descriptive commentary compared to the existing video caption dataset such as MSR-VTT and FSN. The details of the data collection process and data statistics are well described in the paper. 

- The paper has conducted a benchmark on the proposed dataset, and clarified the shortcomings of the baseline models through the error analysis. Since the baseline methods are hard to generate the emotional and descriptive commentary in the proposed dataset, there is room for improvement in the performance of the model for the sports highlight commentary generation task, which opens the new research direction for subsequent studies.
limitations: limitations: - Missing details of some components in the proposed benchmark model.  
The benchmark model includes a prompt embedding updating scheme and Language Decoder Head. For the prompt embedding updating scheme, Figure 5 and 6 look inconsistent. Figure 6 in the paper shows that the prompt embedding updating scheme takes learnable prompt $P$ and multi-modal feature $E_m$ to output $P^{\prime}$ while $P^{\prime}$ is generated from only $P$ input in Fig. 5.  The paper obscures the end-to-end learning pipeline with prompt embedding updates. Also, there are no details of Language Decoder Head in the paper. 

- The same evaluation metrics are used for commentary generation as for conventional Video Captioning.  
The paper mentions that the propsoed dataset tends to provide emotional and descriptieve commentary as the ground truth. However, such commentary can be expressed in a variety of ways. Since the evaluation metrics used in existing video captioning tasks, such as BLUE, focus mainly on the presence or absence of expected words, we are concerned about whether the emotional and descriptive commentary generation can be evaluated appropriately. There is room to consider appropriate evaluation metrics for the novel dataset.

There are some typos such as fist -> first in line 450. The authors should check and correct typos throughout the paper.
paper_summary: paper_summary: This paper proposes a BH-Commentary dataset for sports highlight commentary generation and proposes a model, that integrates visual feature extraction, multi-model feature encoding, and commentary generation task by a unified framework. The effectiveness of the proposed model is verified through experimental evaluation of the BH-Commentary dataset.
strength_weakness: strength_weakness: The strengths of this paper include:
1. This paper proposes a descriptive basketball highlight dataset for automatic commentary generation.
2. This paper builds a benchmark model.
3. This paper analyzes the experimental results.
limitations: limitations: Review and Evaluation of Weaknesses:
1. "Anonymous Authors" seems to have been deleted in the paper template.
2. The validation analysis of the proposed model is insufficient. Table 3 only compared with four baselines and one ablation method.
3. The figures of the paper are confusing. The identifiers in Figure 5 use p_1, v_1, t_1 and are not consistent with the writing method in the text (video: f, text: s), and the expression is unclear.
4. Figure 7 & Figure 8 Except for the language version, it seems that no more additional information is introduced, which is redundant.
paper_summary: paper_summary: This study introduces a novel dataset aimed at automatic sports commentary generation. It includes around 4,000 basketball highlight videos, each paired with professional ground-truth commentaries. Furthermore, an end-to-end framework has been developed for this purpose, employing a lightweight and efficient prompt strategy. Experimental results validate the proposed dataset and confirm the effectiveness of the framework.
strength_weakness: strength_weakness: 1. A high-quality dataset for generating sports commentaries is introduced. 
2. The proposed model outperforms current state-of-the-art methods across various metrics. 
3. The writing and organization are well-executed.
limitations: limitations: 1. The dataset's diversity is limited because it includes only one type of sport, which means its generalization ability is not guaranteed.
2. The authors claim that the proposed dataset provides more emotional and descriptive commentary compared to FSN [71]. More statistics and examples are needed to validate this claim.
3. While qualitative results of the baseline model are provided, results from existing methods are also necessary to demonstrate the superiority of the proposed method.
4. The performance of the proposed model on existing datasets is not discussed.
5. There is no failure case study or discussion of the model's limitations.