HOTVCOM : Generating Buzzworthy Comments for Videos (2024)

Abstract: In the era of social media video platforms, popular “hot-comments” play a crucial role in attracting user impressions of short-form videos, making them vital for marketing and branding purpose. However, existing research predominantly focuses on generating descriptive comments or “danmaku” in English, offering immediate reactions to specific video moments. Addressing this gap, our study introduces HOTVCOM, the largest Chinese video hot-comment dataset, comprising 94k diverse videos and 137 million comments. We also present the ComHeat framework, which synergistically integrates visual, auditory, and textual data to generate influential hot-comments on the Chinese video dataset. Empirical evaluations highlight the effectiveness of our framework, demonstrating its excellence on both the newly constructed and existing datasets.

## introduction with the increasing prevalence of video content on digital platforms, there is an evident significance of video comments in amplifying video reach. specifically, "hot-comments" have the potential to attract considerable user interaction, substantially increasing a video's user impressions, which is essential for product marketing and branding. a typical hotcomment often meets specific standards: receiving a larger number of likes and replies, being highly pertinent to the video content, and including the elements that resonate with viewers, as illustrated in fig. 1 . however, the prevailing literature, such as the work by @xcite and @xcite , largely concentrates on generating descriptive comments or "danmaku", which are closely tied to certain video moments, offering instantaneous reactions. while these comments do engage viewers to some extent, their potential to highlight the entire short-video content or foster deep user interactions is somewhat limited. when aiming to elevate a video's visibility, these real-time reactions might not be as influential as those hot-comments. additionally, most research, including that by @xcite , predominantly focuses on english comments, leaving a gap in the domain of chinese hot-comment generation. in this paper, we construct the first chinese video hot-comment generation dataset hotvcom, including video titles, descriptions, captions, audio speeches, keyframes, and engagement records. compared to the english dataset from @xcite , our dataset hotvcom is more comprehensive, encompassing 93k videos with 137 million comments, making it the largest of its kind. in addition, for the convenience of analysis and targeted generation of comments, videos in hotvcom are further categorized into different themes. such a categorization helps language models to understand the context, therefore generating hot-comments that resonate with specific themes accordingly. a key challenge in video hot-comment generation lies on assessing whether the generated comments are truly impressive to users to boost the interaction. the existing work primarily utilizes rouge @xcite or the number of likes @xcite as metrics. however, these metrics might not always reflect the genuine engagement of a comment. in this paper, we propose a novel comprehensive evaluation metric including the informativeness, relevance, creativity besides user engagement (i.e. likes and replies) of a comment, which thus reveals a well-rounded understanding of a comment's 'hotness'. with the guidance of our evaluation metrics, we then propose a novel video hot-comment generation framework named comheat. we implement the supervised fine-tuning technique to generate preliminary comments and then enhance them with reinforcement learning. by incorporating knowledge-enhanced tree-of-thought method, the comments are further refined to improve the chance of their popularity. in summary, our contributions are: ## datasets in this section, we construct a large-scale chinese short video comment dataset named hotvcom including 94k short videos from douyin with 137 million comments, where the entire process is shown in fig. 2 . we also conduct an extensive exploratory data analysis as shown in fig. 5 . we initially collect douyin videos from various themes in reverse chronological order up till 100k. next, we conduct optical character recognition (ocr) with paddleocr library foot_0 and automatic speech recognition (asr) with xunfei open platform foot_1 on the videos to obtain their video captions and audio speech, respectively, and capture key frames of videos with the k-means clustering algorithm. after that, we also adopt paddleocr to extract the video's title, creation time, publishers' profile and engagement information, which includes the number of likes, comments, shares, and favorites, as well as each comment with its content, commenter's profile and engagement information, which includes the number of likes and replies. furthermore, from the tags marked with "#" in titles and themes provided by the douyin platform, we categorize the videos with gpt-4 foot_2 into 20 themes, including pets, food, etc. we also provide the descriptions for the video content and keyframes, encompassing scenes, objects, primary actions, atmosphere, and emotions with the help of video-chatgpt @xcite and minigpt4 @xcite , respectively. more details are shown in appendix a. ## evaluation we develop comprehensive evaluation metrics for hot-comments from four main aspects: informativeness, relevance, creativity, and user engagement. the informativeness score i quantifies the utility of the information conveyed by a comment from lengths penalty and vocabulary diversity. lengths penalty, denoted as l p , quantify the lengths appropriateness of a comment. vocabulary diversity, denoted as v d , is calculated as the ratio of total bigrams to unique ones of this comment. the calculation process of informativeness score is as follows: com and vid are the representation of a comment and the corresponding video, respectively. w r 1 and w r 2 are trainable weights. for the robustness of the evaluation by chatgpt, we repeat the process five times to obtain the most common result. we find that the consistency rate across the five iterations reaches over 80%. additionally, we randomly sample 1000 cases and confirm that the results indeed align with our definition of keywords. the creativity score c offers a quantitative assessment of a comment's distinctiveness and novelty, which encompasses two primary metrics: the rhetorical technique score, denoted as s r , such as metaphors and irony, and the trending term score, denoted as s t . the calculation process of the creativity score as follows: the user engagement score u represents the level of users' interaction of a comment, which mainly includes likes and replies. the calculation process of user engagement score is as follows: finally, the comprehensive score f of a comment is defined as: furthermore, we manually score each metric (informativeness, relevance, creativity, and user engagement) with a binary 0 or 1. we set the threshold at 0.5 for each metric and use the auc (area under curve) to calculate the consistency between manual scoring and automatic scoring for each metric. we find that auc of informativeness equals 0.83, auc of relevance equals 0.86, auc of creativeness equals 0.87, and auc of user engagement equals 0.80. it suggests that the alignment of each metric with human judgment. ## methods we propose a chinese video hot-comments generation framework named comheat as shown in fig. 3 . initial comments are first generated by the llms through supervised fine-tuning. then we train a reward model based on the comprehensive score of comments and adopt reinforcement learning to refine the popularity of the generated comments. finally, we utilize knowledge-enhanced tree-of-thought method for further optimization to generate hotter comments. ## experiments ## conclusions and future work in conclusion, this study underscores the crucial role of "hot-comments" in enhancing video visibility, setting them apart from the prevalent "danmaku" comments. through the introduction of the extensive chinese video hot-comment dataset hotvcom and the innovative comheat framework, we offer a novel approach to generate relevant and engaging video comments. in the future, we plan to ensure the ethics and fairness of the comheat framework from equal access and fair algorithms. we will provide low-cost or free tools and services, enabling smaller brands to enhance their video visibility. we also intend to include even more categories, especially those that are more niche. moreover, we will explore crosslingual perspectives for video comment generation.