Generating Hashtags for Short-form Videos with Guided Signals
Year: 2023

Abstract:
Short-form video hashtag recommendation (SVHR) aims to recommend hashtags to content creators from videos and corresponding descriptions. Most prior studies regard SVHR as a classification or ranking problem and select hashtags from a set of limited candidates. However, in reality, users can create new hashtags, and trending hashtags change rapidly over time on social media. Both of these properties cannot be easily modeled with classification approaches. To bridge this gap, we formulate SVHR as a generation task that better represents how hashtags are created naturally. Additionally, we propose the Guided Generative Model (GGM) where we augment the input features by retrieving relevant hashtags from a large-scale hashtag pool as extra guidance signals. Experimental results on two short-form video datasets show that our generative models outperform strong classification baselines, and the guidance signals further boost the performance by 8.11 and 2.17 absolute ROUGE1 scores on average, respectively. We also perform extensive analyses including human evaluation, demonstrating that our generative model can create meaningful and relevant novel hashtags while achieving state-of-the-art performance on known hashtags 1.

heading: 1 Introduction
text: Short-form videos on social media are increasingly popular thanks to the proliferation of multimedia technologies and portable devices (Vandersmissen et al., 2014; Montag et al., 2021). To highlight the topics and salient information of the videos, hashtags – words or unspaced phrases prefixed with a “#” – have been widely used. Proper use of hashtags can also increase the probability of the videos being discovered (Cao et al., 2020). In light of this, short-form video hashtag recommendation
∗ Work is done during an internship at Meta. 1The code is released at: https://github.com/
facebookresearch/hashtag-generation
(SVHR), which aims to suggest relevant and meaningful hashtags to content creators when they share videos, has received considerable attention from industry and academia (Li et al., 2019; Jain and Jindal, 2020; Mehta et al., 2021). However, most previous studies on SVHR consider it as a classification problem and rank the hashtags in a small and fixed-size set one by one (Li et al., 2019; Wei et al., 2019; Cao et al., 2020; Yang et al., 2020). These methods are time-consuming and far from the actual application, where users are free to create new hashtags, and trending hashtags change rapidly on social media platforms.
To fill this research gap, we formulate SVHR as a generation task that better represents the process through which hashtags are created by content creators. Figure 1 shows an example of the generation results that include a novel hashtag (#cold-weather). The generative model learns to generate hashtags related to video frames as well as video description.
9482
Additionally, we propose to retrieve hashtags from a large hashtag pool to augment input features and use the retrieved hashtags to guide hashtag generation. Inspired by the effectiveness of visionlanguage models (VLMs) (Radford et al., 2021; Jia et al., 2021; Fürst et al., 2022) in video-text retrieval tasks, we construct our hashtag retriever based on VLM. Then, we build a multimodal hashtag generator to generate hashtags from the retrieved hashtags, video frames, and user-written video descriptions. To leverage multimodal inputs, we introduce a cross-modal attention mechanism (CAM) to fuse information from different modalities. We name the whole architecture Guided Generative Model (GGM).
We conduct experiments to evaluate strong classification baselines, generative models and the proposed GGM on two well-known short-form video datasets: SFVD1 and SFVD2. For the classification models, we regard SVHR as a multi-label classification problem and compute probabilities over all hashtags that appear in training set by a softmax activation so the models can capture the interaction between the labels of each video. Experimental results demonstrate that the generative models outperform the classification models, and the guidance signals further boost the performance by 8.11 and 2.17 ROUGE-1 scores on average. We further create an unseen test set (Simig et al., 2022) for SFVD1 and analyze the models’ performance on it. Results show that generative models are able to generate unseen hashtags that the classification models can never predict. In addition, we assess the generated hashtags with human evaluation since the automatic metrics might underestimate our models’ ability to create novel hashtags. The results from our human evaluations show that GGM is able to create meaningful novel hashtags that are statistically comparable to the ground truth hashtags.
Our contributions are summarized as follows:
• We are the first to formulate SVHR as a generation task that better represents how hashtags are created naturally. We propose the Guided Generative Model (GGM), which leverages the retrieved hashtag to augment the input for hashtag generation.
• We present an extensive analysis of experimental results, including human evaluation, and demonstrate that GGM achieves stateof-the-art performance on two large-scale datasets (SFVD1 and SFVD2).
• Our work benchmarks classification and generative models on SVHR datasets and highlights the advantage of generative approaches, which we hope will catalyze research in this area.

heading: 2 Related Work
text: Short-form Video Hashtag Recommendation. Li et al. (2019) introduced the SVHR task and used graph convolutional network to deal with long-tail hashtags. Several works leveraged user information in SVHR (Wei et al., 2019; Liu et al., 2020). Yang et al. (2020) proposed incorporating sentiment features when recommending hashtags. Cao et al. (2020) focused on modeling the multimodal information of short-form videos. However, most of the previous works consider the SVHR as a binary classification problem and select hashtags from limited candidates by computing the recommended scores one by one (e.g., 101 candidates (Yang et al., 2020; Cao et al., 2020) and 1001 candidates (Wei et al., 2019)). Generally, these approaches are timeconsuming and not practical for real-world applications. Therefore, we formulate SVHR as a generation task which better represents how hashtags are generated naturally by users.
Keyphrase and Microblog Hashtag Generation. Keyphrase generation (KPG) aims to generate phrases that highlight salient information for a piece of text. According to (Meng et al., 2021), existing KPG methods can be divided into One2One (Meng et al., 2017) which generates one keyphrase at a time and One2Seq (Yuan et al., 2020) which generates a sequence of keyphrases at once. In this work, we apply One2Seq framework and randomly shuffle the target hashtags in each batch to mitigate the effect of order when fine-tuning the model. Different from KPG, SVHR requires the model to process multimodal inputs. Recently, Wang et al. (2019b) introduced the microblog hashtag generation task, which can be viewed as a variation of the KPG for the social media domain. Additionally, topic-aware models (Wang et al., 2019a) and news articles (Zheng et al., 2021) are leveraged to improve hashtag generation. To our knowledge, we are the first to generate hashtags for short-form videos.
Retrieval-Augmented Generation. RetrievalAugmented Generation (RAG) has been widely used in NLP tasks, such as neural machine transla-
tion (Gu et al., 2018; Hossain et al., 2020), opendomain question answering (Lee et al., 2019; Guu et al., 2020; Lewis et al., 2020b) and knowledgegrounded dialogue generation (Lian et al., 2019). Recently, some works also utilized this framework in multimodal tasks. Zhang et al. (2021) proposed a Retrieve-Copy-Generate (RCG) model for openbook video captioning. To tackle the Outsideknowledge visual question answering task, (Gao et al., 2022) transformed the image into plain text, performed knowledge passage retrieval, and generated answers entirely in the natural language space. This work extends the RAG framework to multimodal hashtag generation. Both hashtag retriever and generator can accept any advanced models as drop-in replacements.

heading: 3 Methodology
text: In this section, we first introduce our problem definition of the SVHR task. Then, we present our Guided Generative Model (GGM) which generates the hashtag from multimodal inputs.

heading: 3.1 Problem Definition
text: The main objective of SVHR is to generate recommended hashtags given a short-form video and its user-written textual description. To enrich the input signals, we construct a large-scale hashtag pool
as the knowledge base. Note that recommended hashtags should not be limited to the hashtag pool since meaningful novel hashtags are also considered valuable. Formally, the visual information of the video is formulated as frames F , and A denotes the acoustic information of the video. The textual description is defined as a sequence of words D = (d1, ..., d|D|). K is the hashtag knowledge base, and all hashtags in the training set are included by default. We do not include hashtags from external sources since the hashtag styles can vary widely across datasets, as shown in Figure 4. Finally, the models need to recommend the optimal hashtags Y by finding:
arg maxProb(Y |F,A,D,K; θ) (1)
where θ is the set of trainable parameters.

heading: 3.2 Guided Generative Model
text: Figure 2 depicts the architecture overview of GGM, which consists of a VLM-based hashtag retriever, a video encoder, a text encoder and a text decoder.
VLM-based Hashtag Retriever The goal of the hashtag retriever is to find the top-k most relevant hashtags from the hashtag knowledge base K given a video. Inspired by the recent improvements in video-to-text retrieval (Portillo-Quintero
et al., 2021; Luo et al., 2021; Fang et al., 2021), we built our hashtag retriever based on visionlanguage models. The hashtag retriever applies a Bi-encoders framework (Figure 3). The text encoder maps all hashtags in the pool to a list of hashtag representations T = (t1, ..., tj). The vision encoder calculates the frames’ embedding (w1, ...,wm) of each video, where each frame’s embedding wm comes from the representation ([CLS]) token of the vision encoder outputs. Similar to (Luo et al., 2021), the video representation is calculated by adopting a mean pooling mechanism to aggregate the embeddings of all frames, ŵ = mean-pooling(w1, ...,wm). Then, the similarity score sim between the video representation ŵ and each hashtag representation tj is computed by:
sim = tj
T ŵ
∥tj∥ ∥ŵ∥ (2)
The hashtags with top-k similarity scores are selected as the guidance signal for our generative model’s input.
To train the hashtag retriever, we adopt the pretrained VLM (Radford et al., 2021) to initialize the model and follow the same contrastive learning loss. Each training sample consists of video frames and one hashtag corresponding to the video.
Audio-Grounded Video Encoder The audiogrounded video encoder creates the video representation given the audio and frames from the video. We employ a Transformer-based audio encoder (Wu et al., 2022) to encode the sound of the video. The acoustic information is mapped to an audio feature a, and the audio feature will be used directly as the audio input to the audio-vision fusion mechanism. For the video frames, we use a
N layers Vision Transformer (ViT) (Dosovitskiy et al., 2020) to encode them. Each frame is divided into patches and encoded as a frame embedding wm ∈ Rn×d, where n is the number of patches. All video frame embeddings are concatenated to build the visual representation w ∈ Rnl×d.
After that, a cross-modal attention mechanism (CAM) is applied to get the audio-grounded video representation v (Eq. 3). CAM is based on the multi-head attention module in Transformer architecture (Vaswani et al., 2017). The query is linearly projected from the audio feature, and the key and value are linearly projected from the visual feature. In addition, we conduct residual connection (He et al., 2016) between the visual representation and the video representation.
v = CAM(wWq,aWk,aWv) +w (3)
Video-Grounded Text Encoder The videogrounded text encoder takes the video representation and text (i.e., user-written description and guidance signal) as the input to produce the videogrounded text representation. We employ a N layers Transformer text encoder to get text representations. Each layer consists of a bi-directional self-attention sub-layer and a fully connected feedforward network. In order to input both the description and the guidance signal to the text encoder, the input text is formatted into “description [sep] guidance signal”, and the output is a text embedding t ∈ Rk×d, where k denotes the number of input tokens. Similar to the audiogrounded video encoder, we use CAM to fuse the video and text representations (Eq. 4). Finally, the original text representation t is residual connected to the video-grounded text representation t′.
t′ = CAM(tWq,vWk,vWv) + t (4)
Video-Grounded Text Decoder We construct an N -layers video-grounded text decoder following the standard Transformer text decoder. For each layer in the decoder, the bi-directional selfattention is replaced with causal self-attention. Meanwhile, an additional cross-attention sub-layer is inserted to perform multi-head attention over the video-grounded text representation. The decoder generates hashtags token by token. A beginningof-sequence (BOS) token is used to indicate the start of decoding, and an end-of-sequence (EOS) token is used to signal its end. In addition, we use a separator token to separate the generated hashtags.

heading: 4 Experimental Settings

heading: 4.1 Datasets
text: We evaluate our models on two well-known large-scale short-form video datasets, SFVD1 and SFVD2. After filtering out the videos that only have hashtags occurring lower than five times, we obtain 95,265 short-form videos for SFVD1 and 312,778 for SFVD2. The statistics of both datasets are shown in Table 1. Note that all videos in SFVD1 are shorter than seven seconds and videos in SFVD2 are shorter than 90 seconds. For SFVD2, we regard the user tags as the hashtags in our experiments. We randomly split the datasets into three disjoint subsets with 80%, 10% and 10% of the data for training, validation and test sets, respectively.
In order to simulate the real-world scenario where new and/or trending hashtags emerge, we construct an unseen test set for SFVD1 with two steps similar to (Simig et al., 2022): (1) Choose 500 hashtags that appear in the training split, and (2) Move all samples in the original training and test set that contain any of these 500 hashtags to the unseen test set2. Finally, the SFVD1 dataset covers 69,539 samples for training, 9,826 for validation, 8,690 for seen testing and 10,210 for unseen testing. Note that seen hashtags could also appear in the unseen test set samples because videos have multiple labels, and we only ensure that at least one unseen hashtag exists in each sample of the unseen test set. The numbers of seen and unseen hashtags are 10,104 and 570, respectively.

heading: 4.2 Implementation Details
text: For the video pre-processing, we extract frames with a frame rate of 2fps if the video duration is less than seven seconds and uniformly sample 15 frames if the video is longer than seven seconds. ffmpeg is used to extract the audio as a WAV format file from the video. The retrieval pool of hashtags contains all hashtags in the training set.
2Due to strong correlations between labels, there are additional 70 hashtags removed together with the 500 hashtags from the training set and added to the unseen hashtag set.
We further evaluate the unseen hashtags to test the retriever’s generalization ability as shown in Table 4. For the GGM, we initialize the video encoder with the ViT-base model and construct the text encoder-decoder based on the BART-base model. During decoding, we use beam search with a beam size of 5. The decoding process will not stop until the end-of-sequence token is emitted or the length of the generated hashtags reaches to maximum length. We set the maximum length as 32 for SFVD1 and 64 for SFVD2. See Appendix A for more implementation details.

heading: 4.3 Baselines
text: The following baselines are implemented for comparison: 1) BERT (Devlin et al., 2019) takes video description as input for classification. 2) VLM is the same as our VLM-based Hashtag Retriever. 3) ViT (Dosovitskiy et al., 2020) takes video as input for classification. 4) ViT-BERT concatenates the video and description embedding for classification. 5) BART (Lewis et al., 2020a) generates hashtags from the video description. 6) Trocr-fid (Li et al., 2021) generates hashtags based on the video. we apply the fusion-in-decoder (Izacard and Grave, 2021) strategy to let the model accept multi-frame input. 7) VG-BART (Yu et al., 2021) takes video and description to generate hashtags. We replace the visual feature extractor from 3D ResNet (Hara et al., 2017) to ViT for a fair comparison with GGM. Appendix A describes the details of the baseline implementation.
Similar to (Gong and Zhang, 2016; Mahajan et al., 2018), we build multi-label classifiers by minimizing the cross-entropy between the predicted softmax distribution and the target distribution. The reason we do not include previous binary classification approaches on SVHR (Yang et al., 2020; Cao et al., 2020; Wei et al., 2019) is three-fold: (1) They select the hashtags one by one from a pre-defined relatively small number of candidates (e.g., 101). In contrast, we only give the hashtags in training set as prior information. (2) For each test sample, the models need to compute a score for each video-hashtag pair across all hashtag candidates, which is time-consuming and far from the actual application. (3) Multi-label classifiers can encode interrelation among one video’s hashtags, which binary classification approaches cannot.

heading: 4.4 Evaluation Metrics
text: We adopt ROUGE metrics (Lin, 2004) that were initially used for summarization evaluation since we consider SVHR as a generation task. ROUGE1 and ROUGE-2 F1 are used to measure the ngram overlaps. We include ROUGE-2 because some hashtags are combinations of words. We do not employ ROUGE-L because the order of hashtags should not affect the evaluation. The F1 score is used to calculate the exact match of the hashtags between the labels and predictions. In addition, we include the BERTScore (Zhang et al., 2019) to compute the semantic similarity of the hashtags. More details of the evaluation metrics are in Appendix B.1.

heading: 5 Results and Analysis

heading: 5.1 Main Results
text: Effectiveness of Generative Models Table 2 shows the performance of the classification and generation models on the SVHR task. When we only use descriptions as input, BART achieves higher scores than BERT across all metrics. This could be because the BART decoder can better capture the textual information from the input description compared to the classification layer in BERT. On the contrary, ViT surpasses Trocr-fid when only taking videos as input. We speculate that the text decoder of Trocr-fid may lose some visual information through the cross-modal transformation from vision to text while ViT directly maps the video to hashtag labels. When taking both video and
description as input, VG-BART achieves better performance than ViT-BERT. Most importantly, GGM performs significantly better than VG-BART (the SOTA multimodal summarization model) with the help of the guidance signals. Surprisingly, the audio information does not improve the performance much, especially on the SFVD2. It could be because lots of the audio is just random background music and more than 5% of the videos in SFVD2 do not have audio. Additionally, to verify that the reason of the low performance of the classification models is not because we choose top five hashtags, we evaluate ViT-BERT with different numbers of hashtags as final results. Table 3 shows that VG-
BART outperforms ViT-BERT with all different settings, indicating the superiority of using generative models for SVHR.
SFVD1 versus SFVD2 We find that the models generally perform better on SFVD2 than SFVD1 except for ROUGE-2 scores (Table 2). There are two possible explanations. First, as shown in Figure 4, most of the hashtags in SFVD2 are made up of one word (e.g., cat, youth, family), while the hashtags in SFVD1 usually contain multiple words (e.g., #furrypride, #TheRealFamily, #dontjudgeme). It’s easier for models to generate singleword hashtags and thus perform better on SFVD2. And due to the small proportion of multi-word hashtags in SFVD2, ROUGE-2 drops because it calculates the overlap of multi-word hashtags. Second, there are more hashtags (20.94% versus 4.48%) that appear in the corresponding video description of SFVD2 than SFVD1. Thus, the better model performance on SFVD2 could be partially attributed to the fact that it is easier for the models to extract words from the description as hashtags.
VLM Retrieval versus ViT Classification It’s surprising that the performance of VLM is significantly lower than ViT on both datasets. To explore this phenomenon more deeply, we test how ViT and VLM perform discrepantly among varying topk. Table 4 illustrates the recall of the models on the SFVD1 test set with different k. We can see that the performance gap between VLM and ViT
decreases as k increases. We speculate it is because ViT is optimized by cross-entropy loss and therefore learns to predict a sharp distribution, while VLM applies a contrastive loss and outputs a more flattened distribution. The sharp distribution lets ViT perform better when k is small, and the flattened distribution makes VLM more stable when k grows. We further discuss the effectiveness of VLM retrieved hashtags in Section 5.2.

heading: 5.2 Effectiveness of Guidance Signal
text: To explore how different guidance signals can affect generation performance, we first use hashtags selected by ViT to create the ViT-Guided Generative Model (GGMV iT ) as a comparison of the GGM. As we can see from Table 3, the performance of GGMV iT is lower than the GGM when using the top-50 hashtags as the guidance, which contradicts the recall of ViT and VLM, as shown in Table 4. We conjecture that VLM-based Hashtag Retriever is trained on a contrastive learning loss which makes it provides more robust features for the next step generative model. Additionally, Table 3 depicts the performance over different numbers of hashtags in the guidance signal. Note that the result of GGM k=0 can be regarded as an ablation study of our approach without the guidance signal. We find that GGM guided with top-50 hashtags outperforms others. We speculate that fewer hashtags reduce the information in the guidance signal, while numerous hashtags will introduce extra noise to the model.

heading: 5.3 Performance on Unseen Test Set
text: To simulate the new trending hashtags, we construct an unseen test set for SFVD1 and investigate the models’ performance on this test set. Since classification models will never predict unseen hashtags, we only evaluate the generative models for comparable performance. Table 2 and 5 show that both VG-BART and GGM perform much worse
in the unseen test set than in seen test set. When we add the unseen hashtags in the hashtag retrieval pool for creating the guidance signal, GGMunseen achieves better scores in all metrics. As discussed in Section 4.1, seen hashtags could also appear in the unseen test set due to the strong correlations between labels. To explicitly investigate the model’s performance on those unseen hashtags, we calculate the number of unseen hashtags recalled at least once. Results show that GGM has recalled 13 unseen hashtags at least once. When we include the unseen hashtags in the hashtag retrieval pool, GGMunseen recalls 51 unseen hashtags at least once. There are still more than 90% of the unseen hashtags which have never been recalled, indicating that generating unseen hashtags is challenging even with the guidance signal.

heading: 5.4 Novel Hashtag Analysis
text: One advantage of the generative models is that they can create novel hashtags that never appeared in the training set. These novel hashtags are valuable because they increase the diversity of the hashtag recommendation and could become new trends in the social media platform. However, our quantitative results focus on word overlap, which might underestimate the effectiveness of the novel hashtags. Thus, we conduct a case study and human evaluation on the generated novel hashtags.
Case Study We present a case study of the hashtags generated by GGM, shown in Figure 5. It is clear that the model can capture the video and its description to generate novel and meaningful hashtags. For instance, GGM generates #willowtree based on the video description in the first case and creates #cold-weather relevant to the video frames in the second case. Neither of the novel hashtags was in the training set, but they are still meaningful and relevant to the video and description.
Human Evaluation Novel hashtags should be understandable and relevant to the video and description so that users can easily access the correct information. Similar to (Simig et al., 2022), we randomly sample 100 novel hashtags created by GGM and manually evaluate their understandability and relevancy. For a fair comparison, we include and mix 100 ground truth hashtags from the same videos in the human evaluation. Assessments are scored on a scale of one to five, with higher scores being better. Each sample is evaluated by three people, and we average the three scores as the final result. As illustrated in Table 6, both generated novel hashtags and ground truth hashtags achieve high understandability scores (larger than four), indicating that hashtags created by our model are meaningful. Interestingly, our generated hashtags are significantly more relevant to the corresponding video and its description with p-value<0.05. Further analysis shows that ground truth hashtags consist of many generic hashtags such as “#funnny”, “#follow” and “#remake”, which are not closely related to the specific video content. In contrast, the generated novel hashtags can capture more details of the video, better representing the salient information in the video. Hence, our GGM is able to generate novel and meaningful hashtags to improve the diversity of recommended hashtags.

heading: 6 Conclusion and Future Work
text: In this paper, we formulate the short-form video hashtag recommendation (SVHR) as a generation task, and we propose a Guided Generative Model (GGM) that generates hashtags from multimodal inputs and guided signals from a VLM-based Hashtag Retriever. Our work benchmarks classification and generative models on SVHR datasets and highlights the advantage of using generative models. Our GGM achieves state-of-the-art performance, and the human evaluation results show that GGM is able to generate meaningful novel hashtags compa-
rable to ground-truth hashtags. We hope our work can catalyze research on using generative models for SVHR.
For future work, since the hashtag recommendation is a highly concurrent task in the real-world application, we believe improving computational efficiency to strike a balance between accuracy and efficiency is one of the important directions. Besides, the popular trend of short-form videos changes rapidly on the internet, so it’s important for systems to accurately generate hashtags for new trending videos.

heading: 7 Limitations
text: Our methods are currently trained and tested on two SVHR datasets. Gender biases and unethical hashtags could exist in the datasets, which may cause the model trained on these datasets to generate these biases. Besides, although our methods are not language-specific, we only choose the English dataset due to its rich resource. Furthermore, we regard the user tags as the hashtags for SFVD2 in our experiments and there are small differences between the user tags and hashtags. Experiments on more diverse languages and datasets are needed in the future.
As an initial work for SVHR, in our task formulation, our model only take the video and its description as input to predict hashtags and ignore user preference in hashtag recommendations. Exploring user preference is also a promising direction for future work.
We used up to eight A100 GPUs per experiment and it took more than one day to run experiments on SFVD2. More efficient models are needed for real-world applications. Besides, we hope our experimental results can be used as benchmarks for comparison in future work to avoid repeating training.

heading: 8 Ethics Statement
text: Although the generative models can create novel hashtags, which is beneficial for increasing hashtag diversity on social platforms, generative models also have the potential to generate toxic or offensive hashtags. Since we finetune the Pre-trained language models (PLMs) on existing SVHR datasets, undesirable hashtags could come from biases that are encoded in the PLMs (Blodgett et al., 2020) or the undesirable hashtags in the training set. We recommend that when generative models are de-
ployed in real-world applications, additional postprocessing should be carried out to remove undesirable and harmful hashtags. In addition, our hashtag generator will only recommend hashtags to human content creators who ultimately have the responsibility to decide which hashtags should be used.

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section 7 "Limitation"

heading: 3 A2. Did you discuss any potential risks of your work?
text: Both Section 7 "Limitation" and Section 8 "Ethics Statement"

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Yes, we summarize the paper’s main claims in "Abstract" and Section 1 "Introduction"
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? We create NLP models in Section 3 "Methodology"

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 2 and Section 3 and Section 4

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: In Appendix C, we discuss the license of the dataset that we use.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 3, Section 4 and Section 5

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Appendix C

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 1, Section 2 and Section3
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. In Section 4.1 "Datasets" we discuss the data
C 3 Did you run computational experiments? Sections 4 and 5

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 4 and Appendix A
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 4

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4 and Appendix B1
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 5

heading: 3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? Appendix B2

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Appendix B2
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.

heading: 3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data?
text: Appendix B2

heading: Future Work
text: Experiments on more diverse languages and datasets are needed in the future. As an initial work for SVHR, in our task formulation, our model only take the video and its description as input to predict hashtags and ignore user preference in hashtag recommendations. Exploring user preference is also a promising direction for future work. We used up to eight A100 GPUs per experiment and it took more than one day to run experiments on SFVD2. More efficient models are needed for real-world applications. Besides, we hope our experimental results can be used as benchmarks for comparison in future work to avoid repeating training