MediaHG: Rethinking Eye-catchy Features in Social Media Headline Generation
Authors: Boning Zhang
Yang Yang
Venue: EMNLP 2023 Main (2023)
Keywords: headline generation，style-content attractiveness，social media
TL;DR: The paper is about an attractive blog headlines generation model: MediaHG with domain platform features.

Abstract: An attractive blog headline on social media platforms can immediately grab readers and trigger more clicks. However, a good headline shall not only contract the main content but also be eye-catchy with domain platform features, which are decided by the website’s users and objectives. With effective headlines, bloggers can obtain more site traffic and profits, while readers can have easier access to topics of interest. In this paper, we propose a disentanglement-based headline generation model: MediaHG (Social Media Headline Generation), which can balance the content and contextual features. Specifically, we first devise a sample module for various document views and generate the corresponding headline candidates. Then, we incorporate contrastive learning and auxiliary multi-task to choose the best domain-suitable headline, according to the disentangled budgets. Besides, our separated processing gains more flexible adaptation for other headline generation tasks with special domain features. Our model is built from the content and headlines of 70k hot posts collected from REDBook, a Chinese social media platform for daily sharing. Experimental results with language metrics ROUGE and human evaluation show the improvement in the headline generation task for the platform.

MediaHG: Rethinking Eye-catchy Features in Social Media Headline Generation
Boning Zhang and Yang Yang"
Zhejiang University
zhangbnezju. edu. cn and yangyałzju. edu. cn
Abstract
An attractive blog headline social media platforms can immediately grab readers and trigger more clicks. However, a good head- line shall not only contract the main content but also be eye-catchy with domain platform features, which are decided by the website's users and objectives. With effective headlines, bloggers can obtain more site traffic and profits, while readers can have easier access top- ics of interest. In this paper, we propose disentanglementt-aase
kodeer
fla Mabe Mediilii
Figure 1: Comparison and improvement with previous work. (a) Previous Seq2Seq models input the whole document generate headline, resulting in the omis- sion of the main topic and latent features. (b) In con- trast, MediaHG samples document views to capture the main topic and selects the best headline with eye-catchy budgets. Besides, the eye-catehy feature extractor is disentangled with content and style to achieve duality balance.
headline generation model: MediaHG (Social Media Headline Generation), which can balance the content and contextual features. Specifically, we first devise a sample module for various document views and generate the corresponding headline candi- dates. Then, we incorporate contrastive leamn- ing and auxiliary multi-task to choose the best comain-suitable headline, according to the dis- entangled budgets. Besides, separated pro- cessing gains more flexible adaptation for other headline generation tasks with special domain features. Our model is built from the content and headlines of 70k hot posts collected from REDBook, a Chinese social media platform for daily sharing. Experimental results with lan- guage metries ROUGE and human evaluation show the improvement in the headline genera- tion task for the platform!.
easier information search but also strengthen the community of users with the same interests. Users' attention is always limited to attractive headlines that can catch their eyes at first glimpse. As the headline condenses the main topie into a concise and appealing description, a good headline can trig- ger a high click rate. So generating better headlines is significant for media platforms to compete for attracting users' limited attention and deliver users better experiences.
We conduct the research based on vertical Chi- nese social media platform REDBook (Figure 2) because is widely praised by hundreds of millions of users and targeted by the same interest group. As more than 70% of users are female (an offi- cial reported data), the topics and tone of posts are feminine such as some typical words "Babycare", "Makeup" etc. The platform breaks down identity restrictions and allows people to share their col- orful lives and experience, which are of reference value for users. We define the eye-catchy features of posts as the ability to attract users, which can be intuitively measured by the number of "likes",
1 Introduction
Nowadays, in the midst of massive flow of infor- mation on large-scale social network sites (such Facebook and Instagram), users always feel more difficult quickly obtain the information they want. As a result, people tend to focus on more niche platforms, which are often made up of groups of users with common personal inter- ests. The vertical platforms are not only used for
"Conresponding author
'The code is available at https://gitth: com/ Mesenn2000/MediaHG

---

that is, the heat of posts on the platform. Producers can obtain more traffic and profits while getting more likes from the platform, then they may be subscribed by more fans. The advertising revenue of bloggers is usually closely related to both the number and profile of their followers. So only with the help of good headlines, can the bloggers attract their target user group.
pose the MediaHG model which disentangles the eye-catchy features as additional requirements in sequence-to-sqquene training. The model is com- posed of a headline candidates generator and an eye-catchy headline selector. In our setting, the neural abstractive model is responsible for head- line generation, capturing the main topic of the input document, while the selection module with constraint will encourage the adherence of gener- ated headlines to domain eye-catchy features. In- stead of confounding eye-catchy features, we treat the content feature and style feature extraction re- spectively. In the generation period, we devise a random sample module with different parts of the text and generate candidate headlines responding to the content. During selection period, we leverage ranking-baseed based contrastive learning(Hopkins and May. 201 () al., 2020)(Liu et al., 2021) and muuli-taskkLuonn et al., 2015) to select the best headlines among candidates. The selection is decided by the coordinating quality scores of style- content attractiveness. We will describe the specific quality metrics model in detail in the following part. Therefore, candidates are assigned with probabil- ities according their quality, which will further influence the generation model. In other words, the headline generation model not only generates out- put headlines autoregressively but also estimates the probability distribution over candidate head- lines.
By analyzing eye-catchy REDBook headlines with more likes (more than 2k) on the platform, we find that both contents and style influence attrac- tiveness. Since the majority of users are women, topics of interest to women occupy a large part of the topics on the platform. Accordingly, head- lines with domain special topics such as "Lipstick swatches" "ootd" ("Outfit of the Day") are more appealing with more likes. Combined with hot topics, the style of the headline also has great impact on eye-catchy features. For example, when reporting the hot issue "lipsticks sharing". the head- line "New Lipsticks for Winter So Tender like a Creamy Almond Peach!!" (as shown on the plat- form
# i iff T in Chinese) wins over 13,000 likes, while another headline with the same topic but a plain description "Lipstick Share, a New Style for Win- ter" only has 300 likes. We also find the eye-catchy style of the headlines on the platform is not that similar to the style of news media headlines, as more closely resembles the tone of women talking and sharing with their friends.
Our main contributions are listed below:
Focusing relevant works, we found that recent researches simply regard the headline generation task as a typical summarization task (Shu et al., 2018). They only focus on the content parallel to the given reference summary, ignoring the domain eye-catchy features. However, attractive headline- generation tasks have received less attention. A recent clickbait research (Xu al., 2019) lever- ages adversarial training and attractiveness scores module guide the summarization process. An- other (Jin et al., 2020) introduces a novel parameter- sharing scheme to disentangle the attractive style from the text. However, these previous works only concentrate on style and neglect the content impor- tance, which also weighs in eye-catchy headline generation. Disentanglement module is devised to divide the style and content into latent spaces, but style encoder in generator training not flexible enough(Li et al., 2021).
We propose new Headline Generation model namely MediaHG to generate topic-catchy and contextual harmonized headlines for ver- tical niche platforms to enhance the click ratio and draw users' attention. While we base our experiments on a typical vertical platform REDBook, our methods can be adapted to other platforms through the same platform- suitable features extraction methods.
Our model is proved be effective by both automatic and human evaluation scores of flu- ency, consistency, and attractiveness, which means it achieves a style-content dual balance.
To the best of our knowledge, it is the first re- search to focus vertical interest platforms. We also give a new definition of domain eye- catchy headlines that is, those attractive com- binations with topic and style suitable to the platform users.
To address the headline generation issue, we pro-

---

platform hom
tation and color) of images (Chen et al., (Hig) gins et al., 2017)(Luan et al., 2017). Compared the computer vision field, NLP tasks mainly treat sentiment as a salient style and focus on invariant representation learning. It is used to control sen- timent through training discriminator (Hu et al., 2017). Then, disentangled representation learn- ing is further widely adopted in nonparallel text style transfer. For example, separate training with style-specific embeddings and style-specific em- beddings are proposed(Fu et al., 2018). Some work also focuses on disentangling syntax and seman- tic representations in text. VGVAE(Chen et al., 2019a) trains the generative model with multiple losses that exploit aligned paraphrastic sentences and word-order information to get better syntax and semantics representations. We utilize the core prin- ciple of disentangling separate different feature budgets.
Figure 2: REDBook Homepage and Specific Hot Post Page Display. The layout of the browsing page shows only the posts' cover and the title, thus emphasizing the importance of the title.
2
Related work
Headline Generation Text generation has under- gone impressive progress in recent years (Li et al.. 2Chn) et al., 2019)(Liu et al., 2Xee)(Xie et al., 2Cha) et al., 2020), and headline gen- eration occupies the dominance. Most existing works merely focus on document summarization with extractive (Nallapati et al., 2017)(Zhou et al., 201 (Zhang et al., 2018) abstractive (Gao et al.. 2019) 201e)(Chen et al., 2019 m009lss Several meth- ods regard headline generation as a task based on length-controller text summarization. Controlling length in summaries has been addressed by encod- ing positional information (Takase and Okazaki, 2019), length-aware attention mechanism (Liu et al., 2022), and length constraint optimiza- tion(Makino al., 2019). Content guidance in GSum(Dou et al., 2020) is used as the input for its sequence-to-sequeee model, and shifts in guid- ance distribution would require further training. Attractive headline generation is paid less atten- tion by researchers. A sensation scorer(Xu et al., 2019) is designed to judge whether a headline is attractive and guide the headline generation by re- inforcement learning. Also, a parameter-sharing scheme(Jin et al., 2020) is introduced to further ex- tract style from the text. The style-content duality is considered with VAE (Variational AutoEncoder) as a feature extractor (Li et al., 2021) and two dis- entangled space constraints in parallel tasks. Dif- ferently, MediaHG allows flexible shifts in various guidance without expensive retraining. Disentanglement Disentangling neural networks' latent space has been explored in the computer vision domain to factorize the features (such as ro-
Reranking Candidates Recent conditional genera- tion work explores the idea of reranking candidates from different dimensions( Wan et al., 20i2) moto and Matsumoto, 2016). Different search methods have been used in neural language sum- marization models, such as greedy search in Fac- torSum (Fonseca et al., 2022) and beam search (Vi- jayakumar et al., 2016) in SimCLS (Liu and Liu, 2021 Jaccording to a learned evaluation function. The Sen-eeeanddSSelet summarizer (Oved and Levy, 2021) performs random perturbations and uses similar ideas generate candidates ranked cording a coherence model. Unlike only intrinsic importance compared with the original document in SimCLS, content and contextual eye-catchy bud- gets are both considered in our work.
eyt myeiei franure
traming peried
keihage
Medi
M-z
Inforence persed
baadiiee guurratee
the
Grame log
Figure 3: Overview of MediaHG. We divide our models into 3 parts: Sample Module for headlines candidates generation; Headline Generator with different datasets training and inference; Features Reranking to select the domain- best title according to disentangled style and content budgets.

---

3 Approach
containing various lengths of documents and titles. the hyperparameters may be tuned. According the basic REDBook platform format restrictions, the length of the document and title is limited to 1000 characters and 20 characters respectively. So we choose h candidate document views, each with 8f 2/3 sentences of the document. Different choices with appropriate values k are discussed in the ablation study.
In this section, we describe our approach in detail, as shown in Figure 3. Inspired by FAC- TORSUM (Fonseca et al., 2022), we treat the content importance model sampling document views(intrinsic importance), and contextual fea- tures as additional budgets(extrinsic importance). We pre-train the headline generation model with our dataset REDBook (Table 1) to generate candi- dates with intrinsic content and latent features in Scc3.1. Then, a specific metric M is employed to evaluate the effectiveness of different criteria, com- posed of scores from both content and domain style factors in Sec3.2. To assign higher probabilities to a more suitable candidate, we use contrastive learn- ing for better re-ranking. The metric M construc- tion is demonstrated in Sec3.3 which is optimized with multi-task loss.
Powerful sequence-to-sseuueeee PLMs models such as PEGASUS(Zhang et al., 2020) and BART (Lewis et al., 2019) are trained to estimate the prob- ability of a sequence of tokens by minimizing cross- entropy with respect to the data distribution. We hypothesize that these models generate good can- didates to fulfill the content importance and latent style features objectives, which are described be- low.
Learning Maximum likelihood estimation (MLE) is the standard training algorithm. Given the train- ing dataset X' consists of hot post document ( and reference headline H()). the loss is defined as negative log likeliiieiood function:
3.1 Generate Candidates
The headline candidates are generated from two tasks: document parts sampling and correspond- ing headline generation. The candidate document views are generated from different random samples article parts. We hypothesize that the main topic of short posts (limited to 1000 Chinese characters) shall be contracted from various sampled incom- plete parts. Using samples allows the sequence-to- sequence model to focus on concise and appealing topies, as we further considered.
X
1
L (Fint, =
L. (H Fint (0, Dis
X2
log (He (())
(I)
where pe (H))) D(0) is distribution over the possible headline H (Lewis al., 2019).
To generate multiple views for the same docu- ment, we implement the following steps:
For a specific sample { (D(i), H- Eq.2 is equivalent to minimizing the sum of negative log- likelihood of the tokens in the reference headline H whose length is 1, through the cross-entropy loss:
From a document D, we first split the sen- tences and generate a random sample collec- tion of sentences, called document views S,. The number of sentences of each document view in S. is controlled by the sampling pa- rameter sy E [0, 1], so that sents(Sg) se 8f sents(D). The number of samples [S,] is controlled with hyperparameter k.
Esent
(2)
Σ> Pime (b)) D.H22) log to (b) H2
J=1
where H22 denotes the partial reference head-
We generate the headline corresponding to each document view in S, with a pre-trained Sequence-to-sequee
line [ij. -.. , hi h Ptrue is defined as one-hot distribution under the standard MLE framework:
model. The model de-
faults to generate headlines with content and eye-catchy features. The headlines collections h1, h2n...nn) cover ny candidates inside.
{ 1 h h=
Puree (n) D.
(3)
h + h
For each document the RED-IN (Table 1), we repeat the sampling method and headline genera- tion work. While dealing with different datasets
During learning stage, we find the parameters A, minimize the loss above. Since the model is trained with a confounded feature dataset, we notice the

---

results are generated with both content and latent features.
where me score measures the topic components of candidate headline H4, extracted from the docu- ment and style score m, measures the contextual latent features. We fine-tune the model with con- trastive loss (Hopkins and May, 201 I ()hong et al., 2020) which encourages the model to assign higher probabilities to a more suitable candidate as fol- lows:
Inference During inference stage, the abstractive model g is used to generate the candidate headline in an autoregressive manner. It is intractable to enumerate all the possible candidate outputs, so methods such as beam search decoding(Suttskever et al., 2014) are used to reduce the search space. Estimating the probability of the next word h4 is the significant step during the search:
Letr ΣΣ iSi
(Ili + Ais)
max
Pge (he D, 0)
(4)
(7)
which is different from Eq.3 with its own previ- ous predictions resource He instead of reference headline Hz
where H; and Hj are two different candidate head- lines and M (Hs, H*) > M (Hj, H*), Vi. < j by metries M. Xaj is the margin multiplied by the difference in rank between the candidates. i.e.A.y = ()) A.
3.2 Coordinating Headline Selection
Eq.4 implics that the headline generation model g should be able to assign a higher estimated prob- ability to the better candidate summary during in- ference. However, this intuition is not directly captured in the standard MLE objective used in training. No option is adopted for the ordering of imperfect references, which will lead to the ex- istence of multiple generations KKhayralla al., 2020). Therefore, we propose the probability that one candidate should be well-correlated with its quality as evaluated by an automatic feature metric M. It is intractable to enumerate all the possible candidate outputs, so we only require an accurate prediction of the most probable candidate headlines ranking order via beam search (See Appendix A).
Following multi-task fine-tuning (Edunov al., 2017), we combine the contrastive (Eq.7) and cross- entropy (Eq.2) losses to preserve the generation ability of the pre-trained abstractive model:
Lmul = Lsent +
(8)
where is the weight of the contrastive loss. We note that the contrastive and the cross-entropy loss can effectively complement each other. Since the contrastive loss is defined the eye-catchy fea- tures, the token-level cross-entropy loss serves normalization to ensure content-style balanced probability assignment. This optimization loss of the result can be used in the two-stage summariza- tion pipeline.
We use label-smoothing (Szegedy al., 2016) and maintain the general functional form Eq.3, but specify the marginal probability of the non- reference candidates H to be B. Additionally, we encourage the coordination of probabilities and qualities among headline candidates by contrastive learning as follows:
3.3 Disentangled Space Constraint
The disentanglement scores framework shown in Figure 3 consists of content (intrinsic) constraints and appealing style (extrinsic) constraints.
Content Space Constraint As the above style- oriented loss has already imposed constraints on the style information, the content space constraints methods will be discussed in this part. Different from the style constraint design, it is hard to find parallel sentences with the same content but differ- ent styles. Previous work DAHG(Li et al., 2021) used the prototype document and its most simi- lar document to improve the classifier. However, the similarity precision is not clearly defined. The bag-of-words (BOW) method is proposed ap- proximate content information(Jonn et al., 2018) disentanglement in document style transfer tasks, but our generation objectives are concise headlines.
p(H D) - 1
H H* H + H VH,, H, E H. M (H.) > M (H,)
EMCHP(H D) =
(HI, D) > pl (1I D)
(5)
The candidate quality measure M in our work is defined with two scores: content score and style score which responsible for the topic extraction and contextual media style features, separately.
M(H.) = (H,) + (1 - aH,,(H.) (6)

---

Inspired by the original (BOW) method, we use ROUGE(Lin, 2004)scores to measure the main con- tent overlapping.
different bloggers with the consideration of avoid- ing personal-style influence. We randomly divide the REDBook dataset into train set, validation set, and test set. For the inference and selection of the best candidate task, we randomly choose 20k posts (RED-IN). Our dataset contains the hot post pub- lished during 2021, and time is not influencing factor the like counts have already accumulated.
Style Space Constraint We design a multi-task loss that ensures the style information is contained in the space S. Although our dataset for style ex- traction is non-parallel, we assume that each sen- tence is labeled with its style (with domain eye- catchy features or not). We select the eye-catchy headlines from the platform and other plain cor- pora sentences to train the style classifier, Follow- ing the previous work (Hu et al., 20hn)(Shen et al., 20u7)(Fu et al., 201 (Zhao et al., 2018) we treat each sentence with binary style tag (positive or negative).
Document
Title
Dataset
Size
Avg-Len
Avg-Sens
Avg-Len
RED-Tr 70k
148.5
11.7
16.4
RED-In
20k
149.4
11.9
16.4
Table 1: Different REDBook datasets used for HG generation training and budgets constraint inference time, respectively.
To disentangle the style information, two head- lines H" and H" with different labels are selected two candidates for the classifier. Then the head- lines are embedded with the same matrix to obtain the representation h² and ha of HP and H", re- spectively. A two-way softmax layer (equivalent to logistic regression) is applied to the style vector f::
4.2 Baselines
We select the related Seq2seq summarization methods.BAR7LLwwss et al., 2019) and PEGA- SUS(Zhang et al., 2020) as basic large pre-trained standard in the literature. The tokenizers of BART and PEGASUS are also well-established with Chi- nese datasets.
Us (H*) softmax (Wee [s; h"] bas)
(9)
where mmal(s) [Was; bes] are parameters for multi-task learning of style, and ys the output of softmax layer. The classifier is trained with a cross- entropy loss against the ground truth distribution Cs (.), shown as
Implementation Details In the following ex- periments, we use either BART or PEGASUS as a backbone. We label our proposed methods Medi- aHG with several variants: (1) MediaHG-BA is fine-tuned with eye-catchy features based BART. MeddP)GG((PGG) is fine-tuned with eye-catchy features based on PEGASUS, We also change the eye-catchy features influence as (3) MediaHG-c using content constraint only and MediaHG-s us- ing style constraint only. The choice of sample times k also has a great influence on the results. So we set (4) MediaHG-m with k = 10.
Lmul (0p; O mul)
Σ
(1 l, (H)
HE labels
(10)
The optimization can be viewed as multi-task learning loss at the same time. It not only auto- decodes the sentence but also predicts the possible style(Luong et al., 20hh) et al., 2018) ()BalBass et al., 2017).
4.3 Experiment Settings
4 Experiments
Consistent with the platform requirements, we the maximum target title length as 20 characters for all models. According to the average length of the documents and titles, we set the max length of the tokenizer as 512. The encoder and the de- coder of all Seq2Seq models are set as the same parameters as BART(Lewis et al., 2019) and PE- GASUS(Zhang et al., 2020). As have discussed in Sec 3.1, the number of sentences in document views S, is controlled by the sampling factor pa- con- rameter 8 = 2 Another hyperparameter
4.1
Dataset
We collect the dataset from social media plat- form REDBook with plenty of life-sharing "hot post" records. As long as the content/headline is compelling enough, the blog will get more expo- sure and attract more followers. "Likes" is mea- sure of a post's popularity, which is also proof of eye-catchy quality. So we filter 70k different posts with more than 2k "likes", shown in Table for the headline generator training period. To extract the global contextual features, we select the con- tent and corresponding headline of hot post, from
3
trol the number of samples |.Sa] is set as 5 and 10. During the inference period with content and style

---

budgets select the best headline, the eye-catchy feature budgets are respectively set with 0.5 and ? = 0.1.
Model
R-1
R-2
R-L
BLEU
BART
18.26
6.98
16.26
7.49
PEGASUS
20.05
7.17
17.83
10.33
MediaHG-BA
18.50
5.75
16.61
9.84
4.4
Evaluation Metrics
MediallG-c
20.85
7.08
18.68
11.64
MediaHG-s
20.43
7.48
17.92
11.16
ROUGE: We evaluate models using standard full-length ROUGE F1 (Lin, 2004) following pre- vious works(Li et al., 2021)(Gaa))))) et al., 2Xu et al., 2019). ROUGE-I, ROUGE-2 and ROUGE-L refer to the matches of unigrams, bigrams, and the longest common subsequence, respectively.
MediaHG(-PG))
21.46
7.79
19.05
11.26
MediaHG-m 23.66
8.70
20.78
12.30
Table 2: Results of our models MediaHG and baselines on each of the automatic evaluation metrics. MediaHGG- BA uses MediaHG inference methods based on BART as MediaHG(-PG) based on PEGASUS, MediaHG-c and MediaHG-s constrained with single content and style hudgets. MediaHG-m uses k as 10 while other parameters are the same as MediaHG(-PGi 1-2//LL are the ROUGE-1/2/L Fj scores.
BLEU: To evaluate our model more compre- hensively, we also use the metric BLEU proposed by (Papineni et al., 2002) which measures word overlap between the generated text and the ground- truth.
Human Evaluation: As single autometric eval- uation can be misleading (Schluter, 2017), we add human evaluation metrics to our work. We ran- domly sample 500 cases from the test set and ask three familiar and loyal REDBook users as anno- tators to score the headlines generated by BART, PEAGUSUS, and MedialIG. Referring to the gen- der and age distribution of REDBook users, review- ers consist of man and two women about 30 years old.
as the number of sampling increases, experiment time increases correspondingly. The human evaluation based on 3 aspects: sen- tence fluency, content faithfulness, and contextual eye-catchy requirements. The rating score of each model ranges from to 3, with being the best. Additionally, headlines with domain features like female tongue will get better scores in attractive- ness. Table lists the average scores of each model, demonstrating that MedialIG outperforms other baseline models.
4.5 Results
Overall Performance We compare our model with basclines in Table 2. Firstly, PEGASUS still outperforms BART, which means our task needs bit more abstractive summarization. Secondly, our model achieves 21, 21,46, 7.79, 19.05, and 11.26 in terms of ROUGE-I, ROUGE-2, and ROUGE-L respectively outperforming both PEAGASUS and BART and thus proves the superiority of our model. Besides, MediaHG outperforms MediaHG-BA in terms of all the metrics scores. An example of headlines generated by BART, PEAGASUS, and our model MediaHG can be found in Appendix We refer readers to Appendix B for more details.
Flu
Con
Attr
BART
2.05
2.13
1.76
PEGASUS
2.24
2.02
2.03
MediaHG
2.54
2.37
2.43
Table 3: Fluency(Flu), consistency(Con) and attractive- ness(Attr) Attr) comparison by human evaluation. The data the average score the three labelers' results.
4.6 Analysis
We further analyze the contribution of differ- ent module parts from diverse perspectives to gain more insights into method.
We also add MediallG-c and MedialHG-s to see the eye-catchy features set influence. MediaHG-c achieves better automatic scores than MediaHG-s, which means content budget impact on the main topic extraction of the headline. The disparity be- tween MediaHG and Media-c illustrates the style- content duality of a good eye-catchy headline.
Coefficients of Contrastive Loss The global
training loss contains two parts: the cross-entropy
R-1
R-2
R-L
BLEU
MedialIG
21.46
7.79
19.05
11.26
MediaHG-lo
22.05
7.96
20.14
11.87
The MediallG-m model shares the same param- eters with MediaHG except the number of sample times k. The outperformance of MediaHG-m also certificates the importance of sampling. However,
Table 4: Comparison results from MediaHG and fine- tuned twice generation model MediaHG-lo. R-1/2/1. are the ROUGE- 1/2/L Fl scores

---

k
R-1
R-2
R-L
BLEU
20.58
7.46
18.41
10.83
118
21.46
7.79
19.05
11.26
21.89
7.96
19.38
11.47
2020
22.56
8.25
19.85
11.75
23.07
8.46
20.37
12.00
23.23
8.44
20.40
12.10
10
23.66
8.70
20.78
12.30
Figure 4: Model performance with different y coeffi- cients weighting contrastive loss(Eq.7).
Table 6: Results with different sample number of times k.R- are ROUGR-I/2/. Fi scores.
$1
R-1
R-2
R-L
BLEU
1/2
21.25
7.65
18.92
11.35
8f = The results indicate the necessity of sam- pling extracting the main topic.
2/3
21.45
7.78
19.05
11.26
3/4
21.43
7.93
19.10
11.28
Another hyperparameter to control the number of samples [Sp] is set from 4 to 10 due to the actual needs. We see rapid increase from 4 to 5 and then slower increase. As the k rises, more selections given the selector, the scores will increase accordingly. With the dataset length features (Table I), we set the max k as 10 to create various but nonredundant results. At the same time, for the level of higher complexity, the experimental time increases much.
1
20.05
7.17
17.83
10.33
Table 5: Results with different choice of sample size Sy are ROUGR-172 1/2/L Fj scores.
loss and the contrastive loss. In order to study the influence of the contrastive learning module, train our model with different contrastive learning coefficients y. As the cross-entropy loss neces- sary to predict sequential tokens and preserve the generation model ability, we only change the val- ues of (shown in Figure 4). When is smaller than 0.1, the larger 7, the better of performance. But when is bigger than 0.1, the smaller r leads better performance. When is small, the con- trastive learning module has little positive impact on the whole model, the results are getting better. While the y increases, contrastive impact too much on the whole model, which disturbs the training of headline generation.
Conclusion and Future
In this paper, we propose an eye-catchy headline generation model MediaHG for vertical interest social media platforms. Our research is the first one focusing on vertical interest websites. As people's s interests flourish with the information gap broken down, more websites will be designed to appeal to the same interest groups. Our design allows the features extractor approach to be used more flexibly with other websites' data. Both automatic and human evaluation show our improvement headline generation.
Loop As our infer- ence selection results are style-content dual, new set of candidates can be generated in the same way the pre-trained model dynamically and contin- uously. Table illustrates the effectiveness this loop operation. also demonstrates our method's potential improvement in headline generation.
Snoccaatio-FFnttuun
Limitations
When dealing with texts of different lengths, lecting parts and generating headlines may result in redundant similar candidates insufficient infor- mation. necessary to select appropriate model parameters according to the characteristics of posts.
4.7 Ablation Study
In order to verify the effect of each module in MediaHG, we conduct ablation tests in Table 5 and Table 6. As have discussed, the number of sentences in document views S, is controlled by the sampling factor parameter sy [0, 1], so choose 2 2'3-4 to perform experiments respectively shown in Table 6. Different sampling size brings different best scores, but all outperform scores with
A Acknowledgements
This work was supported by NSFC (62322606. 62176233) and the National Key Research and De- velopment Project of China (201AAAA0001900),

---

References
comments. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6399- 6406.
Georgios Balikas, Simon Moura, and Massih-Reza Amini. 2017. Multitask learning for fine mme-grained twit- ter sentiment analysis. Proceedings of the 40th in- lernational ACM SIGIR conference on research and development in information retrieval, pages 1005- 1008.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017. beta-vae: Learning basic visual concepts with a constrained variational framework. In International conference on learning representations.
Zhangming Chan, Xiuying Chen, Yongliang Wang, Jun- tao Li, Zhiqiang Zhang. Kun Gai, Dongyan Zhao, and Rui Yan. 2019. Stick to the facts: Learning towards a üidelity-orienter e-commerce product description generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (FMNLP-IJCNILP, pages 4959-4968.
Mark Hopkins and Jonathan May. 2011. Tuning as ranking. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1352-1362. Edinburgh, Scotland, UK. ACL.
Zhiting Hu. Zichao Yang. Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. 2017. Toward con- trolled generation of text. In International conference on machine learning, pages 1587-1596. PMLR.
Zhangming Chan, Yuchi Zhang, Xiuying Chen, Shen Gao, Zhiqiang Zhang, Dongyan Zhao, and Rui Yan. 2020. Selection and generation: Learning towards muutti-product advertisement post generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP),
Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, and Pe- ter Szolovits. 2020. Hooks in the headline: Learning to generate headlines with controlled styles. arXiv preprint arXiv:2004.01980.
pages
Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2018. Disentangled representation learning for non-paralle, text style transfer. arXiv preprint
Mingda Chen, Qingming Tang. Sam Wiseman, and Kevin Gimpel. 2019a. A multi-task approach for disentangling syntax and semantics in sentence rep- resentations. arXiv preprint arXiv:/904.00 1904.01173.
04339.
Huda Khayrallah, Brian Thompson, Matt Post, and Philipp Kochn. 2020. Simulated multiple reference training improves low-resource machine translation. arXiv preprint arXiv:2004. 14524.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. Advances in neural information processing systems, 29.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: De- noising sequence-to-sequeed pre-training for natural language generation, translation, and comprehension. arXiv preprint 19Xiv:1910.13460
Xiuying Chen, Zhangming Chan, Shen Gao, Meng- Hsuan Yu, Dongyan Zhao, and Rui Yan. 2019b. Learning towards abstractive timeline summarization. In IJCA,, pages 1939-4945.
Zi-Yi Dou, Pengfei Liu, Hiroaki Hayashi, Zhengbao Jiang, and Graham Neubig. 2020. Gsum: A general framework for guided neural abstractive summariza- tion. arXiv preprint 20Xiv:2010.000
Juntao Li, Lidong Bing, Lisong Qiu, Dongmin Chen, Dongyan Zhao, and Rui Yan. 2019. Learning to write stories with thematic consistency and wording novelty. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1715-1722.
Sergey Edunov, Myle Ott. Michael Auli, David Grang- ier, and Mare' Aurelio Ranzato. 2017. Classical structured prediction losses for sequence to sequence learning. arXiv preprint arXiv:/711.040:
Mingzhe Li, Xiuying Chen, Min Yang, Shen Gao, Dongyan Zhao, and Rui Yan. 2021. The style- content duality of attractiveness: Learning to write eye-catching headlines via disentanglement, In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, volume 35, pages 13252 13260.
Marcio Fonseca, Yftah Ziser, and Shay B Cohen. 2022.
Factorizing content and budget decisions in abstrac- tive summarization of long documents by sampling summary views. arXiv preprini arXiv:2205. 12486.
Chin- Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Zhenxin Fu, Xiaoye Tan, Nanyun Peng. Dongyan Zhao, and Rui Yan. 2018. Style transfer text: Explo- ration and evaluation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 32.
Danyang Liu, Juntao Li. Meng-Hsure Yu, Ziming Huang, Gongshen Liu, Dongyan Zhao, and Rui Yan. 2020. A character-centric neural model for auto- mated story generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34. pages 1725-1732.
Shen Gao, Xiuying Chen, Piji Li, Zhaochun Ren, Li- dong Bing. Dongyan Zhao, and Rui Yan. 2019. Ab- stractive text summarization by incorporating reader

---

Yixin Liu, Zi-Yi Dou, and Pengfei Liu. 2021. Refsum: Refactoring neural summarization. arXiv preprint arXiv:2104.07210.
Kai Shu, Suhang Wang, Thai Le, Dongwon Lee, and Huan Liu. 2018. Deep headline generation for click- bait detection. In 2018 IEFE International Confer- ence on Data Mining (ICDM). pages 467-476. IEEE
Yixin Liu and Pengfei Liu. 2021. Simcls: A simple framework for contrastive learning of abstractive summarization. arXiv preprint arXiv:2106.0183
Ilya Sutskever, Oriol Vinyals, and Quoe Le. 2014. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27.
Yizhu Liu, Qi Jia, and Kenny Zhu. 2022. Length control in abstractive summarization by pretraining informa- tion selection. ACL, pages 6885-6895.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818-2826.
Fujun Luan, Sylvain Paris, Eli Shechtman, and Kavita Bala. 2017. Deep photo style transfer. In Proceed- ings of the IEEE conference computer vision and pattern recognition, pages 4990-4998.
Sho Takase and Naoaki Okazaki. 2019. Positional en- coding to control output sequence length. arXiv preprint arXiv: 1904.07418.
Minh-Thang Luong, Quoe Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2015. Multi-task tast sequence sequence learning. arXiv preprint arXiv:1511.0611.
Ashwin K Vijayakumar, Michael Cogswell, Ram- prasath R Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. 2016. Diverse beam search: Decoding diverse solutions from neural se- quence models. arXiv preprint arXiv:1610.0242:
Takuya Maknin, Tomoya Iwakura, Hiroya Takamura, and Manabu Okumura. 2019. Global optimization under length constraint for neural text summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume Long Pupers), pages 1039-1048.
Xiaojun Wan, Ziqiang Cao, Furu Wei, Sujian Li, and Ming Zhou 2015. Multi-document summariza- tion via discriminative summary reranking. arXiv preprint arXiv: 1507.0200..
Tomoya Mizumoto and Yuji Matsumoto, 2016. Discrim- inative reranking for grammatical error correction with statistical machine translation. In Proceedings of the 2016 Conference of the North American Chap- ler of the Association for Compulational Linguisties: Human Language Technologies, pages 1 1133-1138.
Pan Xie, 7hi Cui, Xiuyin Chen, Xiaohui Hu, Jian- wei Cui, and Bin Wang. 2020. Infusing sequen- tial information into conditional masked translation model with self-review mechanism. arXiv preprint arXiv:2010.09/94
Ramesh Nallapati, Feifci Zhai, and Bowen Zhou. 2017. Summarunner: A recurrent neural network based sequence model for extractive summarization doc- uments. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31.
Peng Xu, Chien-Sheng Wu, Andrea Madotto, and Pas- cale Fung. 2019. Clickbait? sensational headline generation with auto-tuned reinforcement learning. arXiv preprint 109.0219.
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Pe- ter Liu. 2020. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In In- terationnl Conference on Machine Learning, pages 11328-1133 1139. PMLR
Nadav Oved and Ran Levy. 2021. Pass: Perturb-and- select summarizer for product reviews. In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joins Conference on Natural Language Processing (Volume 1: Long Papers), pages 351-365.
Xingxing Zhang, Mirella Lapata, Furu Wei, and Ming Zhou. 2018 Neural latent extractive document sum- marization. arXiv preprint /008.0/88007111
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computa- tional Linguistics, pages 311-318.
Junbo Zhao, Yoon Kim, Kelly Zhang, Alexander Rush, and Yann LeCun. 2018 Adversarially regularized au- toencoders. In International conference on machine learning, pages 5902-5911. PMLR.
Natalie Schluter. 2017. The limits of automatic sum- marisation according to rouge. In Proceedings of the 15th Conference of the Furopean Chapter of the Asso- ciation for Computational Linguisties, pages 41-45, Association for Computational Linguisties.
Ming Zhong, Pengfei Liu, Yiran Chen, Danqing Wang. Xipeng Qiu, and Xuanjing Huang. 2020. Extrac- tive summarization as text matching, arXiv preprint arXiv:2004.0879.
Tianxiao Shen, Tao Lei,, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. Advances in neural information processing systems, 30.
Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018. Neural docu- ment summarization by jointly learning score and select sentences. arXiv preprint arXiv: 807 02305.

---

Algorithm 1: Candidates Selection. Input parameters are the candidate headlines H, weight of dual balance ax,cnneett measure method me and style measure Ma, size of beam search b1.b2. The output H is the best headline after selection.
Input: H, a. mey Myg bj, b2
Output: I!
initialization
H* +- 0,
S. f S00,0 < Ø
.M(li)) amo(Hli) + (1 a)m
for H, E H do S S.U (Hi, mc(H.) }}
7 end
8 B +- Sctto((11)
for H; E B do
10
S, + S, U { (H;, m. (H.)) }
end
C Sz.lop(b2)
13 H. + M(H, E C)
14 H A.max
A Appendix
A.1 Algorithm
An accurate prediction of the best headline ranking procedure is required before output. To avoid big waste in enumerating all possible candidates, offer beam search algorithum, especially suitable for plenty of candidates' tasks.
A.2 Case Study
We display an example of headlines generated by BART, PEAGASUS, and our model MediaHG in Table
7. The headline generated by MediaHG is more informative and attractive compared with the BART model and PEGASUS model. We also show the document of the hot post to certify the content consistency of the generated headlines.
The baselines can generate fluent headlines in this case, but they miss the attractive style and will include unattractive content. The headline generated by BART plain statement, neglecting the main attractive topic "green eyeliner painting". only catches the word "hottie", which does not fully encapsulate the character of the article. We also consider the faithfulness of different models. The headline generated by PEGASUS is significantly better than BART, as it catches the main word "green eyeliner painting" "(in blue). But headlines generated with inappropriate elements still exist as we highlight in red. For our model in the case, MediaHG captures the keywords "green eyeliner" and "hot girl", which not only cover the topic but also draw more attention. This case also demonstrates sampling module effects in main topic extraction.

---

English Document
Dear Family Fans, I'm here with a new recreation to unlock the green eyeliner! Please, try it, you can't 't miss it! The whole concept of the look "Forcing to add a little bit of green on this freezing cold day". It's a sign of my determination to be a hot spice girl! Give it a shot! Spice Girl Step Hybrid con: MITATA contact lens. A little bit of a blue-brown gradation makes for very artistic look. It's also quite premium!! Spice Girl Step 2 Hairstyle: I also unlock the double-balls hairstyle today. Very cute, with an American school-style feeling! Spice Girl Step Unique Eye Makeup: the green color of this palette called "huazhixiao" "02, so wonderful, right? Use a small brush and dip in green color to draw eyeliner, dotted with glowing sequins in the crowd! nice!
Chinese Document
tE "Mi
1 11 -
C ! AA -
-
#
PE. 7
#
Ht. nice! !
Reference Title
California Hotties!! Green Eyeliner really looks Pretty! Ди
# A
BART
Hottie, European and American Makeup Style, Hot! BRUR BRRR, B
# !
PEGASUS
European and American Style Makeup, Green Eyeliner Painting Methods, Hot Girl's Fighting Spirit and Determination. 1
f
MediaHG
European and American Style Makeup, Green Eyeliner Painting
Methods, Hot Girls' Good Looking! 1 1 A.
,
Table 7: Case study to verify the effectiveness of MediaHG. To reveal the influence of sampling on style and content space, highlight the comparison between the document and title. The sentence in blue denotes attractive information related to the reference headline and document, and the text in red denotes information related to the document but not proper for headline.


Reviews:
[{"review_id": "cpVImLiQqM", "reviewer": "Reviewer_zt61", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your positive review and valuable feedback, and we hope our response fully resolves your concern.\n\n**W1:** The proposed model is evaluated on sindle dataset.\n\n**A1:** Our method, MediaHG, is the first work focused on vertical niche platforms to improve the click ratio and attract users’ attention. Hence, it is challenging to find similar publicly available datasets with the same background characteristics. We attempt to find related stylist headline generation work, such as “The Style-Content Duality of Attractiveness: Learning to Write Eye-Catching Headlines via Disentanglement”[5], but the link of Kuaibao[6] dataset mentioned in the paper is currently inaccessible. Another dataset, Chinese LSCC news corpus, is also unavailable in recent work,\"SCS-VAE: Generate Style Headlines Via Novel Disentanglement\"[8].\n\nDue to the lack of public available datasets with similar background, we finally find another headline generation work called “Hooks in the Headline: Learning to Generate Headlines with Controlled Styles\" (TitleStylist)[1], which uses the classic summarization dataset CNN[2]. To ensure the robustness and applicability of MediaHG, we compare the results of various models on the CNN dataset and list them as below:\n\n|         | R-1   | R-2   | R-L   |\n| ------- | ----- | ----- | ----- |\n| BART    | 44.16 | 21.28 | 40.90 |\n| PEGASUS | 44.17 | 21.47 | 41.11 |\n|MediaHG|**46.74**|**22.78**|**42.84**|\n\nSince the REDBook dataset is in Chinese, our MediaHG is focused on Chineses corpus summarization using PEGASUS[4] (and BART[3]). As a result, MediaHG performs better than the baseline models BART and PEGASUS on the supplementary selection part. The headline generator part of MediaHG can be replaced with other Seq2Seq models based on multiple datasets with different languages or backgrounds. \n\nWe sincerely hope this explanation addresses your concerns. If you come across any similar datasets, please feel free to recommend them to us!\n\n**W2:** The paper lacks thorough comparison with other SOTA models.\n\n**A2:** We attempt to apply other stylistic headline generation models to the REDBook dataset but find them to be inapplicable. Our model consists of 3 parts:  *Headline Generator* ,*Sample Module* and *Features Reranking*, all trained with REDBook dataset. The input to the reranking part is the output of the headline generator part. For flexibility, our headline generator is based on Seq2Seq summarization models BART and Pegasus to generate headlines candidates. The related previous work TitleStylist[1] requires additional clickbait corpus, SpamClickBait News, to train the clickbait features. It also employs a Seq2Seq architecture[7] to generate three styles (Humor, Romance, and Clickbait), but we only refer to the Clickbait generation. Since it is difficult to find additional Clickbait corpora in Chinese, we use the CNN[2] dataset in TitleStylist for result comparison. The results are as follows:\n\n|         | R-1   | R-2   | R-L   |\n| ------- | ----- | ----- | ----- |\n|TitleStylist|26.6|9.8|23.7|\n|MediaHG|**46.74**|**22.78**|**42.84**|\n\nWe find a significant gap in results between TitleStylist and MediaHG, which may be attributed to the underlying differences in Seq2Seq models. Consistency in training data across different parts may also contribute to the results as it enhances content relevance.\n\nWe also compared our work with other headline generation models on the CNN[2] dataset in the last part 1.\n\n**W3:** The paper shall provide detailed analysis for future work.\n\n**A3:** In the paper, we have mentioned two limitations: parameter selection and clickbait criteria. The parameters may be constrained by our single dataset with specific sentence lengths, headline lengths, characteristic counts, etc. The eye-catchy feature is filtered based on likes, which may be incomplete. Therefore, we suggest considering additional features or human subjective factors for evaluation. As manual selection requires a significant amount of time, we hope future creative work can provide a precise yet convenient evaluation source.\n\n**W4:** The potential misuse of clickbait is not discussed.\n\n**A4:** To avoid solely focusing on clickbait, we prioritize **Rouge** scores to indicate the relevance between generated headlines and corresponding posts in our paper. During the candidate selection process, we also assign separate scores for content and contextual style. The Rouge score can serve as a detection strategy to prevent the misuse of clickbait.\n\n**[Reference]**\n\n[1] Hooks in the headline: Learning to generate headlines with controlled styles.\n\n[2] Teaching machines to read and comprehend. \n\n[3] BART: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. \n\n[4] Pegasus: Pre-training with extracted gap-sentences for abstractive summarization.  \n\n[5] The style-content duality of attractiveness: Learning to write eye-catching headlines via disentanglement. \n\n[6] Automatic article commenting: the task and dataset.\n\n[7] Mass: Masked sequence to sequence pre-training for language generation.\n\n[8] SCS-VAE: Generate Style Headlines Via Novel Disentanglement. \n\nAgain, we appreciate the reviewer's valuable reviews. We hope our response can address your concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper presents MediaHG, a novel disentanglement-based headline generation model that adeptly balances content and contextual features to craft headlines that are fitting for their respective domains. The strengths of this paper are manifold:\n\n1. The introduction of a unique disentanglement-based model for headline generation, which effectively separates content and contextual features, enabling the generation of domain-appropriate headlines. This novel approach could be applied to other domains beyond social media, expanding its potential impact.\n2. The development of a new dataset, REDBook, specifically designed for the evaluation of headline generation models. This dataset enriches the resources available to the NLP community for research in this area.\n3. The empirical demonstration of MediaHG's effectiveness using the REDBook dataset, with comparative analysis against other state-of-the-art models. This rigorous evaluation not only underscores the superior performance of MediaHG but also sets a benchmark for future research in this area.\n4. The potential application of the proposed model to improve the quality of headlines generated for social media posts. This could benefit both content creators and readers by enhancing the informativeness and engagement of social media content.\n", "reasons_to_reject": "The paper presents MediaHG, a novel disentanglement-based headline generation model that balances content and contextual features to create domain-suitable headlines. While the paper makes significant contributions, it also has potential weaknesses:\n1. The proposed model is evaluated on a single dataset, REDBook. This may limit the generalizability of the results to other social media platforms or domains. Future work could involve testing the model on a variety of datasets to ensure its robustness and applicability across different contexts.\n2. The paper does not provide a thorough comparison with other state-of-the-art models for headline generation. This may limit the ability to assess the effectiveness of the proposed model relative to other approaches. Future work could involve a more comprehensive comparison with other models, considering various metrics and scenarios.\n3. The paper does not provide a detailed analysis of the limitations of the proposed model or potential areas for future research. Future work could involve a more in-depth discussion on these aspects, providing a more balanced view of the model and paving the way for further improvements.\n4. The potential misuse of the proposed model for creating clickbait or for other unethical purposes is not discussed in the paper. Future work should consider these ethical implications and possibly propose mechanisms to prevent such misuse.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper presents MediaHG, a novel disentanglement-based headline generation model that adeptly balances content and contextual features to craft headlines that are fitting for their respective domains. The paper addresses the challenge of creating compelling and informative headlines for social media posts. The primary contributions of the paper are threefold:\n\n1. The introduction of a disentanglement-based model, MediaHG, which effectively separates content and contextual features, enabling the generation of domain-appropriate headlines.\n2. The development of a new dataset, REDBook, specifically designed for the evaluation of headline generation models.\n3. The empirical demonstration of MediaHG's effectiveness using the REDBook dataset, with comparative analysis against other state-of-the-art models, underscoring its superior performance.\n", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "w7xw7K2r9J", "reviewer": "Reviewer_miJc", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's constructive feedback and their recognition of our work's potential to foster community development and have a positive impact on others. We understand that the reviewer's concerns primarily revolve around the ethical aspects of our paper, and we aim to address these concerns comprehensively in our response.\n\n**W1:** The paper lacks sufficient information regarding the data collection process.\n\n**A1:** To obtain the data for our study, we utilized web scraping techniques on the publicly available information from the REDBook website. The \"public information\" refers to content that is visible to all platform users. The data we scraped is referred to as the raw dataset, which consists of 1,695,219 posts and 42,447 users. Each post included various features such as the headline, content, likes count, and writer. In order to construct our experimental dataset for eye-catchy detection, we filtered out 70k distinct posts with more than 2k likes. To ensure a diverse selection and avoid personal-style bias, we selected posts in a balanced manner from different bloggers.\n\n**W2:** The absence of a case study to showcase the quality of different models.\n\n**A2:** We appreciate the reviewer's feedback regarding the case study in our work. Due to the constraints of the paper's length, we have included the case study in Appendix B and mentioned it in Section 4.5. The case study presents the results in two languages (English and Chinese) from three models for comparison, with MediaHG achieving the best performance according to the reference. We hope that the inclusion of the case study comparison adequately addresses the reviewer's concerns, and we welcome any further advice they may have.\n\n**W3:** The human evaluation result of MediaHG_BA should be included in Table 3.\n\n**A3:** In our evaluation, we employed human evaluation from three aspects: sentence fluency, content faithfulness, and contextual eye-catchy requirements, specifically assessing the performance of the generated headlines. To highlight the advantages of our design, we compared the performance of MediaHG(-PG) with two baseline models (BART and PEGASUS). Additionally, we introduce an additional candidate selection framework for BART, referred to as MediaHG-BA, which also demonstrates the benefits of our approach. We invite the same evaluators as before to assess the results from MediaHG-BA, and the average scores obtained were as follows: sentence fluency: 2.01, content faithfulness: 2.13, and contextual eye-catching requirements: 1.87.\n\nWe include the MediaHG-BA results in Table 3, which are presented as follows:\n\n|         | Flu  | Con  | Attr |\n|:-----:|:------:|:------:|:------:|\n| BART    | 2.05 | 2.13 | 1.76 |\n| PEGASUS | 2.24 | 2.02 | 2.03 |\n|MediaHG-BA|2.17|2.13|1.87|\n|MediaHG(-PG)|**2.54**|**2.37**|**2.43**|\n\nAgain, we appreciate your valuable reviews and hope our response can fully resolve your concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper proposes a novel MediaHG model to generate eye-catchy headlines to enhance user interaction, where a sample module is designed to obtain different documents views, and contrastive learning and multi-task learning are utilized to choose the best sentence among the headline candidates generated from the headline generator.\n2. Authors collect a headline generation dataset REDBook, which has the potential to foster community development.\n3. The designed modules are general and could be employed on other models. Experimental results indicate the effectiveness of the proposed modules.\n", "reasons_to_reject": "1. The paper lacks sufficient information regarding the data collection process, which raises concerns about potential ethical issues. To ensure transparency and address any ethical considerations, it is essential for the authors to provide detailed insights into their data collection methodology and how they handled any ethical implications that may have arisen during the process.\n2. The absence of a case study to showcase the quality of headlines generated by different models is a limitation of the paper.\n3. The human evaluation result of MediaHG-BA should be included in Table 3.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper focus on generating headlines for social media users’ posts to promote the users’ interaction. To obtain attractive headlines, the authors propose a MediaHG model, which first randomly combines sentences to generate headline candidates and then uses a rerank module to select the best title based on the style and content. Additionally, a dataset that contains popular posts is collected. Experimental results with automatic metrics and human evaluation on the collected dataset show the effectiveness of the proposed model.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "b87VNlovgG", "reviewer": "Reviewer_r8GS", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your positive review and valuable feedback, and we are committed to addressing your concerns fully in our response.\n\n**W1:** The paper dose not clarify whether the dataset will be available for pulic use.\n\n**A1:** Our work is the first to generate topic-catchy and contextually harmonized headlines for vertical niche platforms, aiming to enhance users' experience. In consideration of privacy protection, we will make our dataset available after undergoing a thorough data cleaning process. This will benefit broader research areas such as niche platform preference and stylistic headline generation. We also welcome any attempts to reproduce or validate our findings.\n\n**W2:** The paper would benefit from a deeper comparison with \"The Style-Content Duality of Attractiveness: Learning to Write Eye-Catching Headlines via Disentanglement\".\n\n**A2:** In the mentioned work[5], disentanglement is used to separate the style and content into latent spaces, but the extractor requires an additional language training process. To make our model more flexible and reduce computational complexity, we generate a set of candidates and select the most suitable one simultaneously. In fact, our model introduces a novel approach in constructing the style and content space constraints, with reranking candidates generated by the Seq2Seq model.\n\n**W3:** The paper lacks test on multiple datasets.\n\n**A3:** Our method, MediaHG, is the first work focused on vertical niche platforms to improve the click ratio and attract users’ attention. Hence, it is challenging to find similar publicly available datasets with the same background characteristics. We attempt to find related stylist headline generation work, such as “The Style-Content Duality of Attractiveness: Learning to Write Eye-Catching Headlines via Disentanglement”[5], but the link of Kuaibao[6] dataset mentioned in the paper is currently inaccessible. Another dataset, Chinese LSCC news corpus, is also unavailable in recent work,\"SCS-VAE: Generate Style Headlines Via Novel Disentanglement\"[8].\n\nDue to the lack of public available datasets with similar background, we finally find another headline generation work called “Hooks in the Headline: Learning to Generate Headlines with Controlled Styles\" (TitleStylist)[1], which uses the classic summarization dataset CNN[2]. To ensure the robustness and applicability of MediaHG, we compare the results of various models on the CNN dataset and list them as below:\n\n|         | R-1   | R-2   | R-L   |\n| ------- | ----- | ----- | ----- |\n| BART    | 44.16 | 21.28 | 40.90 |\n| PEGASUS | 44.17 | 21.47 | 41.11 |\n|MediaHG|**46.74**|**22.78**|**42.84**|\n\nSince the REDBook dataset is in Chinese, our MediaHG is focused on Chineses corpus summarization using PEGASUS[4] (and BART[3]). As a result, MediaHG performs better than the baseline models BART and PEGASUS on the supplementary selection part. The headline generator part of MediaHG can be replaced with other Seq2Seq models based on multiple datasets with different languages or backgrounds. \n\nWe sincerely hope this explanation addresses your concerns. If you come across any similar datasets, please feel free to recommend them to us!\n\n**[Reference]**\n\n[1] Hooks in the headline: Learning to generate headlines with controlled styles.\n\n[2] Teaching machines to read and comprehend. \n\n[3] BART: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. \n\n[4] Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. \n\n[5] The style-content duality of attractiveness: Learning to write eye-catching headlines via disentanglement. \n\n[6] Automatic article commenting: the task and dataset.\n\n[7] Mass: Masked sequence to sequence pre-training for language generation.\n\n[8] SCS-VAE: Generate Style Headlines Via Novel Disentanglement. \n\nAgain, we appreciate your valuable reviews and hope our response can fully resolve your concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper addresses a significant problem in the realm of social media, focusing on the balance between content relevance and the appeal of headlines. The paper takes into account the uniqueness of platform features, catering to specific needs of different websites.\n\n2.  The clear delineation between the generation and selection process is commendable. By separating these two steps, the model allows for the possibility to reuse the ranking process in different scenarios. This modular approach could be instrumental in adapting to various use-cases or platforms.\n\n3. The paper conducts human evaluation and provides a detailed case study. By doing so, the authors provide tangible evidence of the efficacy and quality of the generated headlines.", "reasons_to_reject": "1. The paper does not clarify whether the dataset used, sourced from REDBook, will be made available for public use. A publicly accessible dataset would benefit the broader research community, ensuring reproducibility and facilitating comparison with other models. Without this, it becomes challenging for peers to reproduce or validate the study's findings.\n\n2. The idea of bifurcating content and style resembles the approach described in \"The style-content duality of attractiveness: Learning to write eye-catching headlines via disentanglement.\" The current paper would benefit from a deeper, more detailed comparison with this prior work rather than a brief mention or superficial comparison.\n\n3. While the paper claims theoretical flexibility for the MediaHG model, there's a noticeable absence of empirical backing. To substantiate this claim, it would be beneficial to test the model on multiple datasets, ensuring a comprehensive evaluation and lending more credibility to the notion of its adaptability.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper introduces \"MediaHG,\" a disentanglement-based model designed for social media headline generation. This model effectively balances between content and contextual attributes. It is developed using data from 70,000 popular posts on REDBook, a renowned Chinese social media platform. Evaluations using the ROUGE metric and human assessment reveal significant advancements in headline generation for the platform.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Justification_for_Ethical_Concerns": "n/a"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a disentanglement-based headline generation model for social media that balances content (i.e. capturing the main topic of the document) and contextual features (i.e. producing "eye-catchy" headlines). Overall, reviewers are quite positive about this work and I feel that the paper's strengths (e.g. sound and effective model, introduced dataset, application beyond social media) outweigh its weaknesses (e.g. lack of details about the proposed dataset, limited scope of the experiments). It should be noted that some of these concerns were discussed and, I believe, addressed by the author's responses. 

I would also like to see the potential ethical issues related to this work thoroughly discussed in the paper (i.e. misuse of the model for click-bait (#R zt61), sourcing of the REDBook dataset (#R r8GS and miJc)).
Decision: Accept-Main