StyleBART: Decorate Pretrained Model with Style Adapters for Unsupervised Stylistic Headline Generation
Authors: Hanqing Wang
Yajing Luo
Boya Xiong
Guanhua Chen
Yun Chen
Venue: EMNLP 2023 Findings (2023)
Keywords: Headline generation; Style transfer; Efficient NLP; Unsupervised learning

Abstract: Stylistic headline generation is the task to generate a headline that not only summarizes the content of an article, but also reflects a desired style that attracts users. As style-specific article-headline pairs are scarce, previous researches focus on unsupervised approaches with a standard headline generation dataset and mono-style corpora. In this work, we follow this line and propose StyleBART, an unsupervised approach for stylistic headline generation. Our method decorates the pretrained BART model with adapters that are responsible for different styles and allows the generation of headlines with diverse styles by simply switching the adapters. Different from previous works, StyleBART separates the task of style learning and headline generation, making it possible to freely combine the base model and the style adapters during inference. We further propose an inverse paraphrasing task to enhance the style adapters. Extensive automatic and human evaluations show that StyleBART achieves new state-of-the-art performance in the unsupervised stylistic headline generation task, producing high-quality headlines with the desired style.

StyleBART:
Decorate Pretrained Model with Style Adapters for Unsupervised Stylistic Headline Generation
Hanqing Wang Yajing Luo'+ Boya Xiong' Guanhua Chen²
Yun Chen' 1
1 Shanghai University of Finance and Economics
Southern University of Science and Technology [whq, luoyajing, xiongboya)@163.coo sufe edu. cn chengh3@sustech. edu. en yunchen@sufe.cm edu. cn
Abstract
Styk
Stylistic headline generation is the task of gen- erate headline that not only summarizes the content of a news, but also reflects a desired style that attracts users. As syyle-specific news- headline pairs scarce, previous research has focused on unsupervised approaches using a standard headline generation dataset and mono- style corpora. In this work, we follow this line and propose StyleBART, an unsupervised ap- proach for stylistic headline generation. Our method decorates the pretrained BART model with adapters that are responsible for differ- ent styles and allows the generation of head- lines with diverse styles by simply switching the adapters. Different from previous works, StyleBART separates the task of style learn- ing and headline generation, making it pos- sible freely combine the base model and the style adapters during inference. We fur- ther propose an inverse paraphrasing task to enhance the style adapters. Extensive auto- matic and human evaluations show that Style- BART achieves new otate-of-the-art perfor- mance in the unsupervised stylistic headline generation task, producing high-quality head- lines with the desired style. Code is available at https//ggiiubbbooo//sutn///////p
Hamar
Romance
Chollei
"Sew
Syfe
Hiwwor
Remaaee
New Style
Skay Telks
Rive Ceamstio
Story Tellig
New
Npw Tike
Taak (M)
(a) Baselines: N x M
(b) StyleBART. N + M
Figure I: Our StyleBART is training-efficient. (1) For combination of M tasks and N styles, StyleBART only trains M task-specific models and N style-specific modules, then combines them at inference time to sup- port each stylistic generation task. Previous works train separate M x N models. (2) For a new style (task), StyleBART only trains a new style module (task model), however, baselines have to train many additional models to combine the new style (task) with all existing tasks (styles).
ing enough parallel data for SHG is almost im- possible, as the creation of headlines with specific styles demands creativity and often consumes sig- nificant effort. Hence, researchers (Jin et al., 2020; Zhan et al., 2022) in turn explore unsupervised ap- proaches which only require a standard headline generation dataset and non-parallel stylistic text corpora.
1 Introduction
The sequence-to--eqqeecc---ee neural headline generation (HG) model (Cao al., 2018; Lin et al., 2018; Song et al., 2019) has demonstrated its abil- ity to generate factual, concise, and fluent head- lines (Chopra et al., 2016). Yet, the headlines are also expected to have stylistic attributes to draw more attention of the audiences. To address the problem, Jin et al. (2020) propose the task of Stylis- tic Headline Generation (SHG), which aims to gen- erate headline with a specified style such as hu- morous, romantic and clickbaity. However, acquir-
Some existing works propose a pipeline method (Sudhakar et al., 2019; Dai et al., 2019; Krishna et al., 2020) which firstly generates a plain headline and then introduces the pre-specified style with a style transfer model. However, this pipeline method require two models during inference, which brings additional latency and storage cost. Some other works (Jin al., 2020; Zhan al., 2022) jointly learn plain headline generation model on news-headline pairs, and a denoising autoencoder on the stylistic text corpus. However, these ap- proaches require to carefully design the scheduling
"Co-first author with equal contribution "Corresponding author

---

and parameter sharing mechanisms between tasks. Moreover, as the task of headline generation and style learning are entangled, the entire model has to be rebuilt when facing a new style or task.
Ourgut
Upp Prejeive
Aulight
Doour
In this paper. we propose StyleBART, an unsupervised SHG model based on the BART model (Lewis et al., 2020). Instead of using multi- task learning framework, StyleBART disentangles the style and the specific task (c.g. headline gen- eration) by the design of model architecture and training strategy. Intuitively, this enables Style- BART to separately train the style and task mod- ules, then combine them as required during infer- ence (Figure 1). Hence, StyleBART is more flexi- ble and training-efffciint compared with baselines (Jin et al., 2020; Zhan et al., 2022) which require training models for all style and task combinations. Specifically, StyleBART utilizes style adapters as plug-and-play modules for the style control. The style adapters are learned at the pretraining stage through the inverse paraphrasing task on the style corpus, while the base HG model is learned at the fine-tuning stage on news-headline pairs. During inference, achieve unsupervised SHG by switch- ing to the target style adapter. In this way, Style- BART has the same high decoding efficiency as Jin et al. (2020), while overcoming its problems in task scheduling and inefficient training.
Food Forward
Laver Stee
Feod Forward
Aftation
Madeal Sclf-Ataaoo
Morket Sell-Atiiii
The architecture of StyleBART consists of the BART model and the adapter modules. The pretrained BART model is composed of N encoder layers and N decoder layers. The adapter module is inserted after the feed-forward sublayer in each decoder layer.
Figure 2:
tune the base model for a new task (Houlsby et al., 2019), language (Pfeiffer et al., 2020) or domain (Bapna et al., 2019). In StyleBART, we instead use the adapters to control the style of the model output.
Formally, following Houlsby et al. (2019), an adapter module A is composed of layer normaliza- tion (LN) of the input E Rh, down-projection (W down E Rhxb) with the bottleneck dimension b, non-linear function (ReLU), up-projection (Wup E Rlxch,, and a residual connection with the input 2:
1(2) W ReI.U down LN(x)) + (1)
We evaluate StyleBART using the same three SHG tasks described in Jin et al. (2020). Both automatic and human evaluations demonstrate the superiority of our method over baselines.
We insert the adapters into each decoder layer of the base model. For each style, we have set of style adapters. In the following, we use 0 to denote the set parameters of the BART model, and 0% to denote that of the style adapters for style s,.
2 Approach
2.1 Problem Formulation
2.3 Training and Inference
We denote the parallel news-headline dataset as
We divide the training process of StyleBART into three steps (Figure 3): (1) Style adapter pretraining, which learns the style adapters to control the style of the model output by pretraining on the style dataset 7 (2) Headline generation fine-tuning, which optimizes the base model to generate plain headline on the headline dataset D; (3) Stylistic headline generation, which generates stylistic head- line by switching to the target style adapters during inference.
D {xx }. where each pair (z, v) consists of
a news article x and its corresponding plain head- line y. The corpus for the 7th style s; is denoted 7% {t*}}, }. Our goal is to generate stylistic headline n with style s; given the news article Note that this an unsupervised setup, as no news article and stylistic headline pairs are available.
2.2 StyleBART Architecture
As shown in Figure 2, model consists of two parts: the BART model and the adapter modules. We use the BART as the base model for StyleBART. Adapters (Rebuffi et al., 2018) are light bot- tleneck layers inserted into a base model. It is designed as a parameter-efficcan method to fine-
Step 1: Style Adapter Pretraining. The style adapters can be trained with the style dataset T by the denoising auto-encoding task, which recon- structs the sentence t* from gn (t"). Here 9n is a noise function that generates a perturbed version

---

Trainable
Freezing
Haow
Hlmee Adglla
Humoroos Huadine
Rlmmac Adgrec
Decoder Layer
Stylatti
Decoder
Adianca
Plain Headline
Decoder Layer
Roma se Adaper
Romantic Headline
Kikilet Adger
Layer
Clickbuity Headine
Stbokon
Paraphmood
Encoder
Layer
Artele
Layer
Articee
Layer
Step
Step
Step
Figure 3: Training and inference framework of StyleBART. Step Style adapter pretraining: Step Headline generation fine-tuning; Step 3: Stylistic headline generation.
of the input, such as token masking and token dele- tion used during BART pretraining (Lewis al., 2020) and our baselines (Jin et al., 2020; Zhan et al., 2022). However, this training method is subopti- mal.
ing all other parameters with inverse paraphrasing:
0+ arg max
Σ ET
log p(t (p(t")):0"
6%
(2)
Since the input is style-agnostic and the other pa- rameters freezing, the model has to rely the style adapters to control its output style. We also learn a style-less adapter 0 in the same way except replacing T with dataset T%, which con- sists of plain text from BART pretraining.
Following Krishna et al. (2020), the style can be loosely defined as the common patterns of lexical choice and syntactic constructions that are distinct from the content of a sentence. Considering how the noise function In works, we argue that the cor- rupted text 9n. (t*c still contains the stylistic infor- mation. As result, the model may learn to guide the style of its output with the style of its input, which deviates from our goal to control the output style with style adapters. We call this undesirable spurious correlation (Gu et al., 2019). Instead, we propose to address this issue with inverse para- phrasing method.
Step 2: Headline Generation Fine-tuning. In this step, StyleBART learns headline generation on the headline dataset D { (zz y) }. Since the head- line is style-less, we insert the style-less adapters into StyleBART and finetune it on the headline generation task. This step required to force the model to learn the task of headline generation. Dur- ing finetuning, we only update the parameters the encoder while freezing the style-less adapters and all other model parameters:
Inspired by Krishna al. (2020), replace In (1%) with (t)))), which is generated by feed- ing 6a paraphrase model' trained maximize diversity. Table displays examples of stylistic sen- tences and their perturbed versions with the noise function Gn. and the paraphrase model 9p- Intu- itively, the paraphrase model can better strip away information from f that is predictive of its original style. As result, model has to rely the style adapters instead of the input sentence control its output style, enhancing the adapters' ability for style control.
go enc
Σ <ANDED
log x;; 800 8²).
arg max
(3)
In this way, we limit the computational cost, and more importantly, mitigate the catastrophic forget- ting problem in style control, thus facilitating the switching of different style adapters in Step 3.
Step 3: Stylistic Headline Generating. To gen- erate a headline a given style, we can achieve this by replacing the style-less adapters of StyleBART with the corresponding style-specific adapters.
Specifically, to train the adapters for style 8;, we first feed each sentence ( to the paraphrase model and get the perturbed sentence (t))). Then train the corresponding style adapters while freez-
We choose plain text from BART pretraining instead of the headline generation dataset in order to separate the style and downstream task at the data level.
We use the STRAP model (Krishna et al., 2020).

---

Style s;
Stylistic Sentence 1
Noised Text (n((t))
Paraphrased Text 9p(t*))
There are few things more awkward on a blind date than looking up from your phone to realize she's left. She obviously wasn't t blind at all.
She [MASK] blind at all. There are few things [MASK] on a blind date than [MASK] looking up from your phone to realize she's left.
The fact that she left, and the fact that she's obviously not blind, is much more awk- ward.
Humor
He confessed to her that she was his most precious jewel, and that she would always be the rarest gem in his eyes.
[MASK] to her that she was most [MASK] jewel, and that she would [MASK] rarest gem in his eyes.
He confessed that she was the most valuable thing his eyes, and he would never see another one.
Romance
Unbelievable meeting Bob Dylan in Europe for the third time twenty years.
Unbelievable meeting Bob Dylan in [MASK] for the third time twenty years.
The extraordinary meeting with Bob Dylan in Europe for three times.
Clickhait
Table 1: Examples of sentences and their perturbed versions in three styles: humor, romance and clickbait. The noised text indicates the output after the noise function Ins and the paraphrased text indicates the output after passing through the paraphrase model Gp-
StyleBART has already learned the task of headline generation. Therefore, by switching the style-less adapters to the style-specific adapters, we can ob- tain style-specific headline generation model. We use gh 80 and 6% denote the model parameters of the base model, the style-less adapters and the style adapters after Step 2. Then given news arti- cle, StyleBART outputs its style-specific headline y with
one of the datasets used in BART pretraining. For style datasets, we use the same split as Jin et al. (2020). For the style-less dataset, we randomly sample 3,000 sentences for both the validation and test set, leaving the rest as the training set.
3.2 Baselines
We compare StyleBART with the following base- lines:
Neural Headline Generation (NHG): Finetun-
nd arg max max x;; 0,, 00).
(4)
ing all parameters of BART on the plain headline generation dataset.
3 Experiments
Two-Step Decoding( A two-step decoding method that first generates plain headline and then injects style with an unsupervised style transfer model (Dai al., 2019).
3.1 Datasets
Following Jin et al. (2020), the experiment datasets consist of a headline generation dataset CNN-NYT, and three stylistic text datasets for humorous, ro- mantic and clickbaity. The CNN-NYT dataset con- sists of 146K news-headline pairs from two sources: the New York Times (Sandhaus, 2008) and the CNN dataset (Hermann et al., 2015). We use the same dataset split Jin et al. (2020). Each style dataset contains 500K style-specific sentences. The humor and romance datasets are collected from the novels in the corresponding genres of BookCor- pus (Zhu et al., 2015). The clickbait dataset is obtained from The Examiner- Spannner-SSamCCCicBBaai News 3 3 To train the style-less adapters, we also build style-less dataset, which consists of 500K sentences randomly sampled from BookCorpus,
Multitask: A multitask framework that jointly trains on plain headline generation and stylistic text denoising with the BART model.
TitleStylist (Jin et al., 2020): An approach simi- lar to Multitask, but with carefully designed param- eter sharing and switching between tasks.
S-SHG (Zhan et al., 2022)*: An approach that first constructs the stem and syntax of the headline similarly to TitleStylist, and then populates that with substantive context.
3.3 Evaluation Metries
Automatic Evaluation. We use automatic evalu- ation to measure the quality and style of the gener-
"Our reimplemented S-SHG performs slightly worse than the original paper. However, the paper provides no open- source code or implementation details.
hhtp:///ww..aggg...ooddascteeeee examiner

---

Generation Quality
Style Strength PPL-S(()
Style
Method
RI(7)
R2(1)
RU())
BERT(T)
PPL(()
NHG
29.7
10.9
26.3
88.2
52.6
Style-less
StyleBART-N
29.4
10.7
26.2
88.1
55.6
TSD
20.3
5.5
18.5
MultiTask
30.0
11.1
26.6
87.9
52.8
1267.6
Humor
TitleStylist
27.7
10.0
24.5
87.5
39.4
640.9
S-SHG
26.2
8.2
22.5
87.4
107.4
702.2
StyleBART
28.2
9.9
25.0
87.6
42.5
602.9
TSD
20.5
5.8
18.7
MultiTask
30.0
11.4
26.8
88.1
1543.0
Romance
TitleStylist
27.6
9.9
24.5
87.5
39.5
740.1
S-SHG
26.2
8.6
22.5
87.4
101.0
685.0
StyleBART
27.4
24.3
87.4
36.7
560.5
TSD
20.3
7.1
23.2
MultiTask
30.3
11.3
27.0
88.3
52.2
392.4
Clickbait
TitlSttylist
28.2
10.1
25.2
87.8
39.1
256.4
S-SHG
27.0
8.8
23.5
87.6
67.9
234.3
StylBARR
25.3
7.7
23.0
87.5
50.9
115.5
Table Automatic evaluation results of ROUGE, BERTScore, PPL and PPL-S. TSD" result is directly cited from Li et al. (2022), while all other methods are implemented by ourselves with the open-sourced code. StyleBART-N: StyleBART with the style-less adapters.
ated headline. For the quality, 2004) and BERTScore (Zhang* et al., 2020) to
ROUGE (Lin,
each model is chosen as the best.
3,4 Model Configuration
measure the relevance of the generated headlines to the reference headline and PPL to evaluate its lan- guage fluency. The PPL calculated with OpenAl GPT2 (Radford et al., 2019) finetuned on the plain headlines following Zhan et al. (2022). To evalu- ate the style, we use PPL-S, which computes the PPL score of the generated headline using GPT2 finetuned on the corresponding style corpus.
We implemented StyleBART with the pretrained BART model using Hugging Face. We the bottleneck dimension b of the adapters be 64. To pretrain the adapters, we use the AdamW opti- mizer (Kingma and Ba, 2014) with 31 = 0.9 and $2 = 0.999. The learning rate is 5e 5. For fine- tuning on the headline generation dataset, we use the same AdamW optimizer and learning rate. For all steps of training, we use batch size of 8. At in- ference time, use beam search with beam size of 4. All experiments are run single NVIDIA RTX 2080ti GPU, except that LLaMA2- MaAMMnnnnnsTunng run on a single NVIDIA A100 40G GPU
Human Evaluation. We conducted human evaluation to more comprehensively assess our model. We randomly sampled 50 news articles from each test set and asked three judges to rate the outputs from TitleStylist, S-SHG and our Style- BART: To assess generation quality, we ask the judges to rate from 1 to 10 (integer values) from three aspects: 1) Relevance- how semantically rel- evant the headline is to the news article. 2) At- tractiveness- how appealing they feel the headline 3) Fluency- how comprehensive and easy-to- read the headline is. We then report the average score for each aspect across all test samples and judges. For the style evaluation, we ask the judges to choose the best headline of each style from an entire set of TitleStylist, S-SHG and StyleBART's outputs. We then report the percentage of the times
Results and Discussion
4.1 Automatic Evaluation Results
Table shows the automatic evaluation results of our proposed StyleBART and all baselines.
Content Relevance. We use ROUGE (R1,R2, and RL) and BERTScore (BERT) to measure the relevance of the generated headlines the refer- headline. As be seen, NHG, StyleBART- N. and MultiTask achieve the highest relevance score. This can be explained by that more stylistic headlines would lose some relevance as 1) the ref- erence headlines are style-less; 2) the stylistic head-
*We do not include TSD, Multitask as they are much worse according to previous work and our automatic evaluation.

---

Style
Method
Relevance
Attraction
Fliency
Style
Titlettylist
S-SHG
StyleBART
Tiylsttylist
5.82
5,60
7.40
Humor 20%
34%
46 %
Humor
S-SHG
6.13
7.12
8.23
StyleBART
7.04
7.58
8.42
Romance 22%
28%
50 %
Stylst
6.12
6.06
7.78
Clickbait
32%
26%
42% %
Romance
S-SHG
6.37
6.04
7.63
StyleBART
6.70
7.30
8,40
Table 4: The percentage of choices (%) for the most
Stylet
6.16
humorous, romantic clickbaity headlines among Ti- tleStylist, S-SHG, and StyleBART.
Clickbait
S-SHG
6.17
6.65
8.17
StyleBART
6.76
6,96
8.26
Table 3:
Human evaluation results three aspects.
Generation Quality. We assess generation qual- ity in relevance, attraction, and fluency, as shown in Table 3. StyleBART performs the best in all three aspects compared to baselines. The results are slightly inconsistent with automatic quality eval- uation. This may be explained by that automatic quality evaluation metrics favor less stylistic head- lines, while humans do not have such bias.
line may use more words outside the news body for improved creativity (Jin et al., 2020). For stylistic headline generation, TSD has the worst relevance score. StyleBART performs slightly worse than TitleStylist (-0.6 averaged RL and -0.1 averaged BERT) and better than S-SHG (+1.3 averaged RI. and +0.0 averaged BERT), validating StyleBART's ability in this aspect.
Style Strength. We measure the style strength in Table 4. As can be seen, StyleBART has the highest average selection percentage, followed by the S-SHG model. This consistent with what we find in the automatic measurement.
Language Perplexity. We use PPL on the GPT2 fine-tuned on plain headlines to measure the flu- ency of the generated headline. As can be seen, StyleBART surpasses all baselines by signifi- cant margin except that is slightly worse than Ti- deStylist. This may be because the headlines from StyleBART have stronger style and more stylistic headlines would also lose some PPL.
4.3 Comparison with LLMs
In this part, we compare StyleBART with meth- ods using Large Language Models (LLMs), as pre- sented in Table 5. For GPT3.5 -ero.s-prompting we per- form few-shot prompting with the gpt-3.5-turbo" API. We provide stylistic sentences and news-plain headline pairs in the prompt and query the model to generate stylistic headlines for the input news. For LLaMA2- InsTning ing with LoRA on LLaMA2 (Touvron al., 2023) for both the inverse paraphrasing task and the news headline generation task. Then we perform stylis- tic headline generation during testing. Appendix A provides more details. We find that StyleBART overall generates headlines with the best content relevance and the strongest style, demonstrating the superiority of StyleBART even in the era of large language models.
Style Strength. We use PPL-S to measure the style strength of the generated headline. As can be seen, StyleBART generates headlines with the strongest style across all stylistic headline genera- tion tasks. Compared with the baseline TitleStylist (resp. S-SHG), StyleBART obtains 119.5 (resp. 114.2) lower averaged PPL-S score. The base- line MultiTask performs the worst style control. Moreover, StyleBART disentangles style learning and headline generation learning, thus only trains once on the headline generation dataset to support all three styles. In contrast, all baselines except TSD require training three times on the headline generation datuset for the three stylistic generation tasks.6
conduct instruction tun-
4.4 Ablation Study
Design Choices of Style Adapter Pretrain- ing. Table 6 shows the effect of different de- sign choices at the style adapter pretraining step. StyleBART para represents training the adapters
4.2 Human Evaluation Results
Table 3 and Table present the human evaluation results.
'The baselines can also support multiple styles within one model by jointly training on headline generation dataset and all style corpora. However, the performance will decrease according to Jin et al. (2020).
'htps://plafforooooee wwwpopp.co//ooss/////.poo com///ocs/moodellgnn...5 "Note that StyleBART and these two LLM--aaee methods are not strictly comparable due to the difference in model size and training data. We include the comparison as reference.

---

Generation Quality
Style Strength
Style
Method
RI(1)
R2(1)
RL(T)
BERT(T)
PPL())
PPL-S(()
Prompiggo
22.7
5.9
19.5
85.8
1972.4
1161.3
Humor
L..MAA-InsTuning
24.5
7.7
21.6
86.3
622.8
StyleBART
28.2
9,9
25.0
87.6
42.5
602.9
22.4
5.8
19.4
85.9
1561.2
1329.4
Romance
LLaMA2-Ii
2aMMAA--nnTTnnng
26.6
8.5
23.5
87.0
112.6
879 4
StyleBART
27.4
9.3
24.3
87.4
36.7
560.5
GPT3.5 GGP..55-romman prompting
24.5
6.9
21.0
86.2
1074.0
4094.6
Clickbait
LLaMA2- Innnnnnn
27.2
8.9
23.8
86.9
46.2
353.1
StyleBART
25.3
7.7
23.0
50.9
115.5
Table 3: Comparison results with methods based on large language models.
Style
Method
RL
BERT
PPL
PPL-S
Method
Style
RL
BERT
PPL
PPL-S
StyleBART-N
26.2
88.1
55.6
dec
Style-less
-prr
26.4
57.4
x
x
26.9
88.2
55.6
-8g adapters
26.5
88.2
57.6
Style-Icss
x
26.7
88.2
57.8
StyleBART
25.0
87.6
42,5
602.9
/
/
26.4
88.2
55.1
Humor
26.0
88.0
54.4
1033.0
25.0
87.6
42.5
602.9
adapters
18.5
86.2
221.0
1620.0
Humor
47.2
860.8
StyleBART
24.3
36.7
560.5
/
/
26.2
88.2
1011.3
Romance
para
26.2
88.1
51.8
1275.6
x
x
24.3
87.4
36.7
560.5
adapters
18.9
86.4
209.6
1824.9
Romance
x
25.6
88.0
45.2
911.7
StyleBART
23.0
87.5
50.9
115.5
<
26.2
88.2
47.2
1172.4
Clickbait
para
24.2
87.7
43.3
125.8
x
23.0
50.9
-90 adapters
21.0
87.1
53.6
134.2
Clickbait
4
x
23.4
87.6
50.3
139.5
24.8
87.9
42.6
156.5
Table 6: Ablation different design choices for style adapter pretraining (Step 1). -para: StyleBART with adapter pretraining by the denoising task. -8 adapters: StyleBART without pretraining the style-less adapters.
Table 7: Ablation on different choices of trainable pa- rameters for headline generation fine-tuning (Step2). enc: the encoder part. catt: the cross attention part. dec: decoder part.
with the denoising task instead of the inverse para- phrasing task. StyleBART- adapters fine-tune the BART model without style-less adapters at step 2, thus without pretraining the style-less adapters. As can be seen, all methods perform similarly in style-less headline generation. When comes to stylistic headline generation, StyleBART-aa adapters decrease dramatically in both the summa- rization quality and the style control. The inverse paraphrasing task is crucial for style control, while slightly decreasing the relevance of the generated headline to the reference headline. This may be be- cause the reference headlines are style-less, while the headlines produced by using the inverse para- phrasing task have a stronger style.
when generating stylc-less headlines, indicating that fine-tuning only partial parameters is enough for learning the headline generation task. When comparing on the SHG task, only updating the en- coder can best control the style of the generated headlines among all fine-tuning methods. As the style adapters are inserted at the decoder, freezing the decoder and cross- cots-attention can better maintain their style control ability obtained during pretrain- ing.
4.5 Case Study
In this section, we present examples generated stylistic headlines, as shown Table 8. Again, we concentrate on StyleBART, TitleStylist, and S- SHG, as they outperform the other methods.
Trainable Parameters of Headline Generation Fine-tuning. We compare different choices of trainable parameters the headline generation fine- tuning step, as illustrated in Table As can be seen, all fine-tuning methods get similar scores
From this example as well as others, find that all three methods generate relevant and fluent head- lines. However, StyleBART is better at style con- trol. For example, the phrase "won't tt last forever"

---

Cyber monday has been the biggest single shopping day of the year for online retailers, retailers are spreading their online sales throughout
News Abstract
the thanksgiving holiday. As result, cyber monday's growth is flattening. Analyst: cyber monday will phase out eventually.
TidlStyliitt
Cyher monday: the biggest shopping day of the year?
Humor
Cyber monday is the biggest single shopping day of the year for online retailers
S-SHG
StyleBART
Cyher monday flattens out for retailers
TitleStylist
Cyher monday: the biggest single shopping day of the year?
Romance
Cyber monday is still the biggest single shopping day of the year for online retailers
S-SHG
StyleBART
Cyber monday won't last forever
TitleStylist
Cyber monday is flattening
Clickbait
Cyber monday has become the biggest single shopping day of the year for online retailers
S-SHG
StyleBART
Is cyber monday the end of online shopping?
Table 8: Examples of the stylistic headlines generated by different methods.
used the romantic headline by StyleBART is common expression in romantic contexts. Style- BART uses questions to raise the reader's curiosity in clickbaity headline. We also observe that when generating headlines of different styles, StyleBART produces more diverse results, while Siylettylist and S- SHS are more likely to change only few words or expressions.
Style
Method
BERT
PPL
PPL-S
NST
88.5
4.4
Style-less
StyleBART-N
88.5
4.7
Humor
-0.5
+0.4
-17.0
Romance
StyleBART
-0.4
+0.4
-14.4
Clickbait
-1.6
+23.1
-968.7
Table 9: Automatic evaluation results stylistic story telling. NST: fine-tune all BART parameters the plain story telling data. For StyleBART, we report the score changes with respect to the StyleBART-N model.
4.6 Extension to Stylistic Story Telling
To test whether StyleBART flexibly combine tasks and styles, we conduct experiments stylis- tic story telling. Specifically, giving the first sen- tence of a story, this task generates story follow-ups with a desired style. We use the ROCStorics dataset (Mostafazadeh et al., 2016) as the standard story telling dataset which contains around 100,000 sto- ries. We randomly select 4080 stories for both validation and test set and leave the rest as the train- ing set. With the dataset, fine-tune our base model following Step (Section 2.3) and com- bine the fine-tuned base model with existing hu- mor/romance/Clikkbai style adapters for inference. this way, we efficiently build a stylistic story telling model which supports three styles while only requiring fine-tuning once.
bined with different downstream tasks to control the style of the generated text.
Related Work
5.1 Headline Generation
Headline generation is the task of generating rele- vant and concise headlines for given news. It has various application scenarios, such automated news writing (Li al., 2022) and product advertis- ing (Kanungo et al., 2022).
Traditional approaches on headline genera- tion rely on linguistic features and handcrafted rules (Dorr et al., 2003; Knight and Marcu, 2002). With the advancement of neural networks, neural headline generation shows its capacity generate high quality headlines (Rush et al., 2015). However, controlling the style of these headlines remains challenging. Jin al. (2020) propose the first unsu- pervised stylistic headline generation model which relies only a standard headline generation dataset
Table shows the automatic evaluation results. As can be seen, when switching to the style adapters, the model generates more stylistic stories (PPL-S), while the generated stories become less relevant to the reference story follow-ups (BERT).
This consistent with stylistic headline generation, demonstrating that our style adapters can be com-

---

and mono-style corpora. They design multitask learning framework jointly learn both plain head- line generation and stylistic text denoising with carefully designed parameter sharing and switch- ing strategy. Zhan et al. (2022) further extend by decomposing the headline into style and content, They define stem and syntax as the style and gen- erate the style first using similar multitask frame- work Jin et al. (2020). After that, substantive context is populated into that style with the condi- tional masked language modelling task. However, these works jointly learn style and headline gener- ation, thus cannot support freely combination of styles and generation tasks at inference time.
Limitations
This work mainly explores the stylistic headline generation task. We leave the exploration of more combinations of tasks and styles such as stylistic machine translation, stylistic document summariza- tion as future work. At the same time, our cur- rent method achieves stylistic generation by sim- ply switching the adapters, which cannot provide fine- fine-grained control of the style. This hinders Stylc- BART from meeting the diverse user demands re- lated to style control.
Ethics Statement
We present a training-efficient approach to build an unsupervised stylistic headline generation model which disentangles the headline generation learn- ing and style learning. Despite the strong perfor- mance of style control, StyleBART inherits the cietal impacts including some negative ones of the original BART model, such as socictal biases (Mil- ios and BehnamGhader, 2022) and misuse of lan- guage models (Tamkin et al., 2021). The implicit biases are expected to be removed by debiasing either the dataset or the model (Meade et al., 2022; Zhou et al., 2022). StyleBART makes it possible to generate text in various (e.g. clickbait) style which be used to propagate the malicious or offensive content (Welbl et al., 2021). Future explorations are needed mitigate the misuse of StyleBART model.
5.2 Unsupervised Text Style Transfer
Text style transfer is to generate a sentence consis- tent with desired style while preserving the content of the source sentence (Shen al., 2017). Due to the scareity style parallel corpora, unsupervised text style transfer is widely explored previous works. Wu et al. (2019) regard text style trans- fer as cloze task to accomplish sentiment style conversion of sentences. Dai et al. (2019) incor- porates the sentiment style information through reconstruction task. Krishna et al. (2020) redefine the style transfer problem a paraphrase genera- tion problem and propose a method based on re- verse paraphrasing to generate the desired style. Lai et al. (2021) utilize large pretrained models such as BART and GPT-2, using BLEU scores and style classification scores as rewards, to achieve significant improvements in the transformation of formal and informal text. These models can be combined with plain headline generation model achieve stylistic headline generation. However, they require two-steps decoding at inference time, while StyleBART generates stylistic headlines in one decoding step.
A Acknowledgements
This project was supported by National Natural Science Foundation of China (No. 62106138, No. 62306132) and Shanghai Sailing Program (No. 21 21 We thank the anonymous review- ers for their insightful feedbacks on this work.
References
6 Conclusion
Ankur Bapna, Naveen Arivazhagan, and Orhan Firat. 2019. Simple, scalable adaptation for neural machine translation. arXiv preprint arXiv: 190.00999908777
We propose StyleBART, an unsupervised stylistic generation method, which enables to freely com- bine the downstream tasks and styles by disentan- gling style learning and downstream task learning. Experimental results show that our model can gen- erate content-relevant and style-intensive headlines, and can be extended to other stylistic generation tasks.
Ziqiang Cao, Furu Wei, Wenjie and Sujian Li. 2018. Faithful to the original: Fact aware neural abstractive summarization. In thirty-second AAAI conference on artificial intelligence.
Sumit Chopra, Michael Auli, and Alexander M. Rush. 2016. Abstractive sentence summarization with at- tentive recurrent neural networks. In Proceedings of

---

the 2016 Conference of the North American Chap- ter of the Association for Computational Linguisties: Human Language Technologies, pages 93-98, San Diego, California. Association for Computational Linguistics.
Huiyuan Lai, Antonio Toral, and Malvina Nissim. 2021. Generic resources are what you need Style trans- fer tasks without task-specific parallel training data. Proceedings of EMNLP.
Michael Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke 7ettlemoyer. 2020. Bart: Denoising requence-to-sqquene pre-training for nat- ural language generation, translation, and compre- hension. Proceedings of ACL.
Ning Dai, Jianze Liang. Xipeng Qiu, and Xuanjing Huang. 2019. Style transformer: Unpaired text style transfer without disentangled latent representation. Proceedings of the 57th Annual Meeting of the As- sociation for Computational Linguistics, pages 5997- 6007, Florence, Italy. Association for Computational Linguistics.
Zhengpeng Li, Jian Wu, Jiawei Miao, and Xinmiao Yu. 2022. News headline generation based on improved decoder from transformer. Scientific Reports. 12.
Dorr, David M. Zajic, and Richard M. Schwartz. 2003. Hedge trimmer: A parse-and-tre trim approach to headline generation. In HLT-NAACL 2003.
Yein-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. meeting of the association for computational linguistics.
Jiatao Gu, Yong Wang, Kyunghyun Cho, and Vic- tor O.K. 2019. Improved zero-shot neural ma- chine translation via ignoring spurious correlations. In Proceedings of the 57th Annual Meeting of the As- sociation for Computational Linguistics, pages 1258- 1268, Florence, Italy. Association for Computational Linguistics.
Junyang Lin, Sun, Shuming Ma, and Qi Su. 2018. Global encoding for abstractive summariza- tion. arXiv preprint ar5.vv1805.03989.
Nicholas Meade, Elinor Poole-Dayan, and Siva Reddy. 2022. An empirical survey of the effectiveness of debiasing techniques for pre- pre-trained language models. In Proceedings of ACL, pages
Karl Moritz Hermann, Tomas Kocisky, Edward Grefen- stette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in neural information processing systems, pages 1693-1701.
Aristides Milios and Parishad BehnamGhader. 2022. An analysis of social biases present in bert variants across multiple languages. ArXiv, abs/221 1.14402.
Nasrin Mostafazadeh. Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmect Kohli, and James Allen. 2016. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficiere transfer learning for nip. In In- ternational Conference on Machine Learning, pages 2790-2799, PMLR.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, and Pe- ter Szolovits. 2020. Hooks in the headline: Learning to generate headlines with controlled styles. meeting of the association for computational linguistics.
Jonas Pfeiffer, Ivan Vulié, Tryna Gurevych, and Sebas- tian Ruder. 2020, Mad-x: An adapter-basee frame- work for multi-task cross-lingual transfer. arXiv preprint a000022005.00052.
Yashal Shakti Kanungo, Gyanendra Das, and Sumit Negi. 2022. Cobart: Controlled, optimized, bidirec- tional and auto-regressive transformer for ad headline generation. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Min- ing.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei. Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9
Sylvestre-Alvis Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2018. Efficient parametrization of multi- domain deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Diederik P Kingma and Jimmy Ba. 2014. Adam: method for stochastic optimization. arXiv preprint 14Xiv:1412.6988
Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sen- tence summarization. In Proceedings of the 2015 Conference Empirical Methods in Natural Lan- guage Processing.
Kevin Knight and Daniel Marcu. 2002. Summarization beyond sentence extraction: A prohabilistic approach to sentence compression. Artif. Intell., 139:91-107
Kalpesh Krishna, John Wieting, and Mohit lyyer. 2020. Reformulating unsupervised style transfer para- phrase generation. empirical methods natural lan- guage processing.
Evan Sandhaus. 2008.
The new york times annotated corpus. Linguistic Data Consortium, Philadelphia. 6( 6(12):e26752.

---

Tianxiao Shen. Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non- con-parllel text by cross-alignment. Advances in neural information processing systems, 30.
1489-4495. International Joint Conferences on Arti- ficial Intelligence Organization. Main Track.
Tianyi Zhang*, Varsha Kishore*, Felix Wu Kilian Q- Weinberger, and Yoav Artzi. 2020. Bertscore: Eval- uating text generation with bert. In International Conference on Learning Representations.
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie- Yan Liu. 2019. Mass: Masked sequence to sequence pre-training for language generation. arXiv preprint arXiv:1905.024555
Kun Zhou, Beichen Zhang, Xin Zhao, and Ji-Rong Wen. 2022. Debiased contrastive learning of unsupervised sentence representations. Proceedings of ACL. pages 6120-6130.
Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Ma- heswaran. 2019. "transforming" delete, retrieve, gen- erate approach for controlled text style transfer. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Lan- guage Processing (EMNLP-IJCNIPP) pages 3269- 3279, Hong Kong, China. Association for Computa- tional Linguisties.
Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhut- dinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In The IEEE International Con- ference on Computer Vision (ICCV).
Alex Tamkin, Miles Brundage, Jack Clark, and Deep Ganguli. 2021. Understanding the capabilittes, limi- tations, and societal impact of large language models. Preprint arxiv:2102.02500
Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, An- thony Hartshom, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jemya Lee, Di- ama Liskovich, Yinghai Lu, Yuning Mao. Xavier Mar- tinet, Todor Mihaylov, Pushkar Mishra, Igor Moly- bog, Yixin Nie, Andrew Poulton, Jeremy Reizen- stein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subrama- nian, Xiaoqing Ellen Tan, Binh Tang, Ross Tay- lor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Ro- driguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama Open foundation and fine- tuned chat models.
Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hen- dricks, Kirsty Anderson, Pushmeet Kohli, Ben Cop- pin, and Po-Sen Huang. 2021. Challenges in detox- ifying language models. In Findings of EMNLP, pages 2447-2444222 2469.
Xing Wu, Tao Zhang, Liangjun Zang, Jizhong Han, and Songlin Hu. 2019. "mask and infill:: Applying masked language model sentiment transfer.
Jiaao Zhan, Yang Gao, Yu Bai. and Qianhui Liu. 2022. Stage-wise stylistic headline generation: Style gen- eration and summarized content insertion. In Pro- ceedings of the Thirty-Firsl International Joint Con- ference on Artificial Intelligence, IJCAI-22, pages

---

News Style Headline Generation Stage 1: Learning Text Styles
Task:
Stuge
Template
Inverse Paraphrasing Task
Please read the following representative text with particular style understand stylistic features. Style Text: ISTYLE TEXT]
Convert the following sentence to its
Instruction
[STYLE] version.
Input
Text: [PARAPHRASED TEXT]
Output
[STYLE] version text: ISTYLISTIC TEXT]
Training
Stage 2: Learning Headline Generation
Here are some examples of news and corresponding headlines to learn how to generate news headlines: News: [NEWS]
Headline Generation Task
Read following news abstract and write headline for
Instruction
Heudline: [HEADLINE]
Input
News: [NEWS]
Output
Headline: [HEADLINE]
Stage 3: Generating Stylized Headlines
Stylistic Headline Generation
Generate a news headline with the previously learned style based on the news text given ahove:
Read following abstract and
Instruction
(STYLE) version headline
Inference
News: [NEWS]
Style headline:
Input
[NEWS]
Output
[STYLE) version headline:
Table 10:
The data template used GPT3.5-
prompting.
Table 11:
The data template used for LLaMA2- InsTuning training and inference stage.
A
Appendix
A.3 Comparison with TitleStylis-BART
A.1 GPT3.5-prompting Details
TitleStylist is initialized using the MASS model, pretrained model with encoder-decoder struc- ture similar BART. We adopt the methodol- ogy employed in TitleStylist and apply it the BART model. We execute both -ASS (with their open-source code) and TitleStylist- BART (with our reimplemented code). As shown in Table 12, We can find that these two varia- tions yield similar ROUGE and BERTScore re- sults while TineeStylst-MASS gets better PPL. and PPL-S scores. Therefore, we present the results for TiteStyllst-MASS in this paper, it demon- strates better performance and is consistent with the original paper.
We perform few-shot prompting with the gpt-3.5- turbo API. We follow the unsupervised setup of StyleBART which assumes paired news-stylistic headlines are unavailable. As results, we use three non-parallel stylistic sentences, five news- plain headline pairs in data template. Table 10 shows the details. We the the temperature to 0.5.
A.2 LLaMA2-InsTuning Details
We sample 10,000 data points per task9 and con- struct the corresponding instruction data. Table 11 shows our data template in training and inference stages. We utilize the 40,000 instruction data jointly to fine-tune LLaMA2-7B with LoRA. The hyperparameters for the LoRA module are set as follows: lora_ranke = 256, toro_alpho = 256, and lora_ bora_dropout 0.03. The training process employs the AdamW optimizer with parameters 0.9, 32 0.999. We adopt learning rate of 2e 5. The training executed with batch size of 80 for one epoch, amounting to a total of 500 steps. After our training stage, we directly use the inference template in Table 11 to evaluate the fine-tuned model.
"Our training loss shows no significant reductions after 150 steps when using a butch size of 80. Therefore, we do not use the full datasct as larger dataset requires more computation and only brings marginal performance improvement.

---

Generation Quality RL(t)
Style Strength PPL-S(L)
Style
Method
RI(")
R2(t)
BERT(1)
PPL(L)
TiRR
28.2
10.0
24.9
87.7
68.5
839.3
Humor
TideeStylistMASS
27.7
24.5
87.5
39.4
640.9
TitcsRtYlit-BART
28.4
10.1
25.0
87.7
62.6
923.7
Romance
TideStyyist-MASS
27.6
9.9
24.5
87.5
39.5
740.1
BARR
28.2
9.8
24.9
87.7
48.2
192.9
Clickbait
TileSSylist-MASS
28.2
10.1
25.2
87.8
39.1
256.4
Table 12: Comparison results of TitleStylit- and Titeylist-MASS.


Reviews:
[{"review_id": "DsOb87Dy40", "reviewer": "Reviewer_ZP6W", "summary": "", "questions": "1. The TitleStylist paper used a encoder-decoder model and train it from scratch. In your experiments, do you use the method from TitleStylist and apply it to BART model as the baseline? If not, then some results are not fair. ", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1. As shown in Table 2 and 6, fine-tuning only the encoder of BART in step 2 can promote the style strength but it can also hurt the headline generation quality (RL score tells the difference).**\n\nR1. Thank you for the question. \n\n(1) According to Table 6 and line 381-385, finetuning only the encoder gets similar scores with other finetuning methods when generating style-less headlines, indicating that finetuning only the encoder is sufficient for learning on the headline generation task. \n\n(2) We agree that finetuning only the encoder gets lower RL score and BERTScore when generating stylistic headlines, as shown in Table 6. However, this does not necessarily mean that finetuning only the encoder hurts the headline generation quality.  Finetuning only the encoder generates more stylistic headlines (lower PPL-S score in Table 6). As is stated in line 308-312, more stylistic headlines would lose some RL score and BERTScore as 1) the reference headlines are style-less; 2) the stylistic headline may use more words outside the news body for improved style strength. \n\n\n\n**Q2. The baseline TitleStylist has proposed finetuning some layer normalization and cross-attention parameters in the decoder to learn the style, while this work proposes using adapter. Adapter has been introduced in 2020 and has been shown to be good at switching among different settings. So the innovation here is quite incremental.**\n\nR2. We agree that TitleStylist finetunes some layer normalization and cross-attention parameters in the decoder to learn the style, while StyleBART uses adapters to learn the style. However, our method is not simply switching the style control module.\n\n(1) We propose a modular solution for unsupervised stylistic text generation.  As pointed out by reviewer vmEE, our method is much more modular than TitleStylist. As shown in Figure 1, StyleBART can flexibly combine the downstream tasks and the styles at inference time, while TitleStylist needs retraining for each task and style combination. This is achieved through the proposed model architecture and pretraining-finetuning framework of StyleBART instead of just using adapters. \n\n(2) We propose the inverse paraphrasing task to learn the style adapters, while TitleStylist learns the style on the denoising auto-encoding task. As demonstrated in Table 5, the inverse paraphrasing task is important to strengthen the adapters' style control ability. \n\n**Q3. In this era of LLMs, I am wondering whether few-shot in context learning of a LLM like Vicuna-13B can accomplish this task very well.**\n\nR3. Thank you for your suggestion. We conduct few-shot experiments using GPT3.5 (gpt-3.5-turbo). We provide five stylistic sentences and three news-headline pairs in the prompt and query the model to generate stylistic headlines for the input news.  The experimental results show that our approach generates headlines with more relevant content (+4.1 average RL and +1.5 average BERTScore) and a stronger style (-1768.8 average PPL-S). We will add this in the revised version.\n\n| Style     | Method           | R1   | R2   | RL   | BERT | PPL    | PPL-S  |\n| --------- | ---------------- | ---- | ---- | ---- | ---- | ------ | ------ |\n| Humor     | GPT3.5-prompting | 22.7 | 5.9  | 19.5 | 85.8 | 1972.4 | 1161.3 |\n| Humor     | ours             | 28.2 | 10.0 | 24.9 | 87.6 | 42.5   | 602.9  |\n| Romance   | GPT3.5-prompting | 22.4 | 5.8  | 19.4 | 85.9 | 1561.2 | 1329.4 |\n| Romance   | ours             | 27.4 | 9.4  | 24.3 | 87.4 | 36.7   | 560.5  |\n| Clickbait | GPT3.5-prompting | 24.5 | 6.9  | 21.0 | 86.2 | 1074.0 | 4094.6 |\n| Clickbait | ours             | 25.3 | 7.7  | 23.0 | 87.5 | 50.9   | 115.5  |\n\n**Q4. The TitleStylist paper used a encoder-decoder model and train it from scratch. In your experiments, do you use the method from TitleStylist and apply it to BART model as the baseline? If not, then some results are not fair.**\n\nR4. Thank you for pointing out this. \n\n(1) Instead of training from scratch, TitleStylist is initialized from the MASS model, which is a pre-trained model similar to BART with an encoder-decoder structure.\n\n(2) We run both TitleStylist-MASS (with their open-sourced code) and TitleStylist-BART (with our reimplemented code). We find that they get similar ROUGE and BERTScore, while TitleStylist-MASS gets better PPL and PPL-S scores. Therefore, we report TitleStylist-MASS in our paper, which works better and is consistent with the original paper. We will add TitleStylist-BART in the revised version. \n\n| **Style** | **Method**        | **R1** | **R2** | **RL** | **BERT** | **PPL** | **PPL-S** |\n| --------- | ----------------- | ------ | ------ | ------ | -------- | ------- | --------- |\n| Humor     | TitleStylist-BART | 28.2   | 10.0   | 24.9   | 87.7     | 68.5    | 839.3     |\n| Humor     | TitleStylist-MASS | 27.7   | 10.0   | 24.5   | 87.5     | 39.4    | 640.9     |\n| Romance   | TitleStylist-BART | 28.4   | 10.1   | 25.0   | 87.7     | 62.6    | 923.7     |\n| Romance   | TitleStylist-MASS | 27.6   | 9.9    | 24.5   | 87.5     | 39.5    | 740.1     |\n| Clickbait | TitleStylist-BART | 28.2   | 9.8    | 24.9   | 87.7     | 48.2    | 192.9     |\n| Clickbait | TitleStylist-MASS | 28.2   | 10.1   | 25.2   | 87.8     | 39.1    | 256.4     |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This work shows good style strength as shown in Table 2 and enables one model for multiple styles with simply multiple sets of style adapters. \n2. The ablation study of different sets of parameters for headline generation training as shown in Table 6 is interesting and intriguing. \n3. The proposed paraphrasing method for learning the style from corpus is interesting. ", "reasons_to_reject": "1. As shown in Table 2 and 6, fine-tuning only the encoder of BART in step 2 can promote the style strength but it can also hurt the headline generation quality (RL score tells the difference). \n2. The baseline TitleStylist has proposed finetuning some layer normalization and cross-attention parameters in the decoder to learn the style, while this work proposes using adapter. Adapter has been introduced in 2020 and has been shown to be good at switching among different settings. So the innovation here is quite incremental. \n3. In this era of LLMs, I am wondering whether few-shot in context learning of a LLM like Vicuna-13B can accomplish this task very well. \n", "questions_to_authors": "1. The TitleStylist paper used a encoder-decoder model and train it from scratch. In your experiments, do you use the method from TitleStylist and apply it to BART model as the baseline? If not, then some results are not fair. ", "paper_topic_and_main_contributions": "This work proposes StyleBART, an unsupervised approach for stylistic headline generation. This method decorates the pretrained BART model with adapters that are responsible for different styles and allows the generation of headlines with diverse styles by simply switching the adapters. This work separates the task of style learning and headline generation, making it possible to freely combine the base model and the style adapters during inference. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fXvL5RoWal", "reviewer": "Reviewer_B3qY", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1. The setups for the task, and the contributions are limited. The idea is not novel and the paper is not more engaging.**\n\nR1. Thank you for the comments. Our contributions are :\n\n(1) State-of-the-art performance: we propose a simple and effective approach for the unsupervised stylistic headline generation task, and achieve SOTA performance in both automatic and human evaluation.\n\n(2) A more modular approach: our proposed method StyleBART is much modular than previous work, which supports flexible combinations of downstream tasks and styles at inference time.\n\n(3) Better style learning method: we propose inverse paraphrasing to learn the style, which works better than the previous denoising auto-encoding task.\n\nTo make the paper more engaging, we compare StyleBART with methods using LLM. We find that StyleBART works better. We will add this in the revised version.\n\n(1) Few-shot prompting on GPT3.5. We perform few-shot prompting with the gpt-3.5-turbo API. We provide five stylistic sentences and three news-headline pairs in the prompt and query the model to generate stylistic headlines for the input news. The experimental results show that our approach generates headlines with more relevant content (+4.1 average RL and +1.5 average BERTScore) and stronger style (-1768.8 average PPL-S). \n\n(2) Instruction tuning on LLAMA2. We conduct instruction tuning with LoRA on both the inversing paraphrasing task and the news headline generation task. Then we perform stylistic headline generation at test time. We sample 10,000 data points from each training dataset to train, as larger dataset requires more computation and only brings marginal performance improvement (Our training loss curve shows no significant reductions after 150 steps when using a batch size of 80.) The results show that StyleBART generates headlines with more relevant content (+1.8 average RL and +0.8 average BERTScore) and stronger style (-125.8 average PPL-S). \n\n| Style     | Method           | R1   | R2   | RL   | BERT | PPL    | PPL-S  |\n| --------- | ---------------- | ---- | ---- | ---- | ---- | ------ | ------ |\n| Humor     | GPT3.5-prompting | 22.7 | 5.9  | 19.5 | 85.8 | 1972.4 | 1161.3 |\n| Humor     | LLaMA-InsTuning  | 24.1 | 7.5  | 21.5 | 86.6 | 38.9   | 536.2  |\n| Humor     | ours             | 28.2 | 10.0 | 24.9 | 87.6 | 42.5   | 602.9  |\n| Romance   | GPT3.5-prompting | 22.4 | 5.8  | 19.4 | 85.9 | 1561.2 | 1329.4 |\n| Romance   | LLaMA-InsTuning  | 24.3 | 7.9  | 21.6 | 86.5 | 39.1   | 825.7  |\n| Romance   | ours             | 27.4 | 9.4  | 24.3 | 87.4 | 36.7   | 560.5  |\n| Clickbait | GPT3.5-prompting | 24.5 | 6.9  | 21.0 | 86.2 | 1074.0 | 4094.6 |\n| Clickbait | LLaMA-InsTuning  | 27.1 | 9.1  | 23.6 | 87.1 | 41.5   | 294.4  |\n| Clickbait | ours             | 25.3 | 7.7  | 23.0 | 87.5 | 50.9   | 115.5  |\n\n**Q2. There have been no tests to determine the statistical significance of the results.**\n\nR2. Thanks for the suggestion. We will add this in a revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. A novel scenario for stylistic headline generation. It is easy to implement and can achieve better performance.\n2. The paper is well-organized and easy to follow.\n3. This paper showcases thorough experiments on multiple datasets and various backbones. ", "reasons_to_reject": "1. The setups for the task, and the contributions are limited. The idea is not novel and the paper is not more engaging.\n2. There have been no tests to determine the statistical significance of the results.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposed an unsupervised stylistic headline generation model which contains three steps. It separates training the style and the generation modules, then combines them as needed. The experimental results are based on the CNN-NYT and three stylistic text datasets. The proposed StyleBART achieves SOTA performance in the unsupervised stylistic headline generation task.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "T8BtWCXVQk", "reviewer": "Reviewer_vmEE", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1. This paper would be more relevant to the current paradigms with experiments on modern large language models. This includes both open-source models like LLAMA [2] which can be fine-tuned, and closed-source models with just API access like ChatGPT / GPT3.5. The paper currently only does experiments on BART, which was introduced in 2019. It will be interesting to see how much better the proposed approach is after fine-tuning adapters or LoRA [3] on LLAMA, and how that compares to few-shot prompting on ChatGPT / GPT3.5. **\n\nR1. Thank you for your suggestion. We conduct comparison experiments with few-shot prompting on GPT3.5 and instruction tuning on LLAMA2.\n\n(1) Few-shot prompting on GPT3.5. We perform few-shot prompting with the gpt-3.5-turbo API. We provide five stylistic sentences and three news-headline pairs in the prompt and query the model to generate stylistic headlines for the input news. The results show that our method generates headlines with not only more relevant content (+4.1 average RL and +1.5 average BERTScore), but also stronger style (-1768.8 average PPL-S ). We will include this in the revised version.\n\n(2) Instruction tuning on LLAMA2. We conduct instruction tuning using LoRA on both the inversing paraphrasing task and the news headline generation task. Then we perform stylistic headline generation at test time. We sample 10,000 data points from each training dataset to train, as larger dataset requires more computation and only brings marginal performance improvement (Our training loss curve shows no significant reductions after 150 steps when using a batch size of 80.) The results show that StyleBART generates headlines with more relevant content (+1.8 average RL and +0.8 average BERTScore) and stronger style(-125.8 average PPL-S). We will add this in the revised version.\n\nWe do not conduct experiments that directly apply our framework to LLAMA2 for two reasons:\n\n(1) It is a common practice to use instructions to fine-tune the LLM like LLAMA2, instead of tuning with non-instruction data (this is what we do in our proposed framework). \n\n(2) BART is an encoder-decoder model while LLAMA2 is decoder only. As a result, we cannot directly apply StyleBART to LLAMA2 (for example, at step 2 we only finetune the encoder of BART but LLAMA2 has no encoder). \n\nAs a result, we conduct experiments on LLAMA2 with instruction tuning. Due to the limited computation resource and time, we will add the experiment results that directly apply StyleBART to LLAMA2 in the revised version.  \n\n| Style     | Method           | R1   | R2   | RL   | BERT | PPL    | PPL-S  |\n| --------- | ---------------- | ---- | ---- | ---- | ---- | ------ | ------ |\n| Humor     | GPT3.5-prompting | 22.7 | 5.9  | 19.5 | 85.8 | 1972.4 | 1161.3 |\n| Humor     | LLaMA-InsTuning  | 24.1 | 7.5  | 21.5 | 86.6 | 38.9   | 536.2  |\n| Humor     | ours             | 28.2 | 10.0 | 24.9 | 87.6 | 42.5   | 602.9  |\n| Romance   | GPT3.5-prompting | 22.4 | 5.8  | 19.4 | 85.9 | 1561.2 | 1329.4 |\n| Romance   | LLaMA-InsTuning  | 24.3 | 7.9  | 21.6 | 86.5 | 39.1   | 825.7  |\n| Romance   | ours             | 27.4 | 9.4  | 24.3 | 87.4 | 36.7   | 560.5  |\n| Clickbait | GPT3.5-prompting | 24.5 | 6.9  | 21.0 | 86.2 | 1074.0 | 4094.6 |\n| Clickbait | LLaMA-InsTuning  | 27.1 | 9.1  | 23.6 | 87.1 | 41.5   | 294.4  |\n| Clickbait | ours             | 25.3 | 7.7  | 23.0 | 87.5 | 50.9   | 115.5  |\n\n**Q2.** ***Minor*: it would be easier to compare between systems in Table 2,3,4 if a single aggregated score is reported, by combining the metrics using the sentence-level aggregation method in [1].**\n\nR2. Thank you for your suggestions. We will add the sentence-level aggregation method in [1] in a revised version. \n\n**Q3.** ***Minor*: the method introduced in the paper is quite similar to [1]. The main differences are --- 1) the use of style adapters instead of separate models; 2) evaluation on stylized headline generation and storytelling instead of style transfer.**\n\nR3. Thank you for your question. We agree that StyleBART is similar to [1] in learning the style with the inverse paraphrasing task. However, [1] only focuses on style transfer, while our focus is style control in specific tasks like headline generation/storytelling. This makes our method different from [1] in three aspects. \n\n(1) Our problem is more complex than [1]. While [1] only learns on the mono-style text dataset,  we learn on both the task (headline generation/storytelling) dataset and the mono-style text dataset. This makes our problem more complex as we need to carefully design model architecture and training framework to transfer learning between these two datasets.\n\n(2) The model proposed in [1] cannot directly applied in our setup except using the two-step-decoding method which first generates a plain headline and then injects style with [1]. However, the two-step-decoding method brings additional decoding cost and may suffer from the error propagation problem. \n\n(3) One of the key contributions of our method is to provide a modular solution for unsupervised style control in specific text generation tasks, which supports task and style combination at inference time. However, [1] fails to provide such solutions as it only focuses on unsupervised style transfer.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper studies a practically relevant problem in controlled text generation - stylized headline generation.\n\n2. The paper proposes a simple new approach for stylized headline generation, using paraphrasing and style adapters. Human evaluations show that this approach is more effective than existing methods in terms of style transfer accuracy, headline quality and fluency.\n\n3. This paper has a thorough set of experiments, and includes both automatic and human evaluation. The paper also has a good set of ablation studies showcasing the beneift of inverse paraphrasing and style adapters. Finally, the authors also add a qualitative analysis of the generated headlines. The authors also study the benefit of their adapters on stylised story generation, showcasing the modularity of their approach.", "reasons_to_reject": "1. This paper would be more relevant to the current paradigms with experiments on modern large language models. This includes both open-source models like LLAMA [2] which can be fine-tuned, and closed-source models with just API access like ChatGPT / GPT3.5. The paper currently only does experiments on BART, which was introduced in 2019. It will be interesting to see how much better the proposed approach is after fine-tuning adapters or LoRA [3] on LLAMA, and how that compares to few-shot prompting on ChatGPT / GPT3.5.\n\n2. *Minor*: it would be easier to compare between systems in Table 2,3,4 if a single aggregated score is reported, by combining the metrics using the sentence-level aggregation method in [1].\n\n3. *Minor*: the method introduced in the paper is quite similar to [1]. The main differences are --- 1) the use of style adapters instead of separate models; 2) evaluation on stylized headline generation and storytelling instead of style transfer.\n\n**Overall**: The paper is well written and has a good experimental setup. The proposed approach, while similar to [1], is effective at headline generation and much more modular than [1] due to the use of style adapters. However, weakness #1 is a major concern for me, which is what reduced my \"excitement score\". But I am open to further discussion and will raise the excitement score if the authors will add similar experiments on LLAMA in the camera ready version, or add some few-shot experiments with large language models as a baseline.\n\n[1] - https://arxiv.org/abs/2010.05700  \n[2] - https://arxiv.org/abs/2302.13971  \n[3] - https://arxiv.org/abs/2106.09685", "questions_to_authors": "", "paper_topic_and_main_contributions": "Stylized headline generation is the task of generating headlines in a particular style (e.g. humorous, clickbait, etc). This paper presents a new approach for stylized headline generation using style adapters. Adapters are small modules that are added to a pretrained model, and are trained to perform a particular task while keeping the rest of the model parameters fixed. This paper proposes to use one adapter to learn the task of headline generation, and another for the task of style transfer using inverse paraphrasing, a technique introduced in [1]. The entire approach operates in an unsupervised setting, and does not require any paired (article, stylized headline) data.\n\nThe authors conduct a thorough set of experiments on the task of stylized headline generation, and show that their approach is more effective than existing methods in terms of both automatic metrics and human evaluation. The authors also conduct several ablation studies, qualitative analysis and experiments on stylized story generation.\n\n[1] - https://arxiv.org/abs/2010.05700", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "L006: \"researches\" --> \"research\"  \nL044: \"requires\" --> \"require\""}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an unsupervised stylistic headline generation model.

The method is easy to implement and can achieve good performance.
The paper is well-organized and easy to follow.
The experiments are reasonably thorough, including both automatic and human evaluation.

Several reviewers comment that the novelty is limited and similar methods have been proposed before.
Inclusion of statistical significance would strengthen the results.
Also, lack of comparison to LLMs is hindering its relevance among current research directions.
Decision: Accept-Findings