Can Pretrained Language Models Generate Persuasive, Faithful, and Informative Ad Text for Product Descriptions? (2022)

Abstract: For any e-commerce service, persuasive, faithful, and informative product descriptions can attract shoppers and improve sales. While not all sellers are capable of providing such interesting descriptions, a language generation system can be a source of such descriptions at scale, and potentially assist sellers to improve their product descriptions. Most previous work has addressed this task based on statistical approaches (Wang et al., 2017), limited attributes such as titles (Chen et al., 2019; Chan et al., 2020), and focused on only one product type (Wang et al., 2017; Munigala et al., 2018; Hong et al., 2021). In this paper, we jointly train image features and 10 text attributes across 23 diverse product types, with two different target text types with different writing styles: bullet points and paragraph descriptions. Our findings suggest that multimodal training with modern pretrained language models can generate fluent and persuasive advertisements, but are less faithful and informative, especially out of domain.

## introduction generative pretrained language models such as gpt-2 @xcite , t5 @xcite , and bart @xcite have led to impressive gains in language generation applications beyond machine translation, such as story geneneration @xcite , summarization @xcite , and dialogue systems @xcite . although such transformer-based language models @xcite are capable of generating fluent texts through a sequence-to-sequence framework, they still suffer from unfaithfulness and factuality issues @xcite @xcite . in this paper, we comprehensively discuss the utility of modern pretrained language models over an ad text generation task for product descriptions, figure 1 : top: input consisting of an image and textual attributes of a product. bottom: two target texts: bullet points and a paragraph description with a focus on faithfulness, persuasiveness, and informativeness. while previous work has been limited to short ad generation tasks conditioned on titles @xcite , and used traditional neural models @xcite or statistical approaches @xcite , we focus on a data-to-text generation approach to product description generation for an english e-commerce service. specifically, we explore various textual attributes and images as the input, and generate two types of product descriptions: (1) bullet points, and (2) paragraph descriptions (see figure 1 ). bullet points provide a list of key information regarding a product, while paragraph descriptions are made up of sentences structured into a coherent narrative. we argue there are two underlying motivations for the ad text generation task, especially for product descriptions. application-wise, the utility is to improve the seller experience for e-commerce services when registering a new product. the generated descriptions can reduce the need for manual data entry, and potentially improve sales due to better descriptions (in terms of attractiveness, structure, and persuasiveness). research-wise, ad text generation is an under-studied task, and arguably a good proxy for persuasive text generation @xcite @xcite . while previous work has discussed ad text generation of e-commerce service for a few product types such as fashion @xcite , computers @xcite , and house decor @xcite , in this work, we use twenty diverse product types and an additional three product types for out-of-domain prediction. with this setting, we aim to study model generalization and robustness over in-domain and out-of-domain test sets. to summarize our contributions: (1) we study the application of modern pretrained language models based on data-to-text generation for product description in an e-commerce service; (2) we explore multimodal training by incorporating image features for ad generation and perform automatic and manual evaluation; (3) we study model robustness for out-of-domain prediction; and (4) we conduct analysis of attributes that significantly contribute to ad text generation. ## related work data-to-text generation is the task of translating a semi-structured table to natural text, and has been applied in different real-world scenarios, such as weather forecasting reports @xcite , sport @xcite , health-care descriptions @xcite , and biographies @xcite . while the goal of most previous tasks is to generate descriptive text, there are few studies @xcite on data-to-text generation for the advertisement domain, and the work that has been done has tended to focus exclusively on the product type of computer and be based on pre-neural statistical approaches and templatebased techniques. previous work has mostly used titles of ecommerce products to generate short ads in chinese @xcite and @xcite . similarly, @xcite generate a product description for chinese e-commerce, conditioned on the title and a small number of attributes (with an average length of six words). 1 in this work, we comprehensively study product description generation in english based on ten diverse attributes (à la a data-to-text scheme, with the average number of 1 these attributes are not clearly described in the paper. ## data construction we use 200,000 e-commerce products spanning 20 different product types as described in figure 2 . for copyright reasons we are not able to release this data to the public. this dataset is randomly split into 180k/10k/10k training, development and test instances, respectively. we also create an xtreme test set (4,266 samples) in which we filter out test samples that have overlapping descriptions with the training data. lastly, we additionally use three different product types as an out-ofdomain test set, comprising 1,000 products of each of the three produce types: saree, computer, and cellular_phone. in total, there are three different test sets: (1) main; (2) xtreme; and (3) out-of-domain. in ## analysis which attributes contribute to ad generation? to answer this question, we performed an ablation study using the bart models. we decode both bullet points and paragraph descriptions using different numbers of at-tributes as context, and report the average automatic performance in we observe there are three prominent attributes for this task -title, brand, and keyword -for both bullet points and paragraph descriptions. interestingly, using only title can produce 32.98 and 29.93 average performance, and adding keyword to the input boosts performance by 11.05 and 10.57, for bullet points and paragraph descriptions, respectively. ## discussion and conclusion in this work, we described the first attempt at multimodal training for ad generation by incorporating image representations and text embeddings as input. we found that multimodal training yields the best performance in terms of overall scores in the both automatic and manual evaluation. we observe that modern pretrained language models can generate fluent advertisements, but are less faithful and informative, especially in out-of-domain settings. can pretrained language models generate persuasive, faithful, and informative ad text for product descriptions? the answer to this question is yes to a certain extent, particularly for in-domain scenarios. and although the bart models have similar human faithfulness performance to the gold texts, we believe that it does not necessarily imply that they are factually correct and further validation is necessary. one way forward may be to allow human judges to have access to some external knowledge (e.g. search engines or product catalogues), which will help them assess the factuality of the generated texts. furthermore, since the product descriptions in our e-commerce dataset might introduce new information, retrieval augmented generation @xcite @xcite is one potential direction for future work. this is because information on some products is likely to be available on the internet, and incorporating it into the generation model could potentially improve the resulting ad text.