Self-supervised Product Title Rewrite for Product Listing Ads (2022)

Abstract: Product Listing Ads (PLAs) are primary online advertisements merchants pay to attract more customers. However, merchants prefer to stack various attributes to the title and neglect the fluency and information priority. These seller-created titles are not suitable for PLAs as they fail to highlight the core information in the visible part in PLAs titles. In this work, we present a title rewrite solution. Specifically, we train a self-supervised language model to generate high-quality titles in terms of fluency and information priority. Extensive offline test and real-world online test have demonstrated that our solution is effective in reducing the cost and gaining more profit as it lowers our CPC, CPB while improving CTR in the online test by a large amount.

## introduction product listing ads (plas) are crucial online marketing tools for merchants to attract more customers and encourage them to click their ads. they have different names in various ads channels, for example, dynamic product ads in facebook and instagram, shopping ads on google, as shown in fig 1 . plas usually have a limit on display text length, for instance, in google shopping ads, users can see only the first 70 or fewer characters of the title 4 ). therefore, plas titles are expected to reveal the product type and core attributes earlier so that users can clearly identify the product, as illustrated in existing work has made the attempt to generate titles from keywords(de @xcite and product images @xcite , or generate description text @xcite for products, however, little work has investigated the title optimization for plas. at first, we explored the rule-based method by assigning weight to attribute words and reordering the words/chunks by weight. however, the rule-based method heavily relies on the accuracy of attribute detection, phrase boundary detection, and the appropriateness of attribute weights. it is hard to optimize rules without exhausting human effort. therefore, we attempt to use language models in our title rewrite task. the biggest obstacle of model-based method is the lack of high-quality titles regarding fluency and information order as labels for supervised learning. in this work, we solve the problem by performing self-supervised learning. instead of writing highquality titles as labels, we design a multi-level shuffle module that uses titles to generate low-quality pseudo titles. then the language model is trained on (pseudo title, title) pairs, during which it can learn to reorder the words to recover the original titles. moreover, we propose a multi-step training procedure consisting of pre-train and fine-tune to enable the model to generate good titles. the overall framework of our solution is illustrated in fig 2 . moreover, for the sake of information accuracy, we only focus on information reordering and avoid any word insertion, deletion, title suitable for apple 12pro mobile phone case iphone12 protective case genuine leather drop-resistant new style all-inclusive silicone ultra-thin 11pro max high-end for men and women limited optimized mobile phone case silicone ultrathin genuine leather protective case drop-resistant suitable for apple 12pro iphone12 11pro max allinclusive new style high-end for men and women limited. ## method we first introduce our multi-level shuffle module, which creates the (pseudo title, title) pairs for selfsupervised training. then we elaborate on the multi-step training procedure. it is worth noting that our solution can be built upon any language models, such as bart @xcite and gpt2 @xcite . we use the prophet-net @xcite framework in practice as it is superior to bart and gpt2 and has achieved new state-of-the-art in multiple text generation tasks(dayiheng liu and duan, 2020). ## experiment we conduct offline and online test to evaluate the generated title in terms of accuracy, information order, fluency, and real-world profits. ## discussion besides titles, merchants usually also have a great deal of information such as product categories, attributes, etc. we experiment further on how such information helps in model training. ## conclusion we present a practical solution of product title optimization for plas which consists of multi-level shuffling for pseudo title production and multi-step training to generate high-quality titles. it can help merchants conveniently build their own profitable title optimization systems.