Interactive Latent Knowledge Selection for E -Commerce Product Copywriting Generation (2022)

Abstract: As the multi-modal e-commerce is thriving, high-quality advertising product copywriting has gain more attentions, which plays a crucial role in the e-commerce recommender, advertising and even search platforms. The advertising product copywriting is able to enhance the user experience by highlighting the product’s characteristics with textual descriptions and thus to improve the likelihood of user click and purchase. Automatically generating product copywriting has attracted noticeable interests from both academic and industrial communities, where existing solutions merely make use of a product’s title and attribute information to generate its corresponding description. However, in addition to the product title and attributes, we observe that there are various auxiliary descriptions created by the shoppers or marketers in the e-commerce platforms (namely human knowledge), which contains valuable information for product copywriting generation, yet always accompanying lots of noises. In this work, we propose a novel solution to automatically generating product copywriting that involves all the title, attributes and denoised auxiliary knowledge. To be specific, we design an end-to-end generation framework equipped with two variational autoencoders that works interactively to select informative human knowledge and generate diverse copywriting.

## introduction traditional e-commerce platforms solely present a list of products to customers. nowadays, as the multi-modal recommender systems are thriving, the e-commerce platform ecosystem has also been enriched with multi-modal forms, such as product figure 1 : an example of product copywriting generation from our dataset, including product title, attribute, details as well as the product description. the product title and a list of product attributes are always taken as the main model input to generate the product copywriting, exemplified by figure 1 . recently, @xcite proposed to involve the external knowledge (i.e., wikipedia) into the product title and attributes during the generation process of product copywriting. however, the external knowledge and customer reviews are not always available. for example, thousands of newly released products are emerging in the e-commerce platforms, where the external or the customer reviews are not accessible while automatically generating advertising copywriting is critical for such new products to improve the click-through rate and the conversion rate. in practice, we observe that each product (e.g., the hot and newly released items) is accompanied with various product details in the e-commerce platforms (e.g., taobao, jd, and amazon). the product title and associated attributes summarize the main information of a product, while the product details comprise of auxiliary advertising descriptions created by shoppers and marketers which contain salient information that highlight the product properties with advertising phrases (i.e., human knowledge) and thus are beneficial to improving the quality of the generated copywriting. exemplified by figure 1 , the product title and attributes summarize the main functions of "xiaomi box se", while the product details elaborate the product with slogans that are attractive to customers, like "voice control" and "switch channels and adjust the volume by voice". unfortunately, we also observe that such human knowledge also contains redundant pieces, like "i want to watch action video", which might harm the quality of the generated copywriting. one recent work @xcite simply concatenated all the product details with product title and attributes as the model input, without considering the noises contained by the product details. in this work, we propose to select the salient knowledge from the auxiliary product details before we feed such information, associated with product title and attributes, into the model. to be specific, we propose an interactive latent variables model based on transformer architecture @xcite (i.e., ilvt), which is designed to select knowledge from noisy product details and incorporate the selected knowledge into the process of product copywriting generation. to enhance the connection between the process of the knowledge selection and copywriting generation, we sample latent variables from prior description and knowledge latent space separately. during generation phase, ilvt will firstly sample the description latent variable from the description distribution conditioned on the product title and attributes, then sample the knowledge latent variable from knowledge distribution conditioned on product details and the description latent variable. with the interactive latent variables, ilvt can also generate copywritings with strong diversity. without latent variable modules, ilvt degenerates into transformer with copy mechanism, which is a traditional method for copywriting generation in e-commerce platform @xcite . to the best of our knowledge, this is the first work that selects knowledge from product details to improve the generation quality and diversity in product copywriting generation task. to evaluate our proposed method, we collected a chinese product copywriting dataset from the jd platform, named jdk. the dataset consists of 220,000 instances, each of which comprises of a product's title, attributes, details as well as the corresponding description. results on such dataset shows that our proposed method obtains the best performance compared to all baselines, in terms of both automatic and human evaluations. ## experiments we conducted experiments on the jdk dataset. ## implementation details we implement our models on the tesla v100 gpus. for the transformer-based model, the hidden units is 512 and feed-forward hidden size is 2048. both the encoder and decoder has 6 layers with 12 heads. the beam size is 5. the sentence length of title, attributes, description and knowledge are 128, 64, 128 and 512 tokens, respectively. the dropout rate is 0.1. we choose the adam optimizer with β 1 = 0.9 and β 2 = 0.998. the warm-up step is set to 4000 and learning rate is 0.0001. the batch size is 32. to avoid the kl-vanishing problem, we choose kl-annealing trick @xcite with the α=0.00025 and β = 6.25 for both two vaes. hyperparameters are set based on the performance of the validation set. ## conclusion this work studies a novel problem on how to generate informative and diverse product copywriting with auxiliary human-created product details. we propose an interactive latent variables model based on transformer architecture, ilvt, which allows to select salient knowledge from the noisy product details. to better evaluate ilvt model, we construct a large chinese product copywriting dataset, jdk. extensive experiments demonstrate that our proposed model outperforms the baselines with regard to both automatic and human evaluation, illustrating that ilvt can select outstanding knowledge and improve the generation quality and diversity. product title 中 国 风 ::::::::: 圆领印花时 尚 ::::::::: 宽松大码t恤 女 款 夏 季 新 款 ::::: 显瘦短 袖 黑 色 印 花 预 售t恤 chinese style, ::::: round ::::: neck :::::::: printing ::::::: fashion, ::::: loose t-shirt, for women in new summer fashions, ::::::: looking :::: slim with short sleeves and black printing, pre-sale t-shirt attribute set 服饰内衣；t恤；风虎； :::: 圆领； ::::: 印花；时尚； :::: 宽松；大码；女；夏季； 新款；短袖 clothing underwear; t-shirt; recluserecluse; :::::: round ::::: neck :::: colla; printing; fashion; loose design hiding obesity and ::::::: showing :::: slim, the t-shirt adopts ::::: loose :::::: design, unconstrained on figures, slightly chubby body can also handled it easily. the classic ::::: round ::::: neck design fits the neckline, comfortable for wearing. transformer+pse ::::::::::::: 圆领印花宽松版型 :::: 圆领的设计，穿着舒适自在，同时也能修饰颈部线 条，凸显修长的脖颈线条。 ::::: 宽松的版型，不挑身材，遮肉 :::: 显瘦，适合多 种身材。 :::::: round :::: neck :::::::: printing ::::: loose :::::: design. round neck design, comfortable for wearing, embellish and highlight the slender neck line. ::::: loose :::::: design, unconstrained on figures, hiding obesity and :::::::