Reinforced Target-driven Conversational Promotion (2023)

Abstract: The ability to proactively engage with users towards pitching products is highly desired for conversational assistants. However, existing conversational recommendation methods overemphasize on acquiring user preferences while ignore the strategic planning for nudging users towards accepting a designated item. Hence, these methods fail to promote specified items with engaging responses. In this work, we propose a Reinforced Target-driven Conversational Promotion (RTCP) framework for conversational promotion. RTCP integrates short-term and long-term planning via a balanced gating mechanism. Inside which, the dialogue actions are predicted via a knowledge-integrated multi-head attention and guided via reinforcement learning rewards. RTCP then employs action-guided prefix tuning to generate relevant responses. Experimental results demonstrate that our model outperforms state-of-the-art models on both automatic metrics and human evaluation. Moreover, RTCP has a strong capability in quickly adapting to unseen scenarios just by updating prefix parameters without re-training the whole model.

## introduction in recent years, conversational recommender systems (crss) @xcite @xcite @xcite @xcite @xcite have gained considerable attention from both academic researchers and industrial practitioners. inside such systems, most crs models will first infer user preferences via multi-turn conversations and then recommend a set of potential items when appropriate. the essential goal for such models is to solicit user interests accurately and then map such interests to some specific items in repository. however, a more challenging scenario would be target-driven conversational promotion, where the system mimics a salesperson to nudge the users towards accepting a pre-given item. as illustrated in figure 1 , the bot tries to recommend a given target 'kung fu panda 3' to the user via multiple turns of conversation. it gradually nudges the user via chatting about 'jaychou', on his well-known achievements in music, and then recommends his movie, which helps the user to get interested and accept the target with higher chance. such a scenario is highly desired due to the potential for boosting sales revenue and reducing human costs but also calls for higher requirements on wisely promoting items and proactively engaging with users @xcite . although existing crs models @xcite @xcite @xcite @xcite have achieved promising results, they tend to adopt a relatively passive scheme-mainly guessing user preferences via conversations, hence cannot be directly applied to the new proactive nudging setting. generally speaking, the new conversational pro-motion setting brings in two new obvious challenges that press for solutions. firstly, as the virtual assistant needs to converse gradually to convince the user in accepting the target item, it is essential to produce an appropriate plan, e.g. in a sequence of action tuples, that leads to the target item while can also keep the user interested in the conversation. secondly, once the plan is predicted, the other important requirement is to generate proper responses that are in line with the plan accurately. also, it is likely that the model would encounter new conversation scenarios with unseen dialogue plans. the model should be able to adapt to such scenarios quickly and handle them properly, in order to avoid disappointing the users. to address the aforementioned challenges, we propose a systematic framework-reinforced target-driven conversational promotion (rtcp) for the new setting. for planning, rtcp integrates long-term planning and short-term planning via a balanced gating mechanism. the former takes care of achieving the predefined target, while the latter looks into engaging the user and the smoothness of the conversation. specifically, rtcp predicts action tuples in a plan via knowledge-integrated multi-head attention and is guided via estimated reinforcement learning rewards. for response generation, rtcp employs a novel action-guided prefix tuning method to regulate the response generation being in line with the plan. in particular, it pushes action plan factors into different prefix parameters, while leaves the main generation task to the pre-trained language model. this helps the model to learn and adapt better, especially when new action plans happen. to sum up, our contributions are threefold: ## preliminary notations. we denote by g the set of all goals (e.g. chit-chat, movie recommendation, etc), and denote by p the set of all topics (e.g the conjuring, jaychou, etc). at the t-th turn, we represent an action tuple a t by a pair of goal g t and topic p t (i.e a @xmath0, p t )). each conversation @xmath1, k, a) is a tuple of three elements namely the conversation content c, a set of all associated knowledge @xmath2. the conversation context @xmath3, y t )} m @xmath4. the goal of conversational promotion is to interact with the users to persuade them to accept a targeted item. we decompose the problem into 2 sub-tasks including (1) the dialogue strategy planning and (2) the response generation tasks. dialogue strategy planning: formally, given a designated item @xmath5. we aim to produce a plan â = {â 1 , â2 , ..., âm } which is a sequence of action tuples to please the users and recommend the targeted item when appropriate. response generation: for the t-th turn, given historical context x t , knowledge base k and a predicted action tuple ât , we aim to generate a coherent response ŷt to the user. ## methodology we show the overall framework in figure 2 . our rtcp consists of two components namely planning in section 4.1 and generation in section 4.2. ## conclusion in this work, we focused on a promising yet underexplored setting called target-driven conversational promotion. we proposed a novel rtcp framework to produce dialogue action tuples which direct the conversation to a targeted item while also please the user with interesting and relevant topics. it manages to properly generate responses that are closely in-line with the plan while also can adapt quickly to new scenarios. we conducted extensive experiments to compare with a bunch of representative baselines over a rich set of metrics. both automatic evaluation and human evaluation results showed that rtcp significantly outperformed state-of-theart approaches on many important aspects.