End-to-End Conversational Search for Online Shopping with Utterance Transfer (2021)

Abstract: Successful conversational search systems can present natural, adaptive and interactive shopping experience for online shopping customers. However, building such systems from scratch faces real word challenges from both imperfect product schema/knowledge and lack of training dialog data. In this work we first propose ConvSearch, an end-to-end conversational search system that deeply combines the dialog system with search. It leverages the text profile to retrieve products, which is more robust against imperfect product schema/knowledge compared with using product attributes alone. We then address the lack of data challenges by proposing an utterance transfer approach that generates dialogue utterances by using existing dialog from other domains, and leveraging the search behavior data from e-commerce retailer. With utterance transfer, we introduce a new conversational search dataset for online shopping. Experiments show that our utterance transfer method can significantly improve the availability of training dialogue data without crowd-sourcing, and the conversational search system significantly outperformed the best tested baseline.

## introduction search systems play significant roles in today's online shopping experience. in conventional ecommerce search systems, user interacts with the system through typing of keywords, followed by product clicks or keywords modifications, depending on whether returned product list matches with user expectation. the recent success of intelligent assistants such as alexa, google now, and siri enables user to interact with search systems using natural language. for online shopping in particular, it becomes alluring that users can navigate through products with conversations like traditional in-store shopping, guided by a knowledgeable yet thoughtful virtual shopping assistant. however, building a successful conversational search system for online shopping faces at least two real world challenges. the first challenge is the imperfect product attribute schema and product knowledge. while this challenge applies also to traditional search systems, it is more problematic for conversational search because the later depends on product attributes to link lengthy multi-turn utterances (in contrast to short queries in conventional search) with products. most previous conversational shopping search work @xcite @xcite looks for the target product through direct attribute matching, assuming availability of complete product knowledge in structured form. in practice, this assumption rarely holds, and systems designed with this assumption will suffer from product recall losses. the second challenge is the lack of in-domain dialog dataset for model training. constructing a large-scale dialog dataset by crowd-sourcing from scratch is inefficient. popular approaches include machines-talking-to-machines (m2m) @xcite , which generates outlines of dialogs by self-play between two machines, and wizard-of-oz (woz) @xcite , which collects data through virtual conversations between annotators. note that both approaches require manually written utterances. in addition, a line of other work @xcite @xcite constructs conversations from the review datasets such as amazon product data @xcite and lastfm 1 , whereas usage of these datasets is limited to sub-tasks (e.g., dialog policy) due to the absence of user utterances. @xcite collected a dialog dataset for fashion product shopping. however, the described method requires dozens of domain experts to manually create the dialog, and the dataset can hardly be generalized beyond fashion shopping given the lack of utterance annotations. to address the first challenge of imperfect attribute schema and product knowledge, we propose convsearch, an end-to-end conversational search system that deeply combines the dialog and search system to improve the search performance. in particular, the product search module leverages both structured product attributes and unstructured product text (e.g. profile), where the product text may contain phrases matching with utterances when schema is incomplete or when a product attribute value is missing. putting together, our system has the advantage of both reduced error accumulation along individual modules, and enhanced robustness against product schema/knowledge gaps. to address the second challenge of lacking indomain dialog dataset, we propose a jump-start dialog generation method m2m-ut which 1) generates utterance from existing dialogues of similar domains (e.g., movie ticketing @xcite ), and 2) builds dialog outlines from e-commerce search behavior data, and fills them with the generated utterances. the proposed approach significantly reduces manual effort in data construction, and as a result we introduce a new conversational shopping search dataset csd-ut with 942k utterances. note that although the dialogue dataset construction focuses on shopping, the approach described here can be adapted for other task-oriented conversations as well, which we will leave it to future work. our contributions are summarized as follows: conversational search system conversational search task aims to understand user's search intents through multi-round conversational interactions, and return user the desired search item. due to lack of annotated dialog utterances in particular for conversational search tasks, previous work either adopted rule-based utterance parsing or focused only on dialog policy. yan et al. (2017) proposed a rule-based approach to cold-start online shopping dialog systems utilizing user search logs and intent phrases collected from community sites. in another line of work, luo et al. (2020) and zhang et al. (2018) utilized amazon review dataset, lei et al. (2020) and li et al. (2018a) revised user reviews from yelp 3 and lastfm 4 , all of which focused on the conversation policy without utterance understanding. as a comparison, in this paper we focused an end-to-end conversational search system, which fuses both utterance understanding and product search together through multi-task learning. constructing dialog dataset for online shopping rastogi et al. (2020) proposed a crowd-sourcing version of wizard-of-oz (woz) paradigm for collecting domain-specific corpora. in this system, users and wizards were given a predefined task to complete (e.g. find a chinese restaurant in the north). to avoid the distracting latency, users and wizards were asked to contribute just a single turn for each dialogue. saha et al. (2018) built a multi-mode dialog system for fashion with experts and in-house labors. they crawled 1 million fashion items from the web, hand-crafted taxonomy for items, identified the set of fashion attributes, and employed experts to write dialogs. the described methods were highly labor-consuming, and the published dataset did not contain attribute annotation on utterances, making state tracker ⨁ u t s t user dialog policy nlg agent: hi, how can i help you! user: i would like a dark roast coffee. agent: do you have a flavor in mind? user: let's try vanilla. roast_type<:> dark roasted<;> flavor<:>vanilla<;> dialog state … agent action: request(brand) agent response: what brand do you prefer? p 0 p 1 p 2 ⋯ p j multi-head attention product ranking list: [1, 0, 2, 5…] product search bidirectional transformer encoder autoregressive transformer decoder profile starbucks via instant coffee, vanilla latte, 5 count (pack of 1) attributes brand: starbucks item-type: instant-coffee flavor: null roast_type: dark roast caffeine_type:caffeinated … profile starbucks via instant coffee, vanilla latte, 5 count (pack of 1) attributes brand: starbucks item-type: instant-coffee flavor: null roast_type: dark roast caffeine_type:caffeinated … profile starbucks via instant coffee, vanilla latte, 5 count (pack of 1) attributes brand: starbucks item-type: instant-coffee flavor: null roast_type: dark roast caffeine_type:caffeinated … product knowledge base user utterance: starbucks sounds good. product representation q t p 0 p 1 d 0 d 1 ⋯ d j a 0 a 1 ⋯ a j ⋯ d 2 a 2 transformer transformer figure 1: illustration of our end-to-end conversational search system. state tracker module takes utterances to predict dialog state s t using the seq-to-seq transformer. product search module matches products represented by transformers with query representations q t using a multi-head attention mechanism. dialog policy module takes inputs from s t , intent and ranked product list, and decides the responses. nlg module composes system responses as instructed by dialog policy, and displays them to user. it hard for utterance understanding model training. the approach adopted by yan et al. (2017) mines phrases of shopping from community sites and uses crow-sourcing to label utterance intents. although costing less labors, this work did not construct the full dialogs. as a comparison, in this paper we constructed the full shopping search dialogs through real user behavior data, with user utterances filled by transferring from existing dialogs of similar domains. 3 end-to-end conversational search with multi-task learning our conversational search system, depicted in figure 1, consists of four major modules: state tracker (st), product search, dialog policy, and natural language generation (nlg). the state tracker module interprets the dialog content, outputs user intent, along with product attributes that the user is interested in. the product search module returns a list of products which are matching with tracked attributes user interested in. based on output from state tracker, the dialog policy manages agent response according to user intents and the candidate search result, and the nlg module transforms the response into natural language displayed to user. to address the challenge of imperfect product attribute schema/knowledge, our product search module leverages both structured product attributes and unstructured text. to mutually benefit from each other's learning, we integrate state tracker and product search together through multi-task learning, and build an end-to-end trainable search system. ## dialogue dataset construction we address the challenge of lack of conversational shopping search training data by proposing m2m-ut, a method that automatically constructs dialog datasets. unlike previous works @xcite that rely on crow-source to generate utterances, m2m-ut can automatically generates utterances with transfer. we hypothesize that the conversation between the user and the shopping agent is guided by customer's intents that 1) span user's utterance in natural language, and 2) change according to agent's responses. therefore, our dataset construction has two steps: 1) we use utterance transfer (ut) to generate utterances from existing dialog datasets of similar domains, and 2) we generate the outline of dialog using customer browsing records using machine talking to machine (m2m) @xcite . ## conclusion and future work in this work, we built an end-to-end conversation search system for online shopping, where we deeply combined the dialog and search system with multi-task learning. in particular, our product search module leverages both product attribute and text to retrieve products, which mitigates the imperfect product schema/knowledge challenges. to address issue of lacking in-domain dialog dataset, we proposed a dataset transfer method and constructed shopping dialog dataset from user search behavior data and existing dialogs of similar domain. the proposed dataset construction method lowers the cost, making it possible to scale-up to broader use scenarios. we will leave it to future work to expand the methodology across more shopping categories, and broader use scenarios such as clinical conversations and customer service, etc.