Chatbot is increasingly thriving in different domains , however , because of unexpected discourse complexity and training data sparseness , its potential distrust hatches vital apprehension . Recently , Machine-Human Chatting Handoff ( MHCH ) , predicting chatbot failure and enabling human-algorithm collaboration to enhance chatbot quality , has attracted increasing attention from industry and academia . In this study , we propose a novel model , Role-Selected Sharing Network ( RSSN ) , which integrates both dialogue satisfaction estimation and handoff prediction in one multi-task learning framework . Unlike prior efforts in dialog mining , by utilizing local user satisfaction as a bridge , global satisfaction detector and handoff predictor can effectively exchange critical information . Specifically , we decouple the relation and interaction between the two tasks by the role information after the shared encoder . Extensive experiments on two public datasets demonstrate the effectiveness of our model . Chatbot , as one of the recent palpable AI excitements , has been widely adopted to reduce the cost of customer service ( Qiu et al . , 2017 ; Ram et al . , 2018 ; Zhou et al . , 2020 ) . However , due to the complexity of human conversation , auto-chatbot can hardly meet all users ' needs , while its potential failure perceives skepticism . AI-enabled customer service , for instance , may trigger unexpected business losses because of chatbot failures ( Radziwill and Benton , 2017 ; Rajendran et al . , 2019 ) . Moreover , for chatbot adoption in sensitive areas , such as healthcare ( Chung and Park , 2019 ) and criminal justice ( Wang et al . , 2020a ) , any subtle statistical miscalculation may trigger serious health and legal * Corresponding authors . What a business ! It has been a week ! utter 3 We will ship goods about a week after placing the order . Please be patient . utter 2 Sorry , my dear customer . We have pushed the warehouse to ship as soon as possible , and we will compensate the freight for you . utter 7 We will try our best to improve your shopping experience . Thank you for your understanding and patience . consequences . To address this problem , recently , scholars proposed new dialog mining tasks to autoassess dialogue satisfaction , a.k.a . Service Satisfaction Analysis ( SSA ) at dialogue-level ( Song et al . , 2019 ) , and to predict potential chatbot failure via machine-human chatting handoff ( MHCH ) at utterance-level ( Huang et al . , 2018 ; Liu et al . , 2021 ) . In a MHCH context , algorithm can transfer an ongoing auto-dialogue to the human agent when the current utterance is confusing . Figure 1 depicts an exemplar dialogue of online customer service . In this dialogue , the chatbot gives an unsatisfied answer about shipping , thus causing the customer 's complaint ( local dissatisfaction utter 2 and utter 3 ) . Ideally , chatbot should be able to detect the negative ( local ) emotion ( utter 3 ) and tries to appease complaints , but this problem remains unresolved . If chatbot continues , the customer may cancel the deal and give a negative rating ( dialogue global dissatisfaction ) . With MHCH ( detects the risks of utter 2 and utter 3 ) , the dialogue can be transferred to the human agent , who is better at handling , compensating , and comforting the customer and enhance customer satisfaction . This example illustrates the cross-impact between handoff and dialogue ( local+global ) satisfaction . Intuitively , MHCH and SSA tasks can be compatible and complementary given a dialogue discourse , i.e. , the local satisfaction is related to the quality of the conversation ( Bodigutla et al . , 2019a ( Bodigutla et al . , , 2020 ) ) , which can support the handoff judgment and ultimately affect the overall satisfaction . On the one hand , handoff labels of utterances are highly pertinent to local satisfaction , e.g. , one can utilize single handoff information to enhance local satisfaction prediction , which ultimately contributes to the overall satisfaction estimation . On the other hand , the overall satisfaction is obtained by combining local satisfactions , which reflects the quality in terms of answer generation , language understanding , and emotion perception , and subsequently helps to facilitate handoff judgment . In recent years , researchers ( Bodigutla et al . , 2019a , b ; Ultes , 2019 ; Bodigutla et al . , 2020 ) explore joint evaluation of turn and dialogue level qualities in spoken dialogue systems . In terms of general dialogue system , to improve the efficiency of dialogue management , Qin et al . ( 2020 ) propose a co-interactive relation layer to explicitly examine the cross-impact and model the interaction between sentiment classification and dialog act recognition , which are relevant tasks at the same level ( utterancelevel ) . However , MHCH ( utterance-level ) and SSA ( dialogue-level ) target satisfaction at different levels . More importantly , handoff labels of utterances are more comprehensive and pertinent to local satisfaction than sentiment polarities . Meanwhile , customer utterances have significant impacts on the overall satisfaction ( Song et al . , 2019 ) , which motivates us that the role information can be critical for knowledge transfer of these two tasks . To address the aforementioned issues , we propose an innovative Role-Selected Sharing Network ( RSSN ) for handoff prediction and dialogue satisfaction estimation , which utilizes role information to selectively characterize complex relations and interactions between two tasks . To the best of our knowledge , it is the pioneer investigation to leverage the multi-task learning approach for integrating MHCH and SSA . In practice , we first adopt a shared encoder to obtain the shared representations of utterances . Inspired by the co-attention mechanism ( Xiong et al . , 2016 ; Qin et al . , 2020 ) , the shared representations are then fed into the roleselected sharing module , which consists of two directional interactions : MHCH to SSA and SSA to MHCH . This module is used to get the fusion of MHCH and SSA representations . We propose the role-selected sharing module based on the hypothesis that the role information can benefit the tasks ' performances . The satisfaction distributions of utterances from different roles ( agent and customer ) are different , and the effects for the tasks are also different . Specifically , the satisfaction of agent is non-negative . The utterances from agent can enrich the context of customer 's utterances and indirectly affect satisfaction polarity . Thus , directly employing local satisfaction of agent into the interaction with handoff may introduce noise . In the proposed role-selected sharing module , we adopt local satisfaction based on the role information : only the local satisfaction from customer can be adopted to interact with handoff information . By this means , we can control knowledge transfer for both tasks and make our framework more explainable . The final integrated outputs are then fed to separate decoders for handoff and satisfaction predictions . To summarize , our contributions are mainly as follows : ( 1 ) We introduce a novel multi-task learning framework for combining machine-human chatting handoff and service satisfaction analysis . ( 2 ) We propose a Role-Selected Sharing Network for handoff prediction and satisfaction rating estimation , which can utilize different role information to control knowledge transfer for both tasks and enhance model performance and explainability . ( 3 ) The experimental results demonstrate that our model outperforms a series of baselines that consists of the state-of-the-art ( SOTA ) models on each task and multi-task learning models for both tasks . To assist other scholars in reproducing the experiment outcomes , we release the codes and the annotated dataset1 . In this paper , we propose an innovative multi-task framework for service satisfaction analysis and machine-human chatting handoff , which deliberately establishes the mutual interrelation for each other . Specifically , we propose a Role-Selected Sharing Network for joint handoff prediction and satisfaction estimation , utilizing role and positional information to control knowledge transfer for both tasks . Extensive experiments and analyses reveal that explicitly modeling the interrelation between the two tasks can boost the performance mutually . However , our model has not been calibrated to account for user preferences and biases , which we plan to address in future work . Moreover , we will further explore how to adjust the handoff priority with the assistance of personalized information .

Challenge: For chatbots to achieve satisfactory user experience, a system needs to model user satisfaction well but existing systems model local uttrance-level and global dialogue-level disjointly. 
Approach: They propose a Role-Selected Sharing Network which models local dialogue satisfaction estimation and global handoff prediction in one multi-task learning framework.
Outcome: Experiments with two datasets show the proposed model outperforms existing models for both tasks modeled in multi-task learning showing that interrelation helps performance improvements.