S upport N et: Neural Networks for Summary Generation and Key Segment Extraction from Technical Support Tickets (2021)

Abstract: We improve customer experience and gain their trust when their issues are resolved rapidly with less friction. Existing work has focused on reducing the overall case resolution time by binning a case into predefined categories and routing it to the desired support engineer. However, the actions taken by the engineer during case analysis and resolution are altogether ignored, even though it forms the bulk of the case resolution time. In this work, we propose two systems that enable support engineers to resolve cases faster. The first, a guidance extraction model, mines historical cases and provides technical guidance phrases to the support engineers. The phrases can then be used to educate the customer or to obtain critical information needed to resolve the case and thus minimize the number of correspondences between the engineer and customer. The second, a summarization model, creates an abstractive summary of the case to provide better context to the support engineer. Through quantitative evaluation we obtain an F1 score of 0.64 on the guidance extraction model and a BertScore (F1) of 0.55 on the summarization model.

## introduction it is of paramount importance to aws support organization to reduce the resolution time of customer cases to ensure their business runs seamlessly without any downtime. we have a unique challenge in that the customers' issues can be deeply technical and require technically skilled agents to resolve it. there is a rapid increase in the number of users of the services offered by our cloud company and it is important to improve tooling for support engineers (ses) in order to scale. a significant portion of customers' cases are business critical and time-sensitive. cases are created by customers for several reasons such as guidance about a specific service or troubleshooting when a production service is down. a typical customer creates a case with a title of the case (case title) and a detailed correspondence on their issue as a part of the case @xcite . the agents then have to read this case, understand the customer's problem and suggest ways for them to resolve the issues. this requires the agents to spend a lot of time to completely read the case and then guide the customer to solve the problem. we have developed two systems that can use the inputs from the customer in the form of case text (case title + communication text) to speed up the agent's time to resolve a case. the first system is a summarization system that presents the customers problem to the agent to give them a head start in tackling the case. the second system provides them snippets from similar historical cases to reduce the time the agent takes to respond to the customer. prior work in the domain of customer support has focused on improving the time to resolve a case by improving routing and detecting the customer problems into one of several predefined categories @xcite @xcite . while these methods reduce average total time to resolve issues throughout the case journey, they do not focus on reducing the active handle time by ses, i.e., the time a se has to invest to understand and respond to the customer. in our work, we address this gap by introducing two novel solutions as previously described. ses often look to similar previously resolved cases when beginning to tackle a new case according to a few internal studies. a previous similar case provides them troubleshooting resources, hints on root causes, and guidance material that they reuse on the new case. these resources from previous similar cases have been found to reduce the handle time by ses, but it takes time for agent to browse through the results. hence, we built a knowledge mining system based on nlp that allows ses to efficiently look up historical cases without having to read the whole case. typical technical cases contain many conversations, and reading through them is a timeconsuming process. in order to solve this problem, a system was introduced -state of the case (soc) -where ses update a summary of the current state of a case. these updates are made manually by ses while they hand the case back to the customer or to another internal team. in addition to serving as a smooth transition between ses throughout the case lifecycle, the soc of a case was intended to serve as its expert summary view, eliminating the vagueness and jargon that may be present in customers' case text. however, the additional manual effort to fill soc resulted in a low adoption rate of 9.8% over time. we introduce the solution in section 3.2 automating soc generation based on customer communications. ## related work as mentioned earlier, @xcite built systems that could categorize case issues into predefined categories. they also suggest predefined templates to se. specific details of the case are not taken into picture while suggesting these templates to make them more appropriate. our summary model on the other hand ingests the context of the case and generates a personalized problem template that a se can use to update the state of case. prior work @xcite @xcite has looked at providing better information technology service management (itsm) to their customers by building resources that help the customers diagnose their own issue and find a solution for it. however, customers usually try to self-diagnose their issues before cutting a case. hence, our solution focuses on helping the se and improve their efficiency. other work @xcite @xcite has looked at the use of the support case text along with other metadata to classify the intent of the case and improve routing. we on the other hand analyze the case text to provide assistance to se in their day-to-day tasks. ## system and model overview figure 1 shows the overall architecture of the proposed system. there are two major components that we propose in this work. the first one uses a bert and gpt-2 model to allow the se to get a head start in solving the case. the second component runs on the case text of previously solved cases to provide the se with guidance phrases. they can use the predicted guidance phrases to understand how the case can be solved and to also construct a response back to the customer. in this section, we provide details of the models that were built in these components and the steps that we took to train them. ## 5000 241 ## application of the models to support businesses extraction of guidance phrases from case text of previous cases can provide quick tips to ses and make it easier for them while working on a case. every se is required to write a reply to the customer with the steps they can take to resolve the issue. these guidance phrases will help them reuse the text enabling them to spend less time on the reply email. the summary models we have built can be used in a variety of se tasks. we can use the automated summary to provide the context of the ## conclusion it is important for us to scale the support business with our rapid user growth. we describe two components of a system that aims to reduce the time spent by se in resolving a support case. the aim of this work is to promote research at the intersection of nlp and support business. using our models, we were able to achieve an f1 score of 0.64 on the guidance extraction problem and bertscore (f1) of 0.55 on the summarization problem. these promising results shows us that they can be deployed in production, create impact and help ses in their day-to-day tasks. we hope this contribution can lead to better tools that can improve the tooling necessary for support agents to provide a rich customer experience.