Intelligent method to identify complexity of work artifacts

An automated method for determining a complexity of a task. The method includes extracting data from the plurality of historical support tickets to generate training data. The method trains a complexity model to predict a complexity value of a task associated with a support ticket using the training data. The method predicts, using the complexity model, the complexity value of a new task associated with a new support ticket.

BACKGROUND

Support service is one aspect of business that is essential to every industry. Regardless of industry, it is crucial to deliver the best service possible to all customers.

SUMMARY

The disclosed embodiments include an automated method, system, and computer program product for determining a complexity of a task. In an embodiment, the method, system, and computer program product are configured to extract data from a plurality of historical support tickets to generate training data; train a complexity model to predict a complexity value of a task associated with a support ticket using the training data; receive a new support ticket that requests assistance with performance of a new task; and predict, using the complexity model, the complexity value of the new task associated with the new support ticket.

In various embodiments, the process of extracting data from the plurality of historical support tickets to generate training data includes extracting one or more text characters from text fields of the plurality of historical support tickets; generating one or more features of the plurality of historical support tickets by vectorizing the extracted text characters; extracting one or more values from one or more value fields of the plurality of historical support tickets; and generating one or more labels of the plurality of historical support tickets using the one or more extracted values from the one or more value fields of the plurality of historical support tickets.

In various embodiments, the process of generating one or more labels of the plurality of historical support tickets using the one or more extracted values from the one or more value fields of the plurality of historical support tickets includes normalizing each of the extracted values from the one or more value fields of the plurality of historical support tickets so that the one or more extracted values have a maximum value of one; setting a weight value for each of the extracted values; and calculating the one or more labels of the plurality of historical support tickets using the weight value for each of the extracted values.

In various embodiments, the process of calculating the one or more labels of the plurality of historical support tickets using the weight value for each of the extracted values uses formula: L=W1*l1+Σi=2nWili, where L is a label, W is the weight value, l is extracted value/max value, and n is a number of extracted values.

In various embodiments, the process of predicting, using the complexity model, the complexity value of the new task associated with the new support ticket includes determining an initial complexity value of the new task associated with the new support ticket using the complexity model; determining a set of similar support tickets from the plurality of historical support tickets that are similar to the new support ticket; and adjusting the initial complexity value of the new task associated with the new support ticket based on one or more similarity and complexity values of one or more similar support tickets in the set of similar support tickets to determine the complexity value of the new task associated with the new support ticket.

In various embodiments, a weight to the one or more similarity and complexity values of tickets in the set of similar tickets.

In various embodiments, the weight is based on a reporting time of the similar support tickets in the set of similar support tickets.

Other embodiments and advantages of the disclosed embodiments are further described in the detailed description.

DETAILED DESCRIPTION

A customer support engineer usually handles a support ticket per its priority which is mainly evaluated by the time the case has been open, severity level, etc. The priority may be a value (e.g., 1-10) or a status (e.g., normal, rush, etc.) that is printed on a ticket. The priority may be based on how critical a task is, the person requesting the task, or other factors. However, even if two support tickets are assigned similar priorities, there may be reasons for addressing a particular support ticket before another with the same or higher priority. For example, even if two support tickets have the same priority, one support ticket may involve a common or known issue and can be solved very quickly, while the other support ticket may involve a new or complex issue that can take longer to solve. In this case, the ticket with the commonly known issue should be handled first because the problem can be resolved quickly, and thus reducing any impact caused by the issue to the customer. This can increase customer satisfaction by reducing overall response time. Thus, it would be advantageous to be able to automatically identify the complexity of a task related to a support ticket to increase productivity and customer satisfaction.

The present disclosure describes various embodiments that include an automated method, system, and computer program product for determining a complexity of a task. In an embodiment, the system is configured to extract data from a plurality of historical support tickets to generate training data; train a complexity model to predict a complexity value of a task associated with a support ticket using the training data; receive a new support ticket that requests assistance with performance of a new task; and predict, using the complexity model, the complexity value of the new task associated with the new support ticket.

As used within the written disclosure and in the claims, the terms “including” and “comprising” are used in an open-ended fashion, and thus should be interpreted to mean “including, but not limited to.” Unless otherwise indicated, as used throughout this document, “or” does not require mutual exclusivity, and the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

A module or unit as referenced herein may comprise one or more hardware or electrical components such as electrical circuitry, processors, and memory that may be specially configured to perform a particular function. The memory may be volatile memory or non-volatile memory that stores data such as, but not limited to, computer executable instructions, machine code, and other various forms of data. The module or unit may be configured to use the data to execute one or more instructions to perform one or more tasks. In certain instances, a module may also refer to a particular set of functions or computer executable instructions configured to perform a specific task. For example, a module may comprise of software components such as, but not limited to, data access objects, service components, user interface components, application programming interface (API) components; hardware components such as electrical circuitry, processors, and memory; and/or a combination thereof. As referenced herein, computer executable instructions may be in any form including, but not limited to, machine code, assembly code, and high-level programming code written in any programming language.

FIG. 1is a schematic drawing illustrating a process for determining a complexity of a task in accordance with an embodiment of the present disclosure. In the depicted embodiment, a customer102, using a computer or any other suitable electronic device, requests support or assistance by providing information to a customer support system100. The customer support system100uses the information provided by the customer102to generate a new support request ticket104. The ticket104may request support to perform any kind of tasks such as, but not limited to, request support to fix an issue and/or request support to perform a particular task (e.g., cloud application deployment). In accordance with a disclosed embodiment, the ticket104is passed to a complexity module106and a validation module108. The complexity module106, as further described inFIG. 2, is configured to use a complexity model to determine an initial complexity value for the task associated with the ticket104. The validation module108, as further described inFIG. 3, is configured to identify and use similar past ticket requests to correct any deviation in the initial complexity value. The initial complexity value of the new ticket104and the complexity values of the top N similar ticket requests are inputted into a merge module110. N, as used herein, can be any predetermined number or can be a user-specified number. For example, in an embodiment, the top 10 similar ticket requests can be used to correct any deviation in the initial complexity value. In an embodiment, the merge module110, as further described inFIG. 4, is configured to combine the initial complexity value generated by the complexity module106with the average weighted complexity values for the top N similar ticket requests produced by the validation module108to determine a more accurate final complexity value for the task associated with the ticket104. The final complexity value for the task associated with the ticket104can be printed on the ticket104or indicated in some other manner to enable a support engineer112to properly prioritize the task associated with the ticket104with other pending tickets.

FIG. 2is a schematic drawing illustrating a complexity module200in accordance with an embodiment of the present disclosure. The complexity module200is an example of the complexity module106inFIG. 1. In an embodiment, the complexity module200is configured to determine an initial complexity value for the ticket104using a complexity model202. In an embodiment, the complexity model202is built and trained using a machine learning algorithm based on training data from a plurality of historical support tickets204. In an embodiment, the complexity model202is a regression model and is trained using a linear regression learning algorithm for enabling the complexity model202to predict a numeric complexity value of the new ticket104. The plurality of historical support tickets204contain data attributes such as, but not limited to, feature information, resolution or label information, and complexity values for each of the historical support tickets204. The complexity values for each of the historical support tickets204may have been manually determined based on the experiences of one or more support engineers112that dealt with the tasks related to each of the historical support tickets204.

In an embodiment, the plurality of historical support tickets204are passed to a feature generation module206and a label generation module208for extracting the data attributes from each of the historical support tickets204. For example, in an embodiment, the feature generation module206is configured to parse and extract textual features (F) of the plurality of historical support tickets204such as, but not limited to, text characters from a subject, description, version, problem occurred time, or other fields of the plurality of historical support tickets204. In an embodiment, the textual data is converted into vectors so that the features of a historical support ticket204can be compared with the features of other historical support tickets204. In an embodiment, the text vectorization process maps words or phrases from vocabulary to a corresponding vector of real numbers. After the words are converted as vectors, techniques such as Euclidean distance or cosine similarity can be used to identify similar words and/or the semantics/meaning of words or phrases.

In an embodiment, the label generation module208is configured to parse and extract label (L) values, such as, but not limited to, values corresponding to case working hours, time to close, time to resolution, or other fields of the plurality of historical support tickets204. In an embodiment, the label generation module208normalizes each of the extracted label values (l) based on the maximum value of each label (l=value/max value) so that the value of l is between 0 and 1. In an embodiment, a weight value (W) is assigned to each label. The weight value may be based on the experience of support engineers or may be customer specified to indicate the importance of a particular label. For example, the label “time to resolution” may be a more important factor than the label “working hours” and assigned a higher weight. As a non-limiting example, below is chart illustrating the label values and weight for one historical support ticket204.

In an embodiment, a final label value for each label for all the plurality of historical support tickets204can be calculated using the equation: L=W1*l1+Σi=2nWili, where L is a label, W is the weight value, l is extracted value/max value, and n is a number of extracted values/labels.

Once all the feature and label generation are determined in the plurality of historical support tickets204, this information is used as training data to train the complexity model202. In an embodiment, a learning algorithm finds patterns in the training data that map the input feature and label data attributes to the input complexity values of the plurality of historical support tickets204to train the complexity model202. In an embodiment, the complexity model202can then be used to predict an initial complexity value based on the data attributes of a new ticket104as shown inFIG. 2.

FIG. 3is a schematic drawing illustrating a validation module300in accordance with an embodiment of the present disclosure. The validation module300is an example of the validation module108inFIG. 1. As described inFIG. 1, the validation module300is configured to identify and use similar ticket requests to correct any deviation in the initial complexity value of a new ticket104. In the depicted embodiment, the validation module300includes a similarity search module302and an order module310. In an embodiment, the similarity search module302is configured to identify similar tickets to the new ticket104, and the order module310is configured to order the top N tickets by their similarity.

In an embodiment, the similarity search module302includes an embedding module304and a coefficient module306. The new ticket104is initially passed to the embedding module304of the similarity search module302. The new ticket104may include text information such as, but not limited to, a subject, a description, product version, operating system, or any other information that may be associated with a support ticket. The embedding module304is configured to perform similar functions as the feature generation module206inFIG. 2, except that the embedding module304is configured to parse and extract textual features of the new ticket104, as opposed to a historical support ticket204as performed by the feature generation module206inFIG. 2. In an embodiment, the embedding module304converts the textual data that is extracted from each of the fields into vectors. For example, in an embodiment, the embedding module304converts the textual data that is extracted from each of the fields into vectors using Word2Vec. Word2vec is a two-layer neural net that takes a text corpus as input and outputs a set of feature vectors that represent words in that corpus. In an embodiment, the coefficient module306is configured to determine a coefficient value for the new ticket104based on the vectorized extracted data from the new ticket104. The coefficient module306can also determine a coefficient value for each of the historical support tickets204based on the vectorized extracted data from the plurality of historical support tickets204. The coefficient value for the new ticket104can then be compared with the coefficient values of other historical support tickets204to identify similar tickets to the new ticket104. In an embodiment, the coefficient value for a ticket is determined using the Pearson correlation coefficient formula. The Pearson correlation coefficient is a statistical measure of the linear correlation between two variables. The coefficient has a value between +1 and −1. A value of +1 is total positive linear correlation, 0 is no linear correlation, and −1 is total negative linear correlation. The formula for the Pearson correlation coefficient is:

Where

xi=values of the x-variable in a sample

x=mean of the values of the x-variable

yi=values of the y-variable in a sample

y=mean of the values of the y-variable

A non-limiting example of a data table comprising the information of similar tickets to that of the new ticket104created by the similarity search module302is shown below:

Once the similarity search module302identifies similar tickets to the new ticket104, it passes the similar tickets to the order module310. In an embodiment, the order module310includes an order top N module312and a final top N module314. The order top N module312is configured to order the top N tickets received from the similarity search module302based on their similarity. In an embodiment, tickets with the same or similar coefficient values appear in the Top N list. N can be any predetermined number or a user-specified number. The ordered list of top N tickets is then passed to the final top N module314.

In an embodiment, the final top N module314is configured to further order the list of top N tickets based on a second dimension. For example, when two of the historical support tickets204in the ordered list of top N tickets have the same or similar similarity (i.e., coefficient values) to that of the new ticket104, the final top N module314references the complexity score to perform a final ranking/sorting of top N tickets. In an embodiment, the final top N module314utilizes the complexity score of the Top N work ticket that was used for training the complexity model. In an embodiment, the final top N module314is configured to select work tickets (ticket (R)) from a reference pool of tickets318that have a large difference between their complexity score and the initial complexity score of the new ticket104as determined by the complexity module106. In an embodiment, the “large difference” is a differentiation of the label, which is equal to the subtraction of the new ticket complexity score and the historical support ticket complexity score. Ticket (R) represents adding a reference information to ensure that the final selected sample is differentiated. The final top N module314then performs a final sorting to obtain the final top N tickets and outputs the top N tickets sorted by similarity (Si) and complexity (Ci).

FIG. 4is a schematic drawing illustrating a merge module400in accordance with an embodiment of the present disclosure. The merge module400is an example of the merge module110inFIG. 1. In the depicted embodiment, the merge module400includes a weight module402and a merge unit404. As described inFIG. 1, the merge module400receives as input an initial complexity value (C0) determined by the complexity module106, and the top N tickets (Ticketi(Si, Ci)) sorted by similarity (Si) and complexity (Ci) from the validation module108. In the depicted embodiment, the weight module402is configured to add multi-dimensional considerations to the top N tickets, such as time, user experience and other dimensions, so as to add weight to the similar top N tickets. For instance, in an embodiment, the weight module402can apply a time dimension weight to the top N tickets as follows. As an example, assume that the top 3 tickets similar tickets (T1, T2, and T3) to the new ticket have respective complexity values (C1, C2, C3). T1was reported one year ago. T2was reported one month ago. T3was reported one week. According to the reporting times, different time weight values can be applied to the complexity values (C1, C2, C3) of T1, T2, and T3. As an example, a time weight value of 0.1 can be assigned to a reporting time of one year, a time weight value of 0.5 can be assigned to a reporting time of one month, and a time weight value of 0.9 can be assigned to a reporting time of one week. These weight values assign a higher weight to similar tickets that occurred more recently. In an embodiment, the weight complexity values (Cn) of the similar tickets is calculated according to the following formula: Cn=Ci*Wn/Σi=1nWn, where Ciis the complexity value of the top N ticket and Wnis the weight value assigned to the ticket.

The weight module402passes the weighted complexity values (Cn) of the similar top N tickets to the merge unit404. The merge unit404also receives the initial complexity value (C0) determined by the complexity module106. In an embodiment, the merge unit404is configured to combine the initial complexity value (C0) generated by the complexity module106with the average weighted complexity values for the top N similar ticket (Cn) produced by the validation module108to determine a more accurate final complexity value (Cf) for the task associated with the ticket104. In an embodiment, the merge unit404determines the final complexity value (Cf) of the new ticket as follows: Cf=C0*k+Σi=1nCn*(1−k), where C0is the initial complexity value, Σi=1nCnis the average of the weighted top N similar tickets, and k is an adjustment value that can be modified according to the maturity of the complexity model. The merge module400then outputs the final complexity value (Cf) as the predicted complexity of the task associated with the new ticket104.

FIG. 5is a flowchart illustrating a process500for predicting a complexity of a task associated with a support ticket in accordance with an embodiment of the present disclosure. In an embodiment, the process500can be performed by the customer support system100inFIG. 1. The process500begins at step502by extracting data from the plurality of historical support tickets to generate training data. In an embodiment, the process500extracts text characters from one or more text fields of the plurality of historical support tickets, and extracts values from one or more value fields of the plurality of historical support tickets. The process500generates one or more features of the plurality of historical support tickets by vectorizing the extracted text characters, and generates one or more labels of the plurality of historical support tickets using the extracted values from the one or more value fields of the plurality of historical support tickets. In an embodiment, the process500normalizes each of the extracted values from the one or more value fields of the plurality of historical support tickets so that the extracted values have a maximum value of one. The process500can also set a weight value for each of the extracted values. In an embodiment, the process500calculates the labels of the plurality of historical support tickets using the weight value for each of the extracted values.

At step504, the process500trains a complexity model to predict a complexity value of a task associated with a support ticket using the training data. In an embodiment, the process500trains the complexity model using a linear regression learning algorithm. The process500, at step506, receives a new support ticket that requests assistance with performance of a new task.

The process500, at step508, predicts, using the complexity model, the complexity value of the new task associated with the new support ticket. In an embodiment, to predict the complexity value of the new task, the process500determines an initial complexity value of the new task associated with the new support ticket using the complexity model. The process500then determines a set (one or more) of similar support tickets from the plurality of historical support tickets that are similar to the new support ticket. The process500adjusts the initial complexity value of the new task associated with the new support ticket based on similarity and complexity values of tickets in the set of similar support tickets to determine the complexity value of the new task associated with the new support ticket. In an embodiment, the process500applies a weight to the similarity and complexity values of the similar support tickets in the set of similar tickets. For example, the process500can apply a weight based on a reporting time of the similar support tickets in the set of similar tickets. The predicted complexity value of the new task associated with the new support ticket is provides additional information for a support engineer in determining how best to handle support ticket requests.

FIG. 6is a block diagram illustrating a hardware architecture of a system600according to an embodiment of the present disclosure in which aspects of the illustrative embodiments may be implemented. In an embodiment, the customer support system100inFIG. 1is implemented using the hardware architecture of the system600. Additionally, the data processing system600may be configured to store and execute instructions for implementing the complexity module200inFIG. 2, the validation module300inFIG. 3, the merge module400inFIG. 4, and the process500inFIG. 5. In the depicted example, the data processing system600employs a hub architecture including north bridge and memory controller hub (NB/MCH)606and south bridge and input/output (I/O) controller hub (SB/ICH)610. Processor(s)602, main memory604, and graphics processor608are connected to NB/MCH606. Graphics processor608may be connected to NB/MCH606through an accelerated graphics port (AGP). A computer bus, such as bus632or bus634, may be implemented using any type of communication fabric or architecture that provides for a transfer of data between different components or devices attached to the fabric or architecture.

In the depicted example, network adapter616connects to SB/ICH610. Audio adapter630, keyboard and mouse adapter622, modem624, read-only memory (ROM)626, hard disk drive (HDD)612, compact disk read-only memory (CD-ROM) drive614, universal serial bus (USB) ports and other communication ports618, and peripheral component interconnect/peripheral component interconnect express (PCI/PCIe) devices620connect to SB/ICH610through bus632and bus634. PCI/PCIe devices may include, for example, Ethernet adapters, add-in cards, and personal computing (PC) cards for notebook computers. PCI uses a card bus controller, while PCIe does not. ROM626may be, for example, a flash basic input/output system (BIOS). Modem624or network adapter616may be used to transmit and receive data over a network.

HDD612and CD-ROM drive614connect to SB/ICH610through bus634. HDD612and CD-ROM drive614may use, for example, an integrated drive electronics (IDE) or serial advanced technology attachment (SATA) interface. In some embodiments, HDD612may be replaced by other forms of data storage devices including, but not limited to, solid-state drives (SSDs). A super I/O (SIO) device628may be connected to SB/ICH610. SIO device628may be a chip on the motherboard configured to assist in performing less demanding controller functions for the SB/ICH610such as controlling a printer port, controlling a fan, and/or controlling the small light emitting diodes (LEDS) of the data processing system600.

The data processing system600may include a single processor602or may include a plurality of processors602. Additionally, processor(s)602may have multiple cores. For example, in one embodiment, data processing system600may employ a large number of processors602that include hundreds or thousands of processor cores. In some embodiments, the processors602may be configured to perform a set of coordinated computations in parallel.

An operating system is executed on the data processing system600using the processor(s)602. The operating system coordinates and provides control of various components within the data processing system600inFIG. 6. Various applications and services may run in conjunction with the operating system. Instructions for the operating system, applications, and other data are located on storage devices, such as one or more HDD612, and may be loaded into main memory604for execution by processor(s)602. In some embodiments, additional instructions or data may be stored on one or more external devices. The processes described herein for the illustrative embodiments may be performed by processor(s)602using computer usable program code, which may be located in a memory such as, for example, main memory604, ROM626, or in one or more peripheral devices.