Patent Publication Number: US-10771627-B2

Title: Personalized support routing based on paralinguistic information

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a continuation of pending U.S. patent application Ser. No. 16/564,462, filed on Sep. 9, 2019, which is herein incorporated by reference in its entirety. U.S. patent application Ser. No. 16/564,462, filed on Sep. 9, 2019, is a continuation of U.S. patent application Ser. No. 16/177,510, filed on Nov. 1, 2018, and issued on Sep. 10, 2019, as U.S. Pat. No. 10,412,223, which is herein incorporated by reference in its entirety. U.S. patent application Ser. No. 16/177,510, filed on Nov. 1, 2018, and issued on Sep. 10, 2019, as U.S. Pat. No. 10,412,223 is a continuation of U.S. patent application Ser. No. 15/336,521, filed on Oct. 27, 2016, and issued on Nov. 20, 2018, as U.S. Pat. No. 10,135,989, which is herein incorporated by reference in its entirety. 
    
    
     BACKGROUND 
     Field 
     Embodiments of the present invention generally relate to systems for providing support to application users. More specifically, embodiments presented herein provide techniques for routing users to support agents based on paralinguistic features of audio input received from the users. 
     Description of the Related Art 
     Many modern software applications are designed to be as user-friendly as possible. However, users who are unfamiliar with an application interface or functionality often need some assistance to navigate through the application, perform desired tasks, or find answers to general questions. If a user fails to find assistance quickly enough, the user may abandon using the application. In this way, a software vendor can lose current or potential customers by failing to provide adequate support. 
     To provide support, some software vendors employ support agents who are trained to assist customers with common problems. These support agents may communicate with customers via messaging windows or via telephone. Experience levels, language skills, personality types, voice qualities, call-time efficiency, and other attributes vary from one support agent to another. Technical sophistication, personality, and patience also vary from one customer to another. The support issues faced by such agents can also vary widely from encounter to encounter. This variation can lead to very different experiences for customers who contact support agents. For example, a new agent may be able to help a patient customer who speaks clearly with a relatively simple issue, but may struggle to help an irate customer with limited communication skills resolve a complicated or unusual issue. A more experienced agent, however, may be able to calm the irate customer down and resolve the issue relatively quickly, thereby making it more likely that the customer will continue to use the application. 
     Support agents typically work in one or more call centers. Typically, a computing system routes calls to support agents based on agent availability, agent skills, and the phone line through which the calls arrive. For example, if a customer dials a telephone number for billing support in French, the computing system searches through stored agent profiles to identify an available support agent who speaks French and has been trained to provide billing support. The computing system routes the call to the available support agent. If no support agents with matching skills are available, the computing system places the call into a queue. The customer waits on hold in the queue until the call is answered by the next available qualified support agent. 
     SUMMARY 
     One embodiment disclosed herein includes a method for using paralinguistic features to determine whether to present customer assistance information to a user of an application. The method generally includes receiving audio input from the user interacting with the application; extracting a set of paralinguistic features from the audio input, wherein the paralinguistic features characterize acoustic aspects of the audio input distinct from verbal content; predicting, from the paralinguistic features, an emotional state of the user using a predictive model; and upon determining the emotional state satisfies one or more specified conditions, presenting a prompt inviting the user to communicate with a support agent. 
     Another embodiment includes non-transitory computer-readable storage medium containing instructions that, when executed by one or more processors, perform an operation for using paralinguistic features to determine whether to present customer assistance information to a user of an application. The operation generally includes receiving audio input from the user interacting with the application; extracting a set of paralinguistic features from the audio input, wherein the paralinguistic features characterize acoustic aspects of the audio input distinct from verbal content; predicting, from the paralinguistic features, an emotional state of the user using a predictive model; and upon determining the emotional state satisfies one or more specified conditions, presenting a prompt inviting the user to communicate with a support agent. 
     Still another embodiment includes a processor and a memory storing one or more applications that, when executed on the processor, performs an operation for using paralinguistic features to determine whether to present customer assistance information to a user of an application. The operation generally includes receiving audio input from the user interacting with the application; extracting a set of paralinguistic features from the audio input, wherein the paralinguistic features characterize acoustic aspects of the audio input distinct from verbal content; predicting, from the paralinguistic features, an emotional state of the user using a predictive model; and upon determining the emotional state satisfies one or more specified conditions, presenting a prompt inviting the user to communicate with a support agent. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       So that the manner in which the above recited features of the present disclosure can be understood in detail, a more particular description of the disclosure, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only exemplary embodiments and are therefore not to be considered limiting of its scope, may admit to other equally effective embodiments. 
         FIG. 1  illustrates a computing environment in which systems for routing calls based on paralinguistic features may operate, according to one embodiment. 
         FIG. 2  illustrates a detailed view of a mobile device that can be used with systems of the present disclosure, according to one embodiment. 
         FIG. 3  illustrates a method for providing support for a financial application, according to one embodiment. 
         FIG. 4  illustrates a method for implicitly determining how satisfied customers are when assisted by a support agent, according to one embodiment. 
         FIG. 5  illustrates a method for determining an attribute measuring how well a support agent improves customer emotional states, according to one embodiment. 
         FIG. 6  illustrates a method for training a predictive model, according to one embodiment. 
         FIG. 7  illustrates a targeted marketing system that provides targeted advertisements, according to one embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     Embodiments presented herein provide techniques for selecting an agent to whom a user is routed for assistance based on paralinguistic features of audio input received from the user. For example, in one embodiment, a user utters commands processed by a software application. A feature extractor identifies paralinguistic features characterizing the audio input. A predictive model (e.g., a machine-learning model) uses the paralinguistic features to determine a score representing a predicted emotional state of the user. A support routing module selects a support agent to assist the user based on the score and based on a profile of the support agent. 
     For example, the support routing module can compare a profile of the support agent to a threshold level for the attribute that is associated with the score in a mapping and select the support agent based on the comparison. The attribute may be an experience level of the agent, an average customer-satisfaction level for customers the agent assists, or an average score change (e.g., indicating change in emotional state) of customers the agent assists. If a profile for the user is available, the support routing module may also compare the user profile to the agent profile and base the selection on the comparison. 
     Note embodiments described herein using a software application used to manage financial information or accounts as a reference example. Of course one of skill in the art will readily recognize embodiments may be adapted for other types of software applications. 
       FIG. 1  illustrates a computing environment  100  in which systems for routing calls based on paralinguistic features may operate, according to one embodiment. A server  104 , a mobile device  116 , and a support center  122  are connected to a network  102 . Through an application  118  on the mobile device  116 , a user interacts with a financial software application  114  executing on the server  104 . In some embodiments, application  118  is a client interacting with the financial software application  114  on the server  104 . The financial software application  114  may provide tax preparation, personal financial management, or other types of financial services. The application  118  may be a browser or a standalone client used to access services provided by the financial software application  114 . 
     The user communicates with the application  118  using input/output (I/O) components  120 . I/O components  120  include a microphone, a speaker, and a touch screen. The microphone of the I/O components  120  can capture audio input for the application  118 . For example, application  118  may support spoken commands, requests, or other verbal statements to the application  118 . The application  118  sends the audio input captured by the microphone to the server  104 . 
     The feature extractor  115  extracts paralinguistic features from the audio input. The predictive model  110  (e.g., machine-learning model) receives the paralinguistic features from the feature extractor  115  and determines a score for the user. The score can represent an emotional state of the user as inferred by the predictive model  110  based on the paralinguistic features. For example, the score may be a real number ranging from −1 (a very negative emotional state) to 1 (very positive emotional state). 
     Paralinguistic technically means “alongside linguistics” and therefore deals with phenomena that are modulated onto or embedded into the audio input and are distinct from verbal content (i.e., the actual words that are spoken). For example, some of the paralinguistic features may be low level descriptors that are extracted from frames of the audio stream (e.g., 10-30 millisecond frames). These acoustic low level descriptors may include intonation (pitch, etc.), intensity (energy, Taeger functions, etc.), linear prediction cepstral coefficients (LPCC), perpetual linear prediction (PLP) parameters, cepstral coefficients (Mel frequency cepstral coefficients (MFCCs), etc.), formants (amplitude, position, width, dispersion, etc.), spectrum (Mel frequency bands (MFB), NMF-based components, MPEG-7 audio spectrum projection, roll-off, etc.), TF transformation (wavelets, Gabor, etc.), harmonicity (harmonics-to-noise ratio (HNR), noise-to-harmonics ratio (NHR), etc.), and/or perturbation (jitter, shimmer, etc.). Further, the paralinguistic features may include low level descriptors such as phoneme sequences, non-linguistic strings (laughter, sighs, etc.), and/or disfluencies (false starts, filled/unfilled pauses, etc.). The paralinguistic features may also include fundamental frequency, volume, and other measurable characteristics of the audio input. The paralinguistic information may be extracted utilizing software such as EmoVoice, OpenEar, Yaafe, Voice Search DB with Accent and Tone, VoiceVault, AT&amp;T APis, etc. 
     The support routing module  112  compares the score to a threshold score. The threshold score may be a predetermined global value that applies to all users. Alternatively, the threshold score may be a user-specific value that is a function of attributes included in the user profile  106 . Examples of attributes in the user profile  106  include the user&#39;s age, the amount of money the user has spent using the services of the financial software application  114 , the user&#39;s income, the number of times the user has previously received assistance from support agents, and the length of time that the user has been a customer of the financial software application  114 . In one example, the threshold score may be defined as: 
     
       
         
           
             
               1 
               F 
             
             · 
             
               
                 ∑ 
                 
                   i 
                   = 
                   1 
                 
                 k 
               
               ⁢ 
               
                 
                   W 
                   i 
                 
                 · 
                 
                   A 
                   i 
                 
               
             
           
         
       
     
     where k refers to an integer number of attributes in the user profile  106 , i is an integer index ranging from 1 to k, A i  refers to the value of the ith attribute in the user profile  106 , W i  refers to a predefined weight for A i , and F is a scale factor (e.g., used to ensure the possible range of threshold scores is the same size as the range of possible score values). 
     If the predictive model  110  determines a score for the user meets a threshold, the support routing module  112  infers that live assistance from a support agent should be provided to the user. Once the support routing module  112  determines that the score (as determined based on the paralinguistic features) meets the threshold score, the support routing module  112  provides the user with a prompt offering help from a support agent. The prompt may allow the user to communicate with a support agent through a messaging window or through a voice connection (e.g., telephone). In one example, the prompt may provide a button that the user selects to open a messaging window with an agent or to initiate a voice chat with an agent via the network  102 . In another example, the user can request that a support agent call the user at a specified telephone number (e.g., of the mobile device  116 ). 
     In another example, the prompt may also provide a telephone support number (e.g., associated with the support center  122 ) for the user to call. The application  118  may send the telephone number of the mobile device  116  to the support routing module  112 . In addition, the support routing module may identify any other telephone numbers associated with the user profile  106 . The prompt may also allow the user to specify a phone number from which the user will call. When the user calls the telephone support number from any telephone number associated with the user, the caller identification (ID) module  132  notifies the support routing module  112 . The support routing module  112  detects that the user has accepted the prompt based on the notification. 
     Provided the user selects to communicate with an agent, the support routing module  112  compares the score for the user to a mapping  113 . The mapping  113  associates scores with experience levels, customer-satisfaction levels, or levels of other attributes found in the support agent profiles  108 . For example, a score indicating that the customer is in a negative emotional state may result in the support routing module  112  routing the call to a an experienced support agent, or one with a history of favorable outcomes for support encounters initiated with such customers. The support routing module  112  identifies the attribute levels that correspond to the score in the mapping  113 . The attribute levels to which the score maps serve as baselines for selecting a support agent to assist the user. The support routing module  112  compares the attribute levels to the support agent profiles  108  to identify a set of support agents who meet (or exceed) the attribute levels. Furthermore, in one embodiment, the support routing module  112  also determine appropriate areas of expertise (e.g., tax law expertise, expertise with the inner workings of the financial software application  114 , or expertise with a specific language) based on additional context. The additional context may include a telephone number dialed for support, a user-provided description of the issue for which the user seeks assistance, or a page of the financial software application  114  currently loaded in the application  118 . For example, the support routing module  112  may determine that a user-provided description of the issue suggests the user needs an agent who understands income tax law for a specific jurisdiction. The support routing module  112  can ensure that only agents with this particular expertise are included in the set of support agents. The support routing module  112  then indicates the set of support agents to the availability monitor  130 . 
     For each agent in the set, the availability monitor  130  determines how many customers are currently waiting in the corresponding agent queue of the agent queues  126 . If there are multiple agents in the set (i.e., multiple agents who meet the attribute levels to which the score for the user maps in the mapping  113 ), the support routing module  112  can add the user to the agent queue that currently has the smallest number of customers. If there is only one agent in the set, the support routing module  112  adds the user to the agent queue corresponding to that agent. If all or none of the agents that are currently working at agent terminals  124  are in the set, the support routing module can add the user to the default queue. 
     Support agent profiles  108  include attributes of support agents that work at agent terminals  124  at a support center  122 . For each agent working at one of the agent terminals  124 , there is a corresponding agent queue in the agent queues  126 . When an agent becomes available to assist a customer, a call for the next customer in the corresponding agent queue is transferred to the corresponding agent. A customer may be placed at a position in the agent queue that is determined based on the customer&#39;s value or priority level. For example, an angry customer who uses a free version of the financial software application  114  may be placed behind a mildly frustrated customer who uses an expensive version because the latter customer may be considered more valuable. If no customers are in the corresponding agent queue, the agent serves the next customer in the default queue. 
     Each of the support agent profiles  108  may include many agent attributes, such as experience level, language skills, specialized training, age, personality type, gender, voice quality, typing speed. In addition, each of the support agent profiles  108  can include a customer-satisfaction attribute that quantifies satisfaction levels of customers who the agent assists. This customer-satisfaction attribute can be based on explicit customer feedback (e.g., in surveys that ask customers to rate their satisfaction after being assisted by agents). The customer-satisfaction attribute can also be based on implicit indicators of customer satisfaction, such as whether a customer continues to use the financial software application  114  after being assisted by an agent. 
     Depending on the mode of communication selected by the user, the user may be placed in the agent queue in one of several ways. For example, if the user selected to call a telephone support number, the support routing module  112  places the incoming call from the user in the agent queue. If the user selected to communicate with a support agent via a messaging window, the support routing module  112  places an instant-messaging connection with the user in the agent queue. If the user selected to have an agent call the user, the support routing module  112  inserts a placeholder for the user in the agent queue. When the support agent reaches the placeholder in the queue, a telephone associated with the agent terminal where the support agent is working automatically dials the telephone number provided by the user. 
     By default, the agent queues  126  operate on a first-in, first-out basis. However, the support routing module  112  can also sort customers in the agent queues  126  by score so that customers with scores indicating greater negative emotional states are given priority. In another embodiment, the support routing module  112  can sort customers in the agent queues  126  according to a priority level that depends on both profile information (e.g., as found in user profile  106 ) and score. In one example, the priority level for the user may be defined as: 
     
       
         
           
             S 
             + 
             
               
                 ∑ 
                 
                   j 
                   = 
                   1 
                 
                 x 
               
               ⁢ 
               
                 
                   α 
                   j 
                 
                 · 
                 
                   A 
                   j 
                 
               
             
           
         
       
     
     where x refers to an integer number of attributes in the user profile  106 , j is an integer index ranging from 1 to x, A j  refers to the value of the jth attribute in the user profile  106 , α j  refers to a predefined priority-level weight for A j , and S is the score for the user. 
       FIG. 2  illustrates a detailed view of the mobile device  116 , according to one embodiment. As shown, the application  118  includes a support component  202 . The I/O components  120  include a microphone  204 , a speaker  206 , and a touch screen  208 . The application  118  presents output to the user via the touch screen  208  and or the speaker  206 . The application  118  can also receive user input via the touch screen  208 . 
     The microphone  204  receives audio input from the user. The audio input may include any sound the microphone  204  can detect, such as voice commands or spoken data supplied to the application  118 . 
     The support component  202  can transmit audio input to a server (e.g., server  104 ) for analysis. At the server, a predictive model determines a score using paralinguistic features characterizing the audio input. If the score meets a threshold, the support component  202  may present a dialog (or other interface component) informing the user of an option to communicate with a support agent. If the option is accepted, the support component  202  may display an instant-messaging window for the user to communicate with a support agent. Alternatively, the support component  202  may display a telephone number for the user to call to speak with a support agent. The support component  202  may also send a phone number of the mobile device  116  to the server so that an incoming call from the user to a support center can be identified. In another example, the support component  202  may also call the user. The support component  202  may also activate a voice chat feature of application  118  to enable the user to communicate with a support agent. 
       FIG. 3  illustrates a method  300  for providing support for an application, according to one embodiment. At step  302 , a mobile device receives audio input from a user via a microphone. At least some of the audio input corresponds to sound of the user, including spoken words, sighs, laughter, grunts, or breaths. The user may provide the audio input while verbally interacting with one or more screens or pages of an application interface (e.g., shown in a browser). 
     At step  304 , a feature extractor identifies paralinguistic features of the audio input. Paralinguistic technically means “alongside linguistics” and therefore deals with phenomena that are modulated onto or embedded into the audio input. For example, some of the paralinguistic features may be low level descriptors that are extracted from frames of the audio stream (e.g., 10-30 millisecond frames). These acoustic low level descriptors may include intonation (pitch, etc.), intensity (energy, Taeger functions, etc.), linear prediction cepstral coefficients (LPCC), perpetual linear prediction (PLP) parameters, cepstral coefficients (Mel frequency cepstral coefficients (MFCCs), etc.), formants (amplitude, position, width, dispersion, etc.), spectrum (Mel frequency bands (MFB), NMF-based components, MPEG-7 audio spectrum projection, roll-off, etc.), TF transformation (wavelets, Gabor, etc.), harmonicity (harmonics-to-noise ratio (HNR), noise-to-harmonics ratio (NHR), etc.), and/or perturbation (jitter, shimmer, etc.). Further, the paralinguistic features may include low level descriptors such as phoneme sequences, non-linguistic strings (laughter, sighs, etc.), and/or disfluencies (false starts, filled/unfilled pauses, etc.). The paralinguistic features may also include fundamental frequency, volume, and other measurable characteristics of the audio input. The paralinguistic information may be extracted utilizing software such as EmoVoice, OpenEar, Yaafe, Voice Search DB with Accent and Tone, VoiceVault, AT&amp;T APIs, etc. 
     At step  306 , a predictive model determines a score predicting an emotional state (or measures of different emotional states) of the user based on the paralinguistic features. For example, a machine-learning model may be trained to predict an emotional state based on paralinguistic features. There are many different types of inductive and transductive machine-learning models that can be used for the predictive model. Examples of machine-learning models include adsorption models, neural networks, support vector machines, radial basis functions, Bayesian belief networks, association-rule models, decision trees, instance-based models (e.g., k-NN), regression models, Hopfield networks, deep belief networks, and Q-learning models. 
     Many configurations and parameter combinations may be possible for a given type of machine-learning model. With a neural network, for example, the number of hidden layers, the number of hidden nodes in each layer, and the existence of recurrence relationships between layers can vary. True gradient descent or stochastic gradient descent may be used in the process of tuning weights. The learning rate parameter, which partially determines how much each weight may be adjusted at each step, may be varied. Input features may be normalized. Other parameters that are known in the art, such as momentum, may also be applied to improve neural network performance. In another example, decision trees can be constructed using a variety of approaches. Some non-limiting examples include the iterative dichotomiser 3 (ID3), Classification and Regression Tree (CART), and CHi-squared Automatic Interaction Detection (CHAID) methods. These methods may use one or more different metrics to determine the order in which attribute values are examined in decision trees. Some non-limiting examples of such metrics include information gain and Gini impurity. In addition, pruning methods may be applied to improve decision tree performance. Some non-limiting examples of pruning techniques include reduced error pruning, cost complexity pruning, and alpha-beta pruning. 
     Furthermore, individual machine learning models can be combined to form an ensemble machine-learning model. An ensemble machine-learning model may be homogenous (i.e., using multiple member models of the same type) or non-homogenous (i.e., using multiple member models of different types). Individual machine-learning models within an ensemble may all be trained using the same training data or may be trained using overlapping or non-overlapping subsets randomly selected from a larger set of training data. 
     At step  308 , a support routing module compares the score to a threshold score. The threshold score may apply to all users (or groups of users). In other cases, the threshold score may be a user-specific value determined using attributes about that user. If the score does not meet the threshold score, the support routing module determines whether the user has requested assistance (thereby proceeding to step  310 ). 
     Otherwise, at step  309 , the support routing module presents a prompt to communicate with a support agent to the user (e.g., via a display on the mobile device). The prompt may appear to the user in a header bar, a side bar, a pop-up window, or a dialog box. The prompt may present a button the user can select to open a messaging window with an agent or to initiate an Internet telephony connection with an agent. The prompt may also allow the user to request that a support agent call the user at a specified telephone number. The prompt may also provide a telephone support number for the user to call to communicate with an agent. 
     At step  310 , the support routing module determines whether the user has requested assistance. For example, the user can accept the prompt (e.g., by clicking on a button). Alternatively, the user can request assistance directly even if the score does not meet the threshold score. For example, the user may utter a keyword such as “assistance” or “live support” or click on an assistance button. In either case, if the user requests, the support routing module proceeds to step  312 . Otherwise, the method  300  terminates. 
     At step  312 , the support routing module selects a support agent to assist the user based on the score. For example, the support routing module may compare the score to a mapping to determine baseline levels of one or more attributes. The support routing module compares the baseline levels to agent profiles to identify a set of agents who can assist the user. The support routing module can select a support agent from the set based on a number of customers waiting in a corresponding agent queue for the agent. 
       FIG. 4  illustrates a method  400  for implicitly determining how satisfied customers are when assisted by a support agent, according to one embodiment. At step  402 , a recording device records customer calls answered by a support agent for whom the customer-satisfaction attribute is to be determined. 
     At step  404 , an attribute calculator selects an audio recording of a customer call answered by the support agent. At step  406 , a feature extractor extracts paralinguistic features from the audio recording. Since both a customer&#39;s voice and the support agent&#39;s voice may be heard at different points in the audio recording, the feature extractor may be configured to extract the paralinguistic features from portions of the audio recording that include the customer&#39;s voice. 
     At step  408 , a predictive model (e.g., a machine-learning model) determines a score that represents the customer&#39;s emotional state during the call based on the paralinguistic features. In one example, the score may be a real number range from −1 (representing a very negative emotional state) to 1 (representing a very positive emotional state). 
     At step  410 , a function determines an implicit customer-satisfaction level for the audio recording based on the score. The function may be linear or non-linear. In one example, the score can range from −1 to 1, where −1 indicates a very negative emotional state and 1 represents a very positive emotional state. The customer-satisfaction level can range from M 1  (indicating a very low level of customer satisfaction) to M 2  (indicating a very high level of customer satisfaction), where M 1 &lt;M 2 . In this example, the function that defines the customer-satisfaction level based on the score S may be defined as: 
     
       
         
           
             
               
                 1 
                 2 
               
               · 
               
                 ( 
                 
                   
                     M 
                     2 
                   
                   - 
                   
                     M 
                     1 
                   
                 
                 ) 
               
               · 
               S 
             
             + 
             
               M 
               1 
             
             + 
             
               
                 1 
                 2 
               
               · 
               
                 ( 
                 
                   
                     M 
                     2 
                   
                   - 
                   
                     M 
                     1 
                   
                 
                 ) 
               
             
           
         
       
     
     Alternatively, the function that defines the customer-satisfaction level based on the score S may be a non-linear and defined as: 
     
       
         
           
             
               
                 1 
                 2 
               
               · 
               
                 ( 
                 
                   
                     M 
                     2 
                   
                   - 
                   
                     M 
                     1 
                   
                 
                 ) 
               
               · 
               
                 S 
                 3 
               
             
             + 
             
               M 
               1 
             
             + 
             
               
                 1 
                 2 
               
               · 
               
                 ( 
                 
                   
                     M 
                     2 
                   
                   - 
                   
                     M 
                     1 
                   
                 
                 ) 
               
             
           
         
       
     
     Other definitions of the function are also possible. 
     At step  412 , the attribute calculator determines whether there are any additional audio recordings of customer calls answered by the support agent. If there are, steps  404 - 410  are repeated. Otherwise, the method  400  proceeds to step  414 . 
     At step  414 , the attribute calculator collects explicit customer-satisfaction levels reported in post-contact surveys that customers filled out after interacting with the agent. 
     At step  416 , the attribute calculator determines a customer-satisfaction attribute of the support agent based on both the implicit and explicit customer-satisfaction levels. For example, the attribute calculator may define the customer-satisfaction attribute as the mean, median, or mode of the customer-satisfaction levels. The customer-satisfaction attribute is included in a profile for the support agent. 
       FIG. 5  illustrates a method  500  for determining an attribute measuring how well a support agent improves customer emotional states, according to one embodiment. At step  502 , a recording device records customer calls answered by a support agent for whom the attribute is to be determined. 
     At step  504 , an attribute calculator selects an audio recording of a customer call answered by the support agent. At step  506 , a feature extractor extracts paralinguistic features of a customer voice during a first time interval in the audio recording. In one embodiment, the first time interval can begin at the beginning of the call. The length of the first time interval may be a predefined number of minutes or seconds. Alternatively, the length of the first time interval may be a predefined percentage of the total length of the call. 
     At step  508 , a predictive model (e.g., a machine-learning model) determines a first score that represents the customer&#39;s emotional state during the first time interval based on the paralinguistic features extracted during the first time interval. In one example, the score may be a real number ranging from −1 (representing a very negative emotional state) to 1 (representing a very positive emotional state). 
     At step  510 , a feature extractor extracts paralinguistic features of a customer voice during a second time interval in the audio recording. In one embodiment, the second time interval can end at the end of the call. The length of the second time interval may be a predefined number of minutes or seconds. Alternatively, the length of the second time interval may be a predefined percentage of the total length of the call. In one embodiment, the second time interval does not overlap with the first time interval. 
     At step  512 , the predictive model (e.g., a machine-learning model) determines a second score that represents the customer&#39;s emotional state during the second time interval based on the paralinguistic features extracted during the second time interval. At step  514 , the attribute calculator determines a difference between the second score and the first score. For example, the attribute calculator may subtract the first score from the second score to determine the difference. 
     At step  516 , the attribute calculator whether there are any additional audio recordings of customer calls answered by the support agent. If there are, steps  504 - 514  are repeated. Otherwise, the method  500  proceeds to step  518 . 
     At step  518 , the attribute calculator determines an attribute of the support agent based on score differences. For example, the attribute calculator may define the attribute as the mean, median, or mode of the score differences. The attribute is included in a profile for the support agent. 
       FIG. 6  illustrates a method  600  for training a predictive model to predict an emotional state from paralinguistic features of spoken utterances, according to one embodiment. At step  602 , a model creator receives audio recordings and corresponding score labels measuring emotional states. 
     At step  604 , the model creator extracts a set of paralinguistic features from each audio recording. At step  606 , the model creator generates training instances from the features and the score labels. Each training instance comprises the set of features extracted from the corresponding audio recording and the score label of the corresponding to the audio recording. The score label represents the value (e.g., of the target variable) that the predictive model will be trained to predict based on the set of features. 
     At step  608 , the model creator applies a feature-selection method to identify a collection of non-redundant paralinguistic features that are correlated with the scoring labels. Some feature-selection techniques that the model may apply include the Las Vegas Filter (LVF), Las Vegas Incremental (LVI), Relief, Sequential Forward Generation (SFG), Sequential Backward Generation (SBG), Sequential Floating Forward Search (SFFS), Focus, Branch and Bound (B &amp; B), and Quick Branch and Bound (QB&amp;B) techniques. 
     There are several reasons why it is useful to identify the collection of non-redundant paralinguistic features correlated with the scoring labels before training the predictive model. For example, some features extracted at step  604  may not be appreciably correlated to the scoring labels that the predictive model is designed to predict. Irrelevant features can cause overfitting some predictive models. In addition, one feature may be so closely correlated with another feature that it would be redundant to use both for training. Furthermore, redundant features can blunt the accuracy of some distance metrics used in instance-based (nearest neighbor) models. Also, when irrelevant or redundant features are present in training data, most predictive models take longer to train. 
     In general, the number of features included in the collection should be small relative to the total number of training instances in the training data. In some embodiments, the number of features selected for the collection can be at least two orders of magnitude smaller than the number of training instances in the training data. 
     At step  610 , the model creator trains the predictive model to predict the target variable (i.e., the score or scores measuring an emotional state) based on the collection of non-redundant paralinguistic features. The way the training is accomplished depends on which type of machine-learning model used as the predictive model. A neural network, for example, iteratively adjusts weights within layers of network nodes based on an error terms defined by the backpropagation technique. Depending on the learning rate and the number of layers, the neural network may be trained for several hundred epochs (i.e., iterations through the entire set of training data). 
     At step  612 , the model creator stores the predictive model for future use. The predictive model is stored in a format that is accessible to a support routing module. 
       FIG. 7  illustrates a server system  700  that routes users to support agents based on paralinguistic features of user audio input, according to one embodiment. As shown, the server system  700  includes, without limitation, a central processing unit (CPU)  702 , one or more I/O device interfaces  704  which may allow for the connection of various I/O devices  714  (e.g., keyboards, displays, mouse devices, pen input, etc.) to the server system  700 , network interface  706 , a memory  708 , storage  710 , and an interconnect  712 . 
     CPU  702  may retrieve and execute programming instructions stored in the memory  708 . Similarly, the CPU  702  may retrieve and store application data residing in the memory  708 . The interconnect  712  transmits programming instructions and application data among the CPU  702 , I/O device interfaces  704 , network interface  706 , memory  708 , and storage  710 . CPU  702  can represent a single CPU, multiple CPUs, a single CPU having multiple processing cores, and the like. Additionally, the memory  708  represents random access memory. Furthermore, the storage  710  may be a disk drive. Although shown as a single unit, the storage  710  may be a combination of fixed or removable storage devices, such as fixed disc drives, removable memory cards or optical storage, network attached storage (NAS), or a storage area-network (SAN). 
     As shown, memory  708  includes support routing module  112 , financial software application  114 , and feature extractor  115 . As shown, storage  710  includes user profile  106 , support agent profiles  108 , predictive model  110 , and mapping  113 . 
     The financial software application  114  receives audio input from a user. The feature extractor  115  extracts paralinguistic features from the audio input. The paralinguistic features are provided as input to the predictive model  110 . Based on the paralinguistic features, the predictive model  110  determines a score that represents a current emotional state of the user. Based on the score, the support routing module  112  provides a prompt for the user to request assistance from a support agent. When the user accepts the prompt, the support routing module  112  compares the score to the mapping  113  to determine baseline attribute levels for a support agent to assist the user. The support routing module  112  compares the baseline attribute levels to the support agent profiles  108  to identify a support agent to assist the user. The support routing module  112  adds the user to an agent queue for a support agent that meets the baseline attributes. 
     In the preceding, reference is made to embodiments presented in this disclosure. However, the scope of the present disclosure is not limited to specific described embodiments. Instead, any combination of the following features and elements, whether related to different embodiments or not, is contemplated to implement and practice contemplated embodiments. Furthermore, although embodiments disclosed herein may achieve advantages over other possible solutions or over the prior art, whether or not a particular advantage is achieved by a given embodiment is not limiting of the scope of the present disclosure. Thus, the following aspects, features, embodiments and advantages are merely illustrative and are not considered elements or limitations of the appended claims except where explicitly recited in a claim(s). Likewise, reference to “the invention” shall not be construed as a generalization of any inventive subject matter disclosed herein and shall not be considered to be an element or limitation of the appended claims except where explicitly recited in a claim(s). 
     Aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module,” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon. 
     Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples a computer readable storage medium include: an electrical connection having one or more wires, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the current context, a computer readable storage medium may be any tangible medium that can contain, or store a program. 
     While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.