Abstract:
A method, a system, and computer readable medium comprising instructions for monitoring caller experience are provided. The method comprises obtaining caller experience of at least one step in a call flow, selecting a plurality of steps from the at least one step to form a group, presenting caller experience of the plurality of steps in first level of the call flow, and presenting the group as a step in a second level of the call flow.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     The present application is related to commonly assigned U.S. patent application Ser. No. 12/028,248 entitled REAL-TIME MONITORING OF CALLER EXPERIENCE IN A CALL FLOW, U.S. patent application Ser. No. 12/028,229 entitled REAL-TIME ADJUSTMENT FOR BETTER CALLER EXPERIENCE filed on even date herewith, the entire contents of which are incorporated by reference herein. 
     FIELD OF THE DISCLOSURE 
     The present disclosure relates generally to an automated voice response system. More particularly, the present disclosure provides a method, a system, and a computer readable medium comprising instructions for real-time monitoring of caller experience for a group of steps in a call flow of an automated voice response system. 
     BACKGROUND OF THE DISCLOSURE 
     Callers who called into one of many automated voice response systems for services have encountered different experience in navigating through the system. For example, a caller may not have understood a prompt that was provided by the system, and thus, experienced confusion in making a selection. Another example is that a caller may have missed an option that was provided in the selection, and thus, became lost in the system. In yet another example, a caller may have experienced a long delay in obtaining data from the system, and thus, hang up before the service was actually provided. 
     Caller confusions cause delays in providing services, and confused callers are more likely to hang up and retry their calls. In addition, confused callers who are dissatisfied with their services are more likely to terminate their relationships with their service providers. Furthermore, confused callers provide valuable insights in system improvements. For example, the need of confused callers for certain options in the system, the willingness of confused callers to adapt to new options or services introduced in the system, the amount of time confused callers are willing to wait for assistance, etc. Therefore, what is needed is a method and system for providing real-time monitoring of caller experience in an automated voice response system. In this way, service providers may dynamically review where callers are confused or frustrated in the system in order to provide corrective measures to improve system performance. 
     SUMMARY OF THE DISCLOSURE 
     A method for monitoring caller experience is provided. In one embodiment, the method comprises obtaining caller experience of at least one step in a call flow, selecting a plurality of steps from the at least one step to form a group; presenting caller experience of the plurality of steps in first level of the call flow, and presenting the group as a step in a second level of the call flow. 
     In an alternative embodiment, the method for monitoring caller experience comprises identifying a group of steps in a call flow based on their functions, determining a total caller experience for the group of steps, and presenting the total caller experience in the call flow to a service provider. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  depicts a diagram of a system for real-time monitoring of caller experience in a call flow in accordance with one embodiment of the present disclosure; 
         FIG. 2  depicts a flowchart of a process for real-time monitoring of caller experience in a call flow in accordance with one embodiment of the present disclosure; 
         FIG. 3  depicts a flowchart of a process for developing a formula for monitoring caller experience in a call flow in accordance with one embodiment of the present disclosure; 
         FIG. 4A  depicts a diagram illustrating an exemplary monitoring formula in accordance with one embodiment of the present disclosure; 
         FIG. 4B  depicts a diagram illustrating a continuation of an exemplary monitoring formula in accordance with one embodiment of the present disclosure; 
         FIG. 5  depicts a flowchart of an exemplary monitoring call flow in accordance with one embodiment of the present disclosure; 
         FIG. 6  depicts a flowchart of a process for developing a caller experience rating for each step of a call flow in accordance with one embodiment of the present disclosure; 
         FIG. 7  depicts a diagram illustrating an exemplary construction of a caller experience for each step of the call flow in accordance with one embodiment of the present disclosure; and 
         FIG. 8  depicts a diagram illustrating an exemplary color-coded monitoring flow chart in accordance with one embodiment of the present disclosure; 
         FIG. 9  depicts a flowchart of a process for monitoring caller experience of a group of steps in a call flow in accordance with one embodiment of the present disclosure; 
         FIG. 10A  depicts a diagram of an exemplary monitoring call flow in accordance with an alternative embodiment of the present disclosure; 
         FIG. 10B  depicts a diagram of an exemplary main menu call flow in accordance with an alternative embodiment of the present disclosure; 
         FIG. 10C  depicts a diagram of an exemplary payments menu call flow in accordance with an alternative embodiment of the present disclosure; 
         FIG. 10D  depicts a diagram of an exemplary payment sub-menu call flow in accordance with an alternative embodiment of the present disclosure; 
         FIG. 10E  depicts a diagram of an exemplary billing menu call flow in accordance with an alternative embodiment of the present disclosure; 
         FIG. 10F  depicts a diagram of an exemplary service updates menu call flow in accordance with an alternative embodiment of the present disclosure; 
         FIG. 11A  depicts a diagram illustrating an exemplary monitoring formula for a group of steps in accordance with an alternative embodiment of the present disclosure; 
         FIG. 11B  depicts a diagram illustrating a continuation of an exemplary monitoring formula for a group of steps in accordance with an alternative embodiment of the present disclosure; 
         FIG. 12A  depicts a diagram illustrating an exemplary construction of a caller experience for each step of the payments sub-menu call flow in accordance with an alternative embodiment of the present disclosure; 
         FIG. 12B  depicts a diagram illustrating a continuation of an exemplary construction of a caller experience for each step of the payments sub-menu call flow in accordance with an alternative embodiment of the present disclosure; 
         FIG. 13  depicts a diagram illustrating an exemplary color-coded payment sub-menu flow chart in accordance with one embodiment of the present disclosure; 
         FIG. 14  depicts a flow chart of a process for determining overall caller experience of a group of steps in accordance with one embodiment of the present disclosure; and 
         FIG. 15  depicts a diagram illustrating an exemplary color-coded payment menu flow chart in accordance with one embodiment of the present disclosure. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     With reference to the figures and in particular with reference to  FIG. 1 , a diagram of a system for real-time monitoring of caller experience in a call flow is depicted in accordance with an embodiment of the disclosure. In system  100 , caller  102  places a call to voice response system unit  104  via a device, such as a telephone, an Internet Protocol-enabled phone, or a data processing system, such as a computer. Upon receiving the call, voice response system unit  104  guides the caller  102  through a number of selection options in order to obtain the desired services. The voice response system unit  104  is communicably coupled to a database  106 . For each call that the voice response system unit  104  handles, it collects data associated with the call and information about the caller and stores the data and information in database  106 . The information may include the amount of time a caller spent on a prompt, the percentage of callers who reached a certain prompt, etc. 
     The voice response system unit  104  is also communicably coupled to a performance tuning system  108 . Performance tuning system  108  monitors the performance of voice response system unit  104  for each call. Upon completion of a certain number of calls or upon request, performance tuning system  108  generates a tuning analysis report  110 . The tuning analysis report  110  is used for analysis in improving performance and efficiency of the voice response system unit  104 . Systems  104  and  108  may be a data processing system, such as a desktop computer, a laptop computer, a server or any other type of device capable of sending and receive information via a wired or wireless protocol. The transfer of information between system  104 , database  106 , and performance tuning system  108  may be accomplished via at least one of a wireless protocol, a wired protocol and a combination of a wireless protocol and a wired protocol. The steps performed in this figure are performed by software, hardware, firmware, and/or the combination of software, hardware, and/or firmware. 
     In addition to database  106  and performance tuning system  108 , voice response system unit  104  is also communicably coupled to a voice/dial selection system  112 . The voice/dial selection system  112  monitors voice or dial options selected by each caller  102  and generates spindown selection summaries  114 . Spindown selection summaries  114  comprise a summary of selections made by each caller  102 . The summaries provide useful data for analysis of the overall caller experience. Data from tuning analysis report  110  and spindown selection summaries  114  may then be combined into a number of data files  116  for processing. 
     Upon combining the data into data files, a caller experience monitoring system  126  then applies a monitoring formula  118  to data in the data files  116  to identify problem areas in caller experience. In one embodiment, monitoring formula  118  utilizes a set of metrics for monitoring caller experience. For example, one set of metrics may comprise seven criteria used to rate each step of the call flow. These criteria are applied against the data in data files  116  to generate results for review by the service provider  124 . More details about the monitoring formula are discussed with reference to  FIGS. 3, 4A, and 4B  below. 
     The results may be presented in different formats. In one embodiment, the results may be presented in a monitoring flow chart that is color-coded to show problem areas. In an alternative embodiment, the results may be presented on a Web interface to service providers who may access it via the Internet. In addition to presenting the result to service provider  124 , caller experience monitoring system  126  may provide helpful feedback to voice/dial selection system  112  and voice response system unit  104  to take corrective measures in order to improve the overall caller experience. 
     Referring to  FIG. 2 , a flowchart of a process for real-time monitoring of caller experience in a call flow is depicted in accordance with one embodiment of the present disclosure. In this embodiment, process  200  may be implemented as instructions embodied in a computer readable medium executing within caller experience monitoring system  126 . Process  200  begins at step  202  to collect call information and stores the information into a database  202 . Next, process  200  continues to step  204  to collect selections by callers and stores them into spindown selection summaries. Process  200  then continues to step  206  to generate a tuning analysis report based on the data and information stored in the database. Upon generating the tuning analysis report, process  200  continues to step  208  to combine data from the tuning analysis report and the spindown selection summaries into a number of data files. 
     Process  200  then continues to step  210  to apply a monitoring formula to data in the number of data files to generate results for review by service providers. Process  200  then completes at step  212  to present the results in different formats to the service provider, who may then revised their call flow or add new steps in the call flow to improve the caller experience. 
     Referring to  FIG. 3 , a flowchart of a process for developing a formula for monitoring caller experience in a call flow is depicted in accordance with one embodiment of the present disclosure. In one embodiment, process  300  may be implemented as instructions embodied in a computer readable medium executing within caller experience monitoring system  126 . Process  300  begins at step  302  to identify at least one criterion for the set of metrics used to monitor caller experience. The at least one criterion may be inputted into the caller experience monitoring system by a customer or a service provider. 
     Next, process  300  continues to step  304  to assign each of the at least one criterion a full or half value. A full value provides a full rating for the criterion, while a half value provides a half rating for the criterion. Process  300  then continues to step  306  to assign a range of values for each color rating of each of the criteria. For example, for criterion “negative behavior”, a range of 6% to 14% may be assigned to its yellow rating. 
     After a range of values is assigned for each color rating of each criterion, process  300  continues to step  308  to divide the range of values into at least one sub-range. For example, the range of 6% to 14% may be divided into a sub-range of 6% to 8%, a sub-range of 9% to 12%, and a sub-range of 12% to 14%. Process  300  then completes at step  310  to assign a sub-rating for each of the at least one sub-range. For example a sub-rating of 2 may be assigned to the sub-range of 12% to 14%, a sub-rating of 3 may be assigned to the sub-range of 9% to 12%, and a sub-rating of 4 may be assigned to the sub-range of 6% to 8%. 
     Referring to  FIGS. 4A and 4B , diagrams illustrating an exemplary monitoring formula are depicted in accordance with one embodiment of the present disclosure. In this example, monitoring formula comprises a set of metrics  400 . In metrics  400 , seven criteria are defined to monitor caller experience for each step of the call flow. The seven criteria comprise negative behavior  402 , caller discontent  404 , dialog and menus  406 , preference  408 , and persona audio quality  410 . In addition, as shown in  FIG. 4B , the seven criteria comprise navigation length  412  and prompt length  414 . The seven criteria determine the overall customer satisfaction of the caller. 
     Monitoring formula  400  also provides a traffic light-like color rating for each dialog state. A green color-rating  416  represents that the caller experienced great customer satisfaction. A yellow color-rating  418  represents that the caller has serious questions about customer satisfaction. The yellow color-rating  418  notifies the service provider to examine weak areas of the designed call flow and make changes in a timely manner. A red color-rating  420  represents that the caller experience has been seriously impacted. In addition, call flow program efficiency has also been impacted and changes to the system are needed immediately. 
     In this example, each of the color ratings of the seven criteria is defined with a range of values. For example, negative behavior  402  is examined based on the combined one time timeouts and invalids. The range for combined one time timeouts and invalids is set to be fewer than 6% for the green rating, from 6% to 14% for a yellow rating, and over 15% for a red rating. Each of the color ratings is further sub-divided into sub-ratings of 1 to 5. For example, the yellow rating of negative behavior  402  is subdivided into sub-ratings 2 to 4 with rating 2 ranging from 12% to 14%, rating 3 ranging from 9% to 12%, and rating 4 ranging from 6% to 8%. With color-ratings and sub-ratings, monitoring formula  400  provides more accurate representation of caller experience. The lower the number of timeouts and invalids, the better the caller experience. 
     Unlike negative behavior  402 , caller discontent  404  is examined based on combined final timeouts, invalids, and inappropriate hangups. The range for combined final timeouts, invalids, and inappropriate hangups is set to be fewer than 6% for the green rating, from 6% to 9% for a yellow rating, and over 10% for a red rating. The yellow rating of caller discontent  404  is further subdivided into sub-ratings 2 to 4 with rating 2 of 9%, rating 3 ranging from 7% to 8%, and rating 4 of 6%. Similar to negative behavior, the lower the number of timeouts, invalids, and unwanted hangups, the better the caller experience. 
     Dialogue and menus  406  are examined based on three categories: misguided, ambiguous, and distinct. Misguided means that the design of the call flow is counterproductive and it confuses and irritates callers. Ambiguous means some use of best practices, but the step still confuses callers to a point of exhaustion. Distinct means the use of best practices for optimal satisfaction. To achieve distinct quality, dialogues and menus should have low incidence of timeouts and invalids. Thus, the more distinct the use of dialogues and menus, the better the caller experience. 
     Preferences  408  represent callers&#39; responses when the system requests data, for example, account numbers or don′ t -knows. Preferences are measured based on the average of the two lower menu selections. The range for the average is set to be fewer than 2% for the red rating, from 3% to 10% for a yellow rating, and over 10% for a green rating. The yellow rating of preferences  408  is further subdivided into sub-ratings 2 to 4 with rating 2 ranging from 3% to 4%, rating 3 ranging from 5% to 6%, and rating 4 ranging from 7% to 9%. 
     Persona-audio quality  410  measures the quality of a persona that is directed for instructional prompting. Because the system uses dynamic concatenation of life-live audio playback of data inputs entered by the callers, a higher quality the persona means a better caller experience. The quality of the persona is based on whether the persona-audio is an amateur voice and/or multiple voices, an improper voice, or a professional voice. Amateur voice and/or multiple voices are the least consistent with the persona of the service provider and thus receive a red rating. Improper voice receives a yellow rating, because it is less consistent with the persona of the service provider. Professional voice receives a green rating, because it is most consistent with the persona of the service provider. 
     Referring to  FIG. 4B , navigation length  412  measures caller experience based on the length of navigation callers have to go through. The shorter the length of navigation, the better the caller experience. In this example, the range for navigation length  412  is set to be greater than 3 minutes for the red rating, from 1.5 minutes to 3 minutes for a yellow rating, and under 1.5 minutes for a green rating. The yellow rating of navigation length  412  is further subdivided into sub-ratings 2 to 4 with rating 2 ranging from 2 minutes to 3 minutes, rating 3 ranging from 1.5 minutes to 2 minutes, and rating 4 ranging from 1 to 1.5 minutes. 
     Prompt length  414  measures caller experience based on the length of the prompt callers have to go through. The shorter the length of the prompt, the better the caller experience. In this example, the range for prompt length  414  is set to be greater than 30 seconds for the red rating, from 15 to 30 seconds for a yellow rating, and under 15 seconds for a green rating. The yellow rating of prompt length  414  is further subdivided into sub-ratings 2 to 4 with rating 2 ranging from 46 to 59 seconds, rating 3 ranging from 31 to 45 seconds, and rating 4 ranging from 16 to 30 seconds. 
     Based on the seven criteria, an overall caller experience percentage is calculated from monitoring formula  400 . In this example, an overall caller experience percentage of 95% or above achieves a green rating and represents great customer satisfaction, because the callers found this prompt helpful. An overall caller experience percentage of 76% to 94% achieves a yellow rating and represents serious questions in customer satisfaction, because the callers are confused. An overall caller experience percentage of 75% or below achieves a red rating and represents serious impact on overall customer satisfaction, because the callers are misguided. 
     Referring to  FIG. 5 , a flowchart of an exemplary monitoring call flow is depicted in accordance with one embodiment of the present disclosure. Process  500  may be implemented within voice response system, such as voice response system unit  104  when calls are received from callers  102 . In this example, process  500  begins at step  502  to provide a welcome message and a prompt for Spanish-speaking customers to callers. Next, process  500  continues to step  504  to notify the caller of necessary information, such as account or phone number and a prompt for the user to say “continue” if the caller has the necessary information or simply hangup if the caller does not have the necessary information. 
     Process  500  then continues to step  506  to prompt the caller for a customer account number or to say “I don&#39;t have it” if the caller does not have the customer account number. If the caller says “I don&#39;t have it”, process  500  continues to step  508  to prompt the caller for the customer phone number. If the caller enters a customer account number or phone number, process  500  continues to step  510  to prompt the number of receivers. If the caller enters more than ten receivers, process  500  continues to forward the call to an agent at step  511 . If the caller enters ten or less receivers, process  500  continues to step  512  to prompt the caller for a receiver number or to say “I don&#39;t have it” if the caller does not have the receiver number. 
     If the caller enters a receiver number, process  500  continues to step  516  to prompt the caller to confirm the access card number. If the caller says “I don&#39;t have it”, process  500  continues to step  514  to prompt the caller for an access card number, process  500  continues to step  518  to prompt the caller for multiple receivers. If the caller has multiple receivers, process  500  continues to step  520  to prompt the caller to confirm that the receiver is provided by the service provider. If the caller does not have multiple receivers, process  500  continues to step  522  to prompt the caller to confirm that the receiver is plugged in. Process  500  then continues to step  524  to prompt the caller to confirm the make of the receiver, to step  526  to confirm the model of the receiver, and to step  528  to confirm the serial number of the receiver. Process  500  then continues to step  530  to prompt the caller for the location of the receiver and to step  532  to activate the located receiver. After the receiver is activated, process  500  continues to step  534  to prompt the caller for local channels check. If the user confirms that local channels are received, process  500  terminates at step  536  to notify the caller that activation is complete. 
     After a monitoring call flow process is provided, caller experience monitoring system  126  may construct a caller experience for each step of the call flow. Referring to  FIG. 6 , a flowchart of a process for developing a caller experience rating for each step of a call flow is depicted in accordance with one embodiment of the present disclosure. In one embodiment, process  600  may be implemented as instructions embodied in a computer readable medium executing within caller experience monitoring system  126 . 
     Process  600  begins at step  602  to assign a sub-rating to each criteria of each step of the call flow based on the combined data in the data files. The sub-rating may be assigned based on the range of values assigned to each sub-range. Next, process  600  continues to step  604  to calculate a weight for each criterion of each step based on previously-assigned value for the criterion and the assigned sub-rating from step  602 . Process  600  then continues to step  606  to calculate a total weight for each step. The total weight may be obtained by totaling all the weights calculated in step  604 . Process  600  then completes at step  608  to divide the total weight calculated in step  606  by the number of calls that were processed through each step. For example, if 20 calls were processed through the welcome message call step, the total weight is divided by 20. 
     Referring to  FIG. 7 , a diagram illustrating an exemplary construction of a caller experience for each step of the call flow is depicted in accordance with one embodiment of the present disclosure. In metrics  700 , the combined data for each criteria of call flow step “welcome message” is listed under column  702 . For example, the combined data for caller discontent is 3%. A sub-rating  706  is assigned for each criteria based on the range of values assigned to each sub-range. In this example, a sub-rating of 5 is assigned for the caller discontent for a sub-rating ranging from 0 to 5%. Because the combined data of 3% falls within this sub-rating, a sub-rating of 5 is assigned to this criterion. 
     After the sub-rating is assigned, a weight  708  is calculated for each criteria based on the previously-assigned value for each criteria and the sub-rating. In this example, a weight of 5 is calculated for caller discontent based on the full value that was previously assigned for this criteria and the sub-rating of 5. If a half value is assigned for this criterion, a sub-rating of 2.5 is calculated. After a weight is calculated for each criterion, a total weight is calculated by totaling all the calculated weights. In this example, the total weight of call flow step “welcome message” is 15. Once the total weight is obtained, a caller experience for this step is obtained by dividing the total weight by the number of calls that were processed through this call flow step. In this example, the number of calls that were processed through this step is 20. Therefore, the total weight of 15 is divided by 20 to obtain a caller experience of 0.75. Based on the caller experience of this step, a color rating may be assigned to this call flow step and an overall caller experience may be developed. 
     As discussed above, the result of caller experience may be presented to service providers in many formats. In one embodiment, the result may be presented in a monitoring flow chart that is color-coded to show problem areas. Referring to  FIG. 8 , a diagram illustrating an exemplary color-coded monitoring flow chart is depicted in accordance with one embodiment of the present disclosure. Some steps in process  800  are color-coded based on the caller experience obtained using process  600  in  FIG. 6 . For example, if the caller experience is greater than 0.95, the step is green color-coded. If the caller experience ranges from 0.76 to 0.94, the step is yellow color-coded. If the caller experience is less than 0.75, the step is red color-coded. 
     In  FIG. 8 , red color-coding is represented by crosshairs. Red color-coding means that the caller is misguided. Yellow color-coding is represented by vertical lines. Yellow color-coding means that the caller is confused. Green color-coding is represented by horizontal lines. Green color-coding means that the call flow step is helping the caller. In addition to red, yellow, and green color-codings, other indications may be implemented to represent different level of caller experience without departing the spirit and scope of the present disclosure. For example, an orange color-coding may be implemented to represent a caller experience that is more than confusing but less than misguiding the caller. 
     In this example, steps  802  and  814  are red color-coded, which means that these steps misguided callers. These steps seriously impacted caller experience and program effectiveness, and thus, need correction immediately. Steps  804 ,  812 ,  822 ,  826 ,  834 , and  836  are yellow color-coded, which means that these steps confused the callers. These steps raise serious questions on customer satisfaction. These steps also identify weak areas that require correction in a timely manner. Steps  806 ,  808 ,  816 ,  820 ,  824 ,  828 ,  830 , and  832  are green color-coded, which means that these steps are helping the callers. These steps provide great customer satisfaction and do not require modification. 
     Based on color-codings, service providers may conduct focus groups to identify corrective actions to increase program effectiveness and efficiencies. For example, service providers may redesign the dialogue or options of weak steps to increase usage by callers. The new dialogue may then be implemented in the voice response system  104  and the new option may be implemented in the voice/dial selection system  112 . Once new dialogue or option is implemented, process  200  may be repeated to monitor caller experience of the improved program. 
     In addition to analyzing and presenting the caller experience for each step of the call flow, an aspect of the present disclosure analyzes aggregated steps of a call flow based on their functions and presents caller experience for the aggregated steps to the service providers. Referring to  FIG. 9 , a flowchart of a process for monitoring caller experience of a group of steps in a call flow is depicted in accordance with one embodiment of the present disclosure. 
     In one embodiment, process  900  may be implemented as instructions embodied in a computer readable medium executing within caller experience monitoring system  126 . Process  900  begins at step  902  to obtain the caller experience for each step of a call flow. The caller experience may be obtained using process  600  in  FIG. 6 . Next, process  900  continues to step  904  to group a number of steps in the call flow based on their functions. For example, a new service step, a transfer step, and a disconnect step may be grouped to perform a service update function. Process  900  then continues to step  906  to present the caller experience of the steps in each group based on their function to the service provider. Process  900  then completes at step  908  to present the group as a step in the overall call flow. In one embodiment, the caller experience of the steps is presented at a different level than the level in which the group is presented. For example, the caller experience of the service update steps is presented at a lower level than the level in which the group service update is presented. 
     Referring to  FIG. 10A , a diagram of an exemplary monitoring call flow is depicted in accordance with an alternative embodiment of the present disclosure. Process  1000  may be implemented within voice response system, such as voice response system unit  104  when calls are received from callers  102 . In this example, process  1000  begins at step  1002  to provide customer greeting to the caller. Next, process  1000  continues to step  1004  to prompt the caller for a preferred language. Process  1000  then continues to step  1006  to authenticate the caller based on an account number provided by the caller. If the caller is authenticated, process  1000  completes at step  1008  to provide a main menu to the caller. 
     Referring to  FIG. 10B , a diagram of an exemplary main menu call flow is depicted in accordance with an alternative embodiment of the present disclosure. Process  1008  may be implemented within voice response system, such as voice response system unit  104  when calls are received from callers  102 . In this example, process  1008  begins at step  1010  to prompt the caller to select payments, service updates, or password resets. If the caller selects payments, process  1008  continues to step  1012  to provide the payments menu to the caller. If the caller selects service updates, process  1008  continues to step  1014  to provide a service updates menu to the caller. If the caller selects password resets, process  1008  continues to step  1016  to provide password resets to the caller. 
     Referring to  FIG. 10C , a diagram of an exemplary payments menu call flow is depicted in accordance with an alternative embodiment of the present disclosure. Process  1012  may be implemented within voice response system, such as voice response system unit  104  when calls are received from callers  102 . In this example, process  1012  begins at step  1018  to notify the caller of payment information, such as account balance, due date, last payment date, and the like. Process  1012  then continues to step  1020  to prompt the caller to select either payments or billing. If the caller selects payments, process  1012  continues to step  1022  to provide a payment sub-menu to the caller. If the caller selects billing, process  1012  continues to step  1024  to provide a billing menu to the caller. 
     Referring to  FIG. 10D , a diagram of an exemplary payments sub-menu call flow is depicted in accordance with an alternative embodiment of the present disclosure. Process  1022  may be implemented within voice response system, such as voice response system unit  104  when calls are received from callers  102 . If the caller selects payments, process  1022  begins at step  1026  to prompt the caller to select making a payment, request extension, sign up for bank draft, or find a payment location. 
     If the caller selects request extension, process  1022  continues to step  1028  to determine whether or not the caller is eligible for extension. If the caller is eligible, process  1022  continues to step  1030  to notify the caller of new due date and the amount due. Process  1022  continues to step  1032  to prompt the caller to select “repeat that”, “accept extension date”, or an agent to make other arrangements. 
     If the caller is not eligible for extension, process  1022  continues to step  1034  to notify the caller that extension is not eligible. Process  1022  then continues to step  1036  to prompt the caller to select an agent to make other arrangements, “main menu”, or simply hang up. 
     If at step  1026  the caller selects sign up for bank draft, process  1022  continues to step  1038  to provide instructions on how to enroll online. Process  1022  then continues to step  1040  to prompt the caller to select “repeat that”, “request a form” or an agent to make other arrangements. 
     If at step  1026  the caller selects find a payment location, process  1022  continues to step  1042  to prompt the caller to select a mailing address for payment or to find nearby location. If the caller selects mailing address, process  1022  continues to step  1044  to notify the caller of the mailing address. If the caller selects finding nearby location, process  1022  continues to step  1046  to prompt the caller to enter a zip code. Once the caller enters a zip code, process  1022  continues to step  1048  to notify the caller of the nearest three locations. Process  1022  then continues to step  1050  to prompt the caller to select “main menu” or simply hang up. 
     If at step  1026  the caller selects making a payment, process  1022  continues to step  1052  to connect the caller with an agent to make payment arrangements. 
     Referring to  FIG. 10E , a diagram of an exemplary billing menu call flow is depicted in accordance with an alternative embodiment of the present disclosure. Process  1024  may be implemented within voice response system, such as voice response system unit  104  when calls are received from callers  102 . If the caller selects billing, process  1024  begins at step  1054  to prompt the caller to select making payments, deposits, or transfers. 
     If the caller selects transfers, process  1024  continues to step  1056  to connect the caller with an agent to make a transfer. If the caller selects payments, process  1024  returns to step  1012  to provide the caller with a payment menu. If the caller selects deposits, process  1024  continues to step  1058  to prompt the caller to select whether the caller wants to make an application or security deposit. If the caller selects application deposit, process  1024  continues to step  1060  to notify the caller of application deposit frequent asked questions (FAQ). If the caller selects security deposit, process  1024  continues to step  1062  to notify the caller of security deposit frequent asked questions (FAQ). Process  1024  then completes at step  1064  to prompt the caller for “main menu”, “repeat” or simply hang up. 
     Referring to  FIG. 10F , a diagram of an exemplary service updates menu call flow is depicted in accordance with an alternative embodiment of the present disclosure. Process  1014  may be implemented within voice response system, such as voice response system unit  104  when calls are received from callers  102 . If the caller selects billing, process  1014  begins at step  1060  to prompt the caller to select new service, transfer service, or disconnect service. 
     If the caller selects new service, process  1014  continues to step  1062  to prompt the caller for the type of new account. If the caller selects either residential or commercial account, process  1014  continues to step  1064  to connect the caller with an agent. If the callers selects either transfer service or disconnect service, process  1014  continues to steps  1066  and  1068  to connect the caller with an agent for such services. 
     Referring to  FIGS. 11A and 11B , diagrams illustrating an exemplary monitoring formula for a group of steps are depicted in accordance with an alternative embodiment of the present disclosure. In this example, monitoring formula comprises a set of metrics  1100 . In metrics  1100  as shown in  FIGS. 11A and 11B , nine criteria are defined to monitor caller experience for each step of the call flow. Similar to metrics  400  in  FIG. 4A , the nine criteria comprise negative behavior  1102 , caller discontent  1104 , dialog and menus  1106 , preference  1108 . As shown in  FIG. 11B , the nine criteria also comprise persona audio quality  1110 , navigation length  1112 , and prompt length  1114 . In addition, the nine criteria comprises two new criteria as shown in  FIG. 11A , recognition rate  1116  and out of grammar  1118 . 
     Recognition rate  1116  is examined based on the rate of voice recognition. The range for voice recognition rate is set to be greater than 92% for the green rating, from 71% to 91% for a yellow rating, and less than 70% for a red rating. The yellow rating of recognition rate  1116  is further subdivided into sub-ratings 2 to 4 with rating 2 ranging from 71% to 77%, rating 3 ranging from 78% to 84%, and rating 4 ranging from 85% to 91%. The higher the voice recognition rate, the better the caller experience. 
     Out of grammar  1118  is examined based on the percentage out of grammar utterances. The threshold for out of grammar  1118  is set to be 10% or less out of grammar utterances for the green rating, from 10% to 20% for a yellow rating, and greater than 20% for a red rating. The yellow rating of out of grammar  1118  is further subdivided into sub-ratings 2 to 4 with rating 2 ranging from 14% to 19%, rating 3 ranging from 10% to 13%, and rating 4 ranging from 6% to 9%. The lower the percentage out of grammar utterances, the better the caller experience. 
     Based on the nine criteria, an overall caller experience percentage is calculated from monitoring formula  1100 . In this example, an overall caller experience percentage of 90% or above achieves a green rating and represents great customer satisfaction, because the callers found this prompt helpful. An overall caller experience percentage of 66% to 89% achieves a yellow rating and represents serious questions in customer satisfaction, because the callers are confused. An overall caller experience percentage of 65% or below achieves a red rating and represents serious impact on overall customer satisfaction, because the callers are misguided. 
     Referring to  FIGS. 12A and 12B , diagrams illustrating an exemplary construction of a caller experience for each step of the payments sub-menu call flow is depicted in accordance with an alternative embodiment of the present disclosure. In metrics  1200 , the combined data for each criteria of payment sub-menu call flow step “payment menu” is listed under column  1202 . For example, the combined data for criteria “negative behavior” is 40%. A sub-rating  1204  is assigned for this criteria based on the range of values assigned to each sub-range. In this example, a sub-rating of 1 is assigned for the criteria “negative behavior” for a sub-rating ranging from 0 to 5. Because the combined data of 40% falls within this sub-rating, a sub-rating of 1 is assigned to this criterion. 
     After the sub-rating is assigned, a weight  1206  is calculated for criteria based on the previously-assigned value for each criteria and the sub-rating. In this example, a weight of 1 is calculated for negative behavior based on the full value that was previously assigned for this criterion and the sub-rating of 1. If a half value is assigned for this criterion, a sub-rating of 0.5 is calculated. After a weight is calculated for each criterion, a total weight is calculated by totaling all the calculated weights. In this example, the total weight of call flow step “payments menu” is 21. Once the total weight is obtained, a caller experience for this step is obtained by dividing the total weight by the number of calls that were processed through this call flow step. In this example, the number of calls that were processed through this step is 35. Therefore, the total weight of 21 is divided by 35 to obtain a caller experience of 0.6. Based on the caller experience of this step, a color rating may be assigned to this call flow step and an overall caller experience may be developed. 
     After the caller experience for each step of the payment sub-menu is determined, a number of steps for the payments sub-menus is grouped and the caller experience of these steps are presented for the group as a whole. As discussed above, in addition to displaying the caller experience of the steps of a call flow to service providers, aspects of the present disclosure is capable of displaying the caller experience of a group of steps to service providers based on their functions. 
     Referring to  FIG. 13 , a diagram illustrating an exemplary color-coded payment sub-menu flow chart is depicted in accordance with one embodiment of the present disclosure. Each step in process  1300  is color-coded based on the caller experience obtained using process  600  in  FIG. 6 . For example, if the caller experience is greater than 90%, the step is green color-coded. If the caller experience is from 66% to 89%, the step is yellow color-coded. If the caller experience is less than 65%, the step is red color-coded. 
     In  FIG. 13 , red color-coding is represented by crosshairs. Red color-coding means that the caller is misguided. Yellow color-coding is represented by vertical lines. Yellow color-coding means that the caller is confused. Green color-coding is represented by horizontal lines. Green color-coding means that the call flow step is helping the caller. In addition to red, yellow, and green color-codings, other indications may be implemented to represent different level of caller experience without departing the spirit and scope of the present disclosure. For example, an orange color-coding may be implemented to represent a caller experience that is more than confusing but less than misguiding the caller. 
     In this example, step  1326  is red color-coded, which means that this step misguided the callers and seriously impact caller experience and program effectiveness. Thus, it needs correction immediately. Steps  1338 ,  1340 ,  1342 , and  1346  are yellow color-coded, which means that these steps confused the callers. These steps raise serious questions on customer satisfaction. These steps also identify weak areas that require correction in a timely manner. Steps  1332 ,  1336 , and  1350  are green color-coded, which means that these steps are helping the callers. These steps provide great customer satisfaction and do not require modification. 
     By providing color-codings of a group of steps based on their functions, service providers may monitor caller experience at any level of the call flow and make corrections to only certain groups of steps as they see necessary. In this way, service providers may monitor and adjust to part or whole call flow based on their need. In addition, by identifying the specific groups of steps that present problems, service providers may update design of only those steps instead of the entire call flow. This helps service providers in improving program efficiency and effectiveness by reducing down time. 
     In addition to analyzing a group of steps based on their function and displaying the caller experience for those steps as a group, an aspect of the present disclosures provides the ability to determine the overall caller experience for a group of steps and present the overall caller experience of the group as a step in the overall call flow. 
     Referring to  FIG. 14 , a flow chart of a process for determining overall caller experience of a group of steps is depicted in accordance with one embodiment of the present disclosure. In one embodiment, process  1400  may be implemented as instructions embodied in a computer readable medium executing within caller experience monitoring system  126 . Process  1400  begins at step  1402  to identify a group of steps based on their functions, for example, a group of steps that perform payment functions. Next, process  1400  continues to step  1404  to identify caller experience of each step in the group. The caller experience of each step may be identified using process  600  as described  FIG. 6 . Process  1400  then continues to step  1406  to add caller experience of each step of the group to obtain a total caller experience for the whole group. For example, the weight of each step in  FIGS. 12A and 12B  may be added to obtain a total caller experience for the payment sub-menu. Returning to  FIG. 12A , the weight of call flow step “payment menu” is 21 while the weight of call flow step “bank draft return” is 18. Thus, the total caller experience is 39. 
     After a total caller experience is obtained, process  1400  continues to step  1408  to determine a total number of calls that were processed by the group as a whole. For example, referring to  FIG. 12A , call flow step “payment menu” has processed 35 calls while call flow step “bank draft return” has processed 25 calls. The total number of calls processed by the group if the group only consists of these two steps is 60. 
     Process  1400  then completes at step  1410  to divide the total caller experience by the total number of calls processed by the steps. Continuing with the above example, the total caller experience  39  is divided by the total number of calls  60  and an overall total caller experience of 65% is obtained. 
     Based on the color-codings as specified in metrics  110  in  FIG. 11B , a red color-coding is assigned to a total caller experience of less than 65%. As a result, an aspect of the present disclosure presents the total caller experience of the group as a color-coded step in the overall call flow. 
     Referring to  FIG. 15 , a diagram illustrating an exemplary color-coded payment menu flow chart is depicted in accordance with one embodiment of the present disclosure. Because the total caller experience for the payment sub-menu is red color-coded, payment menu  1522  is red color-coded (represented by crosshairs) in process  1500 , which means that this step misguided the callers and seriously impacted caller experience and program effectiveness. Thus, it needs correction immediately. By providing the total caller experience of a group and presenting the total caller experience as a step in the overall call flow, service providers may identify problem areas early in the call flow and make appropriate correction to the group. In addition, service providers may isolate certain steps of a group and perform analysis of those steps to determine their effect on the total caller experience. In this way, program effectiveness and efficiency may be improved without affecting operations of the overall call flow. 
     Although the above descriptions set forth preferred embodiments, it will be understood that there is no intent to limit the embodiment of the disclosure by such disclosure, but rather, it is intended to cover all modifications, substitutions, and alternate implementations falling within the spirit and scope of the embodiment of the disclosure. The embodiments are intended to cover capabilities and concepts whether they be via a loosely coupled set of components or they be converged into one or more integrated components, devices, circuits, and/or software programs.