Abstract:
A data store (e.g., a relational database, an object-oriented database, an unstructured file system, an in-memory data structure, etc.) and associated methods are disclosed for predicting the accuracy of agent responses in a call center. In accordance with the illustrative embodiment, agents in a call center are prompted at different times during their workday for estimates of how much longer they will be servicing their current call, and the estimates are stored in the data store along with the actual times required for completion of the call. Statistics for each agent&#39;s prediction accuracy are computed and stored in the data store along with the raw data, and can then be used to assess the accuracy of subsequent agent responses to queries.

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
       [0001]    The present invention is a continuation-in-part of U.S. patent application Ser. No. 12/490,238, filed 23 Jun. 2009 (now pending), entitled “Servicing Calls in Call Centers Based On Estimated Call Value” (Attorney Docket: 630-451us), which is incorporated by reference. 
     
    
     FIELD OF THE INVENTION 
       [0002]    The present invention relates to call centers in general, and, more particularly, to a data store for predicting agent response accuracy. 
       BACKGROUND OF THE INVENTION 
       [0003]    A call center is a facility where telephone calls are handled by human operators known as agents, usually with some amount of computer automation. Typically, a call center has the ability to handle a considerable volume of calls, providing functions such as routing calls to agents, logging calls, recording calls, and so forth. Call centers are used by many mail-order catalog organizations, telemarketing companies, computer product help desks, government agencies, and large enterprises. 
         [0004]    Typically, when a call is received at a call center, the call is automatically routed to an agent who is not currently engaged in a conversation with another caller. If all agents are busy, typically the call is inserted into a queue and the caller is placed “on hold.” Callers on hold are typically played a message such as the following: “All agents are currently assisting other customers. Please stay on the line and the next available agent will take your call. Calls are answered in the order that they are received”. 
       SUMMARY OF THE INVENTION 
       [0005]    The present invention comprises a data store (e.g., a relational database, an object-oriented database, an unstructured file system, an in-memory data structure, etc.) and associated methods for predicting the accuracy of agent responses in a call center. In particular, in accordance with the illustrative embodiment, agents in a call center are prompted at different times during their workday for estimates of how much longer they will be servicing their current call, and the estimates are stored in the data store along with the actual times required for completion of the call. Statistics for each agent&#39;s prediction accuracy are computed and stored in the data store along with the raw data, and can then be used to assess the accuracy of subsequent agent responses to queries. In accordance with the illustrative embodiment, for example, the data store is used to assess the accuracy of subsequent agent estimates of remaining service times, and these assessments are used in determining how high-value incoming calls are routed to agents in the call center. 
         [0006]    The illustrative embodiment comprises a data store comprising: a first value that is based on an estimate of how much longer an agent of a call center will be serving a call, wherein the estimate is provided by the agent during the call; and a second value that is based on the actual remaining time required for the agent to service the call. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0007]      FIG. 1  depicts the salient elements of a call center, in accordance with the illustrative embodiment of the present invention. 
           [0008]      FIG. 2  depicts the contents of data store  120 , as shown in  FIG. 1 , in accordance with the illustrative embodiment of the present invention. 
           [0009]      FIG. 3  depicts a data schema for call table  200  and agent accuracy table  210 , as shown in  FIG. 2 , in accordance with the illustrative embodiment of the present invention. 
           [0010]      FIG. 4  depicts a flowchart of the salient tasks of a method for obtaining and storing information in call table  200 , in accordance with the illustrative embodiment of the present invention. 
           [0011]      FIG. 5  depicts a flowchart of the salient tasks of a method for computing and storing information in accuracy table  210 , in accordance with the illustrative embodiment of the present invention. 
           [0012]      FIG. 6  depicts a flowchart of the salient tasks of a method of handling calls at call center  100 , as shown in  FIG. 1 , in accordance with the illustrative embodiment of the present invention. 
       
    
    
     DETAILED DESCRIPTION 
       [0013]    For the purposes of this specification, a “call” is defined to encompass all types of communications including a traditional voice telephone call, a videophone call, an email message, a text-based instant message, etc. 
         [0014]      FIG. 1  depicts the salient elements of call center  100 , in accordance with the illustrative embodiment of the present invention. As shown in  FIG. 1 , call center  100  comprises agent data-processing systems  110 - 1  through  110 -N, where N is a positive integer, and data store  120 . 
         [0015]    Each data-processing system  110 - i,  where i is an integer between 1 and N inclusive, is one of a personal computer, notebook computer, smartphone, etc. that is associated with a respective agent in call center  100 . In accordance with the illustrative embodiment, each data-processing system  110 - i  is capable of generating and directing output to an agent (e.g., a message prompting an agent for information during a call, etc.), of receiving input from an agent, of storing information in data store  120 , and of retrieving information from data store  120  (e.g., via the submission of queries, etc.). As will be appreciated by those skilled in the art, in some embodiments of the present invention data-processing system  110 - i  might also be capable of performing other tasks, such as monitoring the performance of an agent during calls, executing one or more software applications that might be used by an agent (e.g., an order-entry application, an inventory application, etc.), and so forth. 
         [0016]    Data store  120  comprises memory (e.g., disk-based storage, random-access memory, flash memory, etc.) that is capable of storing information concerning calls to call center  100 , and concerning agents&#39; estimates of call service times, as described in detail below and with respect to  FIGS. 2 and 3 . In accordance with the illustrative embodiment, data store  120  is a relational database; however, it will be clear to those skilled in the art, after reading this disclosure, how to make and use other types of embodiments of data store  120  (e.g., an object-oriented database, an unstructured file system, an in-memory data structure, etc.). 
         [0017]    The Data Store 
         [0018]      FIG. 2  depicts the contents of data store  120 , in accordance with the illustrative embodiment of the present invention. As shown in  FIG. 2 , data store  120  comprises call table  200  and agent accuracy table  210 , the organization and contents of which are described in detail below and with respect to  FIG. 3 . 
         [0019]      FIG. 3  depicts a data schema for call table  200  and agent accuracy table  210 , in accordance with the illustrative embodiment of the present invention. Each row in call table  200  corresponds to a call to call center  100 , and each row in agent accuracy table  210  corresponds to an agent of call center  100 . As shown in  FIG. 3 , table  200  comprises columns  301  through  309 , and table  210  comprises columns  321  through  324 . 
         [0020]    Column  301  stores an indicium of a call (e.g., a call serial number, etc.) and serves as a key of call table  200 . 
         [0021]    Column  302  stores an indicium of the agent that serviced the call (e.g., an agent identification number, etc.). 
         [0022]    Column  303  stores an indicium of the caller (e.g., a customer identification number, the phone number of the caller, etc.). 
         [0023]    Column  304  stores an indication of which work stage the call was in (e.g., opening salutation stage, order placement, order confirmation, etc.) when the agent provides an estimate of how much longer the agent will be servicing the call (i.e., the remaining service time for the call). 
         [0024]    Column  305  stores an indication of one or more capabilities required by the agent to service the call (e.g., the ability to authorize refunds, the ability to access a particular software application, etc.). In accordance with the illustrative embodiment, an entry of column  305  might be empty, indicating that no special capabilities were required for the call. 
         [0025]    Column  306  stores an indicium of one or more transactions that occurred during the call (e.g., a purchase order, etc.). In accordance with the illustrative embodiment, an entry of column  305  might be empty, indicating that no transactions occurred during the call. 
         [0026]    Column  307  stores an estimate, provided by the agent, of the remaining service time for the call. In accordance with the illustrative embodiment, an estimate might be a single value, or might be a range (e.g., a minimum-maximum range, a 90% confidence interval, etc.). 
         [0027]    Column  308  stores the time at which the agent provided the estimate of column  307 . 
         [0028]    Column  309  stores the actual remaining service time for the call (i.e., how much time was actually required for the agent to complete servicing of the call, measured from the time that the agent provided his or her estimate). 
         [0029]    Column  310  stores the difference between the actual and estimated remaining service times for the call. As will be appreciated by those skilled in the art, in some embodiments of the present invention, when the estimate is a range, the difference might be a single value (e.g., the difference between the actual time and the midpoint of the estimated range, etc.), while in some other embodiments, the difference might also be represented by a range. 
         [0030]    Column  311  stores information about any transfer(s) during the call (e.g., the time at which a transfer was made, the agent to whom the call was transferred, etc.). 
         [0031]    Column  312  stores the number of times, if any, that the caller was placed on hold during the call. 
         [0032]    Column  313  stores how long the caller was on hold during the call. As will be appreciated by those skilled in the art, in some embodiments, column  312  might store a single value that indicates that the total time that the caller was on hold, while in some other embodiments, column  313  might store a plurality of values for each time the caller was placed on hold. 
         [0033]    Column  314  stores the of time at which the call ends. 
         [0034]    Column  321  stores an indicium of an agent of call center  100  (e.g., an agent identification number, etc.) and serves as a key of agent accuracy table  210 . 
         [0035]    Column  322  stores the mean of the actual-versus-estimated time differences for each agent (i.e., the mean of the entries of column  309  for each particular agent). 
         [0036]    Column  323  stores the standard deviation of the actual-versus-estimated time differences for each agent. 
         [0037]    Column  324  stores one or more additional accuracy measures for each agent (e.g., the maximum actual-versus-estimated time difference, a high-order moment of the time differences, etc.). 
         [0038]    As will be appreciated by those skilled in the art, some other embodiments of the present invention might omit one or more data of the illustrative embodiment, or might store other data in addition to those of the illustrative embodiment, and it will be clear to those skilled in the art, after reading this disclosure, how to make and use such embodiments. As will further be appreciated by those skilled in the art, some other embodiments of the present invention might employ an alternative schema for storing and organizing information in data store  120 , and it will be clear to those skilled in the art, after reading this disclosure, how to make and use such embodiments. 
         [0039]    Building the Data Store 
         [0040]      FIG. 4  depicts a flowchart of the salient tasks of a method for obtaining and storing information in call table  200 , in accordance with the illustrative embodiment of the present invention. In accordance with the illustrative embodiment of the present invention, the tasks of the method of  FIG. 4  are performed by a data-processing system of call center  100  not depicted in  FIG. 1  (e.g., a server, a router, etc.). As will be appreciated by those skilled in the art, in some embodiments of the present invention, this data-processing system might also handle other functions of call center  100  (e.g., queue management, application logic, interactive voice response [IVR] system functions, etc.), while in some other embodiments of the present invention, this data-processing system might be dedicated to performing the tasks of the method of  FIG. 4 . Moreover, it will be clear to those skilled in the art, after reading this disclosure, which tasks depicted in  FIG. 4  can be performed simultaneously or in a different order than that depicted. 
         [0041]    At task  410 , a signal is transmitted to one or more agents of call center  100  that prompts the agent for an estimate of how much longer the agent will be serving his or her current call. As will be appreciated by those skilled in the art, in some embodiments of the present invention, the prompting signal might be transmitted to all of the agents of call center  100  every time task  410  is executed in the loop of  FIG. 4 , while in some other embodiments, the agents of call center  100  might be “cycled through,” one at a time, at each iteration of the loop, while in still some other embodiments, one or more agents of call center  100  might be selected at random at each iteration of the loop, while in yet other embodiments, some other scheme might be employed for determining to which agent(s) the prompting signal is transmitted. 
         [0042]    In accordance with the illustrative embodiment, the prompt is conveyed to an agent via a video display terminal; however, as will be appreciated by those skilled in the art, in some other embodiments of the present invention, the prompt might be conveyed to an agent in some other manner (e.g., via audio output to a wireless headset, via audio output to a speaker connected to a data-processing system, etc.). As will further be appreciated by those skilled in the art, in some embodiments of the present invention the prompt might enable an agent to provide a range (e.g., a minimum-maximum range, a 90% confidence interval, etc.), while in some other embodiments of the present invention, the prompt might allow the agent to specify only a single value. 
         [0043]    At task  420 , estimates are received from the agents, in well-known fashion. As will be appreciated by those skilled in the art, in some embodiments of the present invention the agents might specify their responses via a graphical user interface of a data-processing system, while in some other embodiments of the present invention, the agents might specify their responses in some other fashion (e.g., via a microphone connected to a data-processing system, etc.) 
         [0044]    At task  430 , a new row is created in call table  200  for each agent prompted at task  410 . 
         [0045]    At task  440 , columns  301  through  314  are populated accordingly for each of the new rows created in task  430 . As will be appreciated by those skilled in the art, the information required to populate columns  305  and  306  might come from a variety of sources, such as a transaction server, a call queue management server, and so forth. 
         [0046]    At task  450 , a signal is received that indicates that a prompted agent A has completed servicing his or her current call, and specifies the time at which the call was completed. 
         [0047]    At task  460 , if the caller was in fact connected to agent A, columns  305  and  306  and columns  309  through  314  of the row corresponding to agent A&#39;s call are populated accordingly. As will be appreciated by those skilled in the art, the contents of columns  305  and  306  might be populated or updated based on information from a transaction server, a call queue management server, etc. (for example, a transaction might have been initiated during a call after the agent provided the estimate, in which case the contents of column  306  would be updated accordingly). 
         [0048]    Task  470  checks whether all of the prompted agents&#39; calls have been completed. If so, execution proceeds to task  480 , otherwise execution continues back at task  450 . 
         [0049]    Task  480  waits for a time interval. As will be appreciated by those skilled in the art, in some embodiments of the present invention the length of this time interval might be determined randomly each time task  480  is executed, while in some other embodiments, the waiting time interval might always be the same pre-determined length, while in still some other embodiments, the length of the waiting time might be determined in some other fashion. After task  480 , the method of  FIG. 4  continues back at task  410 . 
         [0050]      FIG. 5  depicts a flowchart of the salient tasks of a method for computing and storing information in accuracy table  210 , in accordance with the illustrative embodiment of the present invention. In accordance with the illustrative embodiment of the present invention, the tasks of the method of  FIG. 5  are performed by a data-processing system of call center  100  not depicted in  FIG. 1  (e.g., a server, a router, etc.). As will be appreciated by those skilled in the art, in some embodiments of the present invention, this data-processing system might also handle other functions (e.g., the method of  FIG. 4 , queue management, application logic, interactive voice response [IVR] system functions, etc.), while in some other embodiments of the present invention, this data-processing system might be dedicated to performing the tasks of the method of  FIG. 5 . Moreover, it will be clear to those skilled in the art, after reading this disclosure, which tasks depicted in  FIG. 5  can be performed simultaneously or in a different order than that depicted. 
         [0051]    At task  510 , variable S is initialized to the set of all agents of call center  100 . 
         [0052]    At task  520 , variable A is set to an agent of set S. As will be appreciated by those skilled in the art, in some embodiments an agent of set S might be selected randomly, while in some other embodiments an agent might be selected in some other fashion (e.g., selecting the agent of set S with the smallest agent identification number, etc.). 
         [0053]    At task  530 , a new row R is created in agent accuracy table  210 , and an indicium of agent A is stored in column  321  of row R. 
         [0054]    At task  540 , a query is executed that selects all rows of call table  200  for which column  302  matches agent A, and the result set of the query is stored in variable S′. 
         [0055]    At task  550 , the mean of the values of column  310  in result set S′ is computed and stored in column  322  of row R. 
         [0056]    At task  560 , the standard deviation of the values of column  310  in result set S′ is computed and stored in column  323  of row R. 
         [0057]    At task  570 , one or more additional accuracy measures (e.g., maximum difference, a high-order moment, etc.) are computed based on the values of column  310  in result set S′, and are stored in column  323  of row R. 
         [0058]    At task  580 , A is removed from set S. 
         [0059]    Task  590  checks whether set S is empty. If not, execution continues back at task  520 , otherwise the method of  FIG. 5  terminates. 
         [0060]    Using the Data Store to Predict Accuracy of Agent Responses 
         [0061]    Once data store  120  has been built, it can be used to predict the accuracy of agent responses. In particular, in accordance with the illustrative embodiment, data store  120  is used to predict the accuracy of agent responses in the handling of calls at call center  100 , as described below and with respect to  FIG. 6 . 
         [0062]      FIG. 6  depicts a flowchart of the salient tasks of a method of handling calls at call center  100 , in accordance with the illustrative embodiment of the present invention. In accordance with the illustrative embodiment of the present invention, the tasks of the method of  FIG. 6  are performed by a data-processing system of call center  100  not depicted in  FIG. 1  (e.g., a server, a router, etc.). As will be appreciated by those skilled in the art, in some embodiments of the present invention, this data-processing system might also handle other functions of call center  100  (e.g., queue management, application logic, interactive voice response [IVR] system functions, etc.), while in some other embodiments of the present invention, this data-processing system might be dedicated to performing the tasks of the method of  FIG. 6 . Moreover, it will be clear to those skilled in the art, after reading this disclosure, which tasks depicted in  FIG. 6  can be performed simultaneously or in a different order than that depicted. 
         [0063]    At task  610 , one or more signals are received that indicate that: (i) a call C to call center  100  is queued for service, and (ii) there is a non-empty set S of agents who are currently serving calls that are estimated to be less valuable than call C; and the incident is stored in data store  120 . As will be appreciated by those skilled in the art, the value of a queued call might be estimated in a variety of ways (e.g., based on the identity of the caller, etc.); similarly, the value of a call that is currently being serviced might be estimated in a variety of ways (e.g., based on the identity of the caller, based on the dollar amount of merchandise already ordered during the call, etc.) As will further be appreciated by those skilled in the art, in some embodiments of the present invention, the data-processing system performing the tasks of  FIG. 6  might be responsible for estimating the value of queued calls and currently-served calls, while in some other embodiments of the present invention, one or more other data-processing systems of call center  100  not depicted in  FIG. 1  (e.g., a queue management server, a billing application server, etc.) might handle these functions. 
         [0064]    At task  620 , a signal is transmitted to each of the agents of set S that:
       (i) notifies the agent of call C,   (ii) prompts the agent for an estimate of how much longer the agent will be serving his or her current call,   (iii) optionally indicates that the agent should finish servicing his or her current call as quickly as possible, and   (iv) indicates which agents have been notified.       
 
         [0069]    In accordance with the illustrative embodiment, the notification, prompt, and indication are conveyed to the agent via a video display terminal (e.g., a “dumb” terminal, a monitor connected to a data-processing system, etc.). As will be appreciated by those skilled in the art, in some other embodiments of the present invention, some or all of the notification, prompt, and indication might be conveyed to the agent in some other manner (e.g., via audio output to a wireless headset, via audio output to a speaker connected to a data-processing system, etc.). 
         [0070]    As will further be appreciated by those skilled in the art, for embodiments in which the indication (i.e., option (iii)) is omitted, the notification and prompt might implicitly have the effect of encouraging the agent to wrap up his or her current call as quickly as possible. As will yet further be appreciated by those skilled in the art, in some embodiments of the present invention the prompt might enable an agent to provide a range (e.g., a minimum-maximum range, a 90% confidence interval, etc.), or might enable an agent to decline servicing call C (in which case the agent is removed from consideration as a potential candidate to service call C.) 
         [0071]    At task  630 , responses from the agents of set S are received and logged in data store  120 , in well-known fashion. As will be appreciated by those skilled in the art, in some embodiments of the present invention the agents might specify their responses via a graphical user interface of a data-processing system, while in some other embodiments of the present invention, the agents might specify their responses in some other fashion (e.g., via a microphone connected to a data-processing system, etc.) 
         [0072]    Task  640  branches based on whether at least one of the responses received at task  630  is acceptable (e.g., whether there is a response that specifies a maximum waiting time below a given threshold, etc.). If so, execution continues at task  660 , otherwise execution proceeds to task  650 . 
         [0073]    At task  650 , a signal is transmitted to each of the agents of set S that:
       (i) indicates that none of the agent responses were acceptable, and   (ii) prompts the agent for another estimate of how much longer the agent will be serving his or her current call.
 
After task  650 , execution continues back at task  630 .
       
 
         [0076]    At task  660 , one of the agents of set S is selected to service call C, based on:
       (i) the agent responses received at task  630 , and   (ii) the accuracy statistics stored in table  210  of data store  120  for each of the agents of set S;
 
and the selected agent is logged in data store  120 . As will be appreciated by those skilled in the art, the accuracy statistics in table  210  might be used in a variety of ways at task  660 ; for example, in some embodiments of the present invention, an agent&#39;s response might be increased or decreased in accordance with the mean difference (i.e., column  322 ) stored in table  210 , while in some other embodiments of the present invention, an agent&#39;s response might be adjusted based on a combination of the accuracy measures of table  210 . In still other embodiments of the present invention, an agent might be removed from consideration outright if the agent&#39;s historical errors exceed a particular threshold.
       
 
         [0079]    At task  665 , a signal is transmitted to the selected agent that notifies the agent that he or she has been selected to service call C. 
         [0080]    At task  670 , a signal is transmitted to the agents of set S that were not selected; this signal notifies the agents that another agent will be servicing call C. 
         [0081]    At task  675 , a signal is received that indicates that an agent A of set S (either the selected agent or an agent who was not selected) has completed servicing his or her current call, and is now available. 
         [0082]    At task  680 , the following data are written to data store  120 : the estimate provided by agent A, the actual remaining time of agent A&#39;s current call, whether the caller was connected to an agent, and whether the time and decision criteria were acceptable. 
         [0083]    Task  685  branches based on whether agent A is the agent selected to service call C. If so, execution proceeds to task  690 , otherwise execution continues at task  695 . 
         [0084]    At task  690 , call C is routed to the selected agent A, in well-known fashion. 
         [0085]    Task  695  branches based on whether all agents of set S have completed their calls. If not, execution continues back at task  675 , otherwise the method of  FIG. 6  terminates. 
         [0086]    As will be appreciated by those skilled in the art, in some embodiments of the present invention it might be advantageous to adjust the timing and repetition of the loop of tasks  630  through  650  (i.e., pinging the agents and assessing their responses) so that agents are not overly “bombarded” and are given sufficient time to respond to prompts. As will further be appreciated by those skilled in the art, in some embodiments of the present invention it might be advantageous to change the threshold of what constitutes “acceptable” during subsequent iterations of this loop. For such embodiments, it will be clear to those skilled in the art, after reading this disclosure, how to modify the method of  FIG. 6  accordingly. 
         [0087]    As will further be appreciated by those skilled in the art, in some embodiments of the present invention, data store  120  might be used to predict the accuracy of agent responses in other tasks and contexts. For example, in some embodiments, agent response accuracy metrics for in data store  120  might be used in the establishment of agent schedules at call center  100 . Alternatively, in some other embodiments of the present invention, agent response accuracy metrics in data store  120  might be used in tasks where agents are asked to predict other types of information, perhaps even unrelated to remaining service time. For example, some embodiments might use agent response accuracy metrics in data store  120  to assess an agent&#39;s ability to accurately predict a caller&#39;s degree of satisfaction with the agent&#39;s performance during a call (e.g., a “satisfaction score” from 1 to 10, etc.). As another example, some embodiments might use agent response accuracy metrics in data store  120  to assess a trouble-ticket agent&#39;s ability to accurately predict when he or she will get back to a customer (e.g., how long a technician will need to troubleshoot a problem before the customer can be called back with an answer, how long a claim clerk will need to look at something before the customer can be called back with appropriate advice, etc.). 
         [0088]    As will yet further be appreciated by those skilled in the art, in some embodiments in which information unrelated to—or only partially related to—remaining service time is solicited from agents, data store  120  might explicitly store the agents&#39; responses and associated information. For example, in embodiments in which data store  120  is used to assess an agent&#39;s ability to accurately predict a caller&#39;s satisfaction score, data store  120  might contain agent predictions and actual satisfaction scores for previous calls, as well as statistics based on these data (e.g., mean difference, maximum difference, etc.) as in the illustrative embodiment. 
         [0089]    It is to be understood that the disclosure teaches just one example of the illustrative embodiment and that many variations of the invention can easily be devised by those skilled in the art after reading this disclosure and that the scope of the present invention is to be determined by the following claims.