BRAND PROXIMITY SCORE

Methods and systems for automatically assessing proximity of an enterprise's brand to a customer are disclosed. A processing device obtains a first sub-score indicative of a degree of completion of a task that involves the enterprise providing a service to the customer, a second sub-score indicative of a level of user engagement between the enterprise and the customer, and a third sub-score indicative of efficiency of the task that involves the enterprise providing the service to the customer. The processing device then combines the first sub-score, the second sub-score and the third sub-score to determine a composite brand proximity score (BPS) indicative of the proximity of the enterprise's brand to the customer.

TECHNICAL FIELD

Embodiments of the disclosure relate generally to task automation, and specifically to a score that reflects how proximal a brand of an enterprise is to a customer base.

BACKGROUND

Traditional customer surveys and scores created from those surveys involve enterprises explicitly asking the customers on the experience and/or how the customer will act in the future, such as recommending the enterprise's services or goods to their networks and friends. However, the traditional methods lack the ability to intelligently and automatically harness information from the customers over a period of time in a non-invasive way as the workflow progresses towards completion of a task.

SUMMARY

Enterprises leverage service engagement platforms (sometimes simply called a service platform, or user engagement platform) to interact with their customers. Service engagement platforms automate the enterprise workflow, interact with customers to broker information and build proximity with customers through conversations. The enterprise workflow is task-oriented. Examples of task may include booking a ticket, registering an account, resolving a claim, collecting user feedback etc.

The service engagement platform disclosed here may use various mechanisms such as chatbots, conversational artificial intelligence (AI) etc. for conversing with the customers while improving workflow efficiency. The service engagement platform automates at least parts of the enterprise workflow, interacts with customers to broker information and builds proximity with customers through conversations. The enterprise workflow is task-oriented. Examples of task may include booking a ticket, registering an account, resolving a claim, collecting user feedback etc.

The service engagement platform supports two-way text-based interaction centered around business-necessitated engagements between enterprises with their customers as they interact over a period of time. Note that the term ‘enterprise’ broadly encompasses an entity (which can be a business entity or a person) who serves a customer. The customer is sometimes referred to as ‘end-user’ or simply user, though based on the context, the term ‘user’ may also indicate the entity that is referred to as an enterprise elsewhere. Based on customer engagements and behaviors, the service engagement platform described here gradually derives a score that would convey how proximal the brand of the enterprise is to their customer base. This score is expected to be a standard of measurement for enterprises getting into a complete messaging-based interaction with their customers.

Generally speaking, a Brand Proximity Index (BPI) reflects how well an enterprise builds its relationship with its customer base over time. The BPI is computed from each individual customer interaction and each conversation flow has a brand proximity score (BPS).

The brand proximity score computation combines statistical processing and machine learning. Some of the important aspects that are taken into consideration are: overall task completion, the level of the customer's engagement and the efficiency of the system.

The measure of engagement is data-driven and uses the historical multi-turn conversational data to estimate the continuation to respond to each module. Multi-turn conversational modelling concatenates contextual utterances to ensure conversation consistency. The more effort it takes for a customer to respond, the higher the engagement score. A long text response from a customer will have a higher engagement score than a single click on a multiple choice tab. This is regardless of the content of the response such as a negative feedback. The level of engagement also incorporates the response time. Generally, short response time will have a higher engagement score than a lagged response time when all other conditions are the same. A task efficiency score will incorporate the system latency and heavily depend on whether the task is completed and the number of steps it took to complete. For a successfully completed task, the more steps it takes, the score will be slightly lower than those with fewer steps. For a task that is not completed, the task efficiency score significantly suffers and give score credits for each additional step it has completed so far.

Specifically, an aspect of the present disclosure describes methods and systems for automatically assessing proximity of an enterprise's brand to a customer. A processing device obtains a first sub-score indicative of a degree of completion of a task that involves the enterprise providing a service to the customer, a second sub-score indicative of a level of user engagement between the enterprise and the customer, and a third sub-score indicative of efficiency of the task that involves the enterprise providing the service to the customer. The processing device then combines the first sub-score, the second sub-score and the third sub-score to determine a composite brand proximity score (BPS) indicative of the proximity of the enterprise's brand to the customer.

Note that the term “service” is broadly interpreted to encompass providing information about or delivery of tangible goods too. Also, the term “user engagement” means engagement with a customer of the enterprise, where an “enterprise” can be an organization, or an individual, or a team of individuals that provide the service to the customer. Also, the term score is used generically, though when a score has multiple components, those multiple components can be indicated as sub-score.

DETAILED DESCRIPTION

Embodiments of the present disclosure are directed to determining a score indicative of how an enterprise's brand becomes proximal to a customer (or end-user) over time via progressive interactions.

FIG. 1illustrates an enterprise's workflow block diagram100, according to an embodiment of the present disclosure. The customer (also called user or end-user) interacts via the service platform110with the enterprise's task workflow112. The service platform110and task/workflow112together represent the enterprise's user engagement interface108. Each end-user (such as the three end-users102,104, and106shown here as an example, though any number of end-users can be supported) interaction and system execution records are logged and stored into the engagement record database114(e.g., a persistent database). The latest sequence of records are periodically retrieved (arrow marked 1) and brand proximity scores (BPS) are computed based on those interaction records. This operation is shown as engage metric computation116. The score is updated and inserted (arrow marked 2) in a separate database (BPI database118), where BPI is an abbreviation of Brand Proximity Index, elaborated below. The enterprise viewer sends a request to an engagement visualization engine120(as indicated by arrow marked 3) which sends a query (as indicated by the arrow marked 2 going to the BPI database118) to the BPI database and the scores are sent back to an enterprise view in a machine122(arrow marked 4).

FIG. 2illustrates a scoring engine layout200, according to an embodiment of the present disclosure. The records of engagement are fetched (shown as arrows marked 1) from a records database114and sorted by end-user session identifying indicia (abbreviated as session id). For example, records for engagement a with an end-user could be sorted as201having indivudual record components202,204and204. For each engagement, a sequence of machine and user interaction records are sorted in ascending timestamp order and then a copy of session engagement is passed through multiple scoring components (shown as arrows marked 2). For example, in an example where three scoring components are used, the individual components can be a completion score (208), an engagement score (210), and an efficiency score (212) are normalized (226) and generates a BPS score ‘BPS_a’. Similarly, engagement b (205) creates its own completion score214, engagement score216, and efficiency score218, which are normalized (228) to generate a BPS score ‘BPS_b’. Similarly, engagement c (206) creates its own completion score220, engagement score222, and efficiency score224, which are normalized (230) to generate a BPS score ‘BPS_c’. The individual BPS score for a, b, c are weighted (232) based on their relative importance and yield an overall brand proximity score (BPS) with the engagement at time t. This is explained further below.

The other engagement records that fall into the same time window (t, t+delta) are computed in the same manner and the BPS_a, BPS_b, BPS_c etc will be weighted based on the activeness of the end-users. The enterprise level BPS at time t is computed along with the BPI from last period time (i.e. at time (t−1)) as retrieved from BPI scoring database234. In one embodiment, the moving average of the last K periods can be used as one of the time series smoothing (236) method which yields a more robust estimate.

FIG. 3illustrates an engagement score component300A and an efficiency score component300B, according to embodiments of the present disclosure. The completion score component308created from the engagement record301gives a constant score to the engagement which is evaluated as completed. The flow is marked as completed if the interaction passed through a set of predefined workflow sections.

The engagement score component300A is based on two sub-components. The continuation score function indicated within the completion score component308gives a high score to a deeper engagement with a discounting factor309. The response time reward function within the response time score component311assigns a high score for short average user response time. Normalization327is applied to calculate the engagement score310.

The efficiency score component300B evaluates how well the workflow system handles the end-user's response and whether the primary objective is achieved or not. The response time for each system interaction is stored in an array in303. If the session is evaluated as completed at the decision block304, the response time array is padded with 0 (at reward padding block305), otherwise, the response time array is padded with ‘infinite’ (at penalization padding block306). The exponential score function takes the conceptual infinite and yields 0 score for that step. The temporary array which stores the system response time at each step is passed to the efficiency scoring component307, and then a weighted layer313which puts a higher weight on the critical interaction step. The prior probability for an end-user to continue at each step pj[u] can be used as weights. The efficiency score312is output as a result of these operations.

FIG. 4illustrates choosing a score function, according to an embodiment of the present disclosure. The choice of scoring function depends on the presence of a substantial amount of the historical dataset. With a variety of historical engagement records, the maximum likelihood estimate of the probability for a user to continue at one state is more data driven (shown as400B in the right half ofFIG. 4) than the human preference. However, the empirical expert scoring system (shown as400A in the left half ofFIG. 4) is a hands-on approach with a set of carefully crafted prior distributions. The choice of parameter of prior distribution (e.g., a statistical prior distribution for turns in a multi-turn conversation) represents the expert's belief on how the data distribution looks like (step 1). A score function can also be selected for continuation (step 2). The lookup scoring table (e.g., table409) also gives flexibility to compare different types of interaction and the application context. Examples of context can be ‘clicks’ on weblinks or SMS messages. The end goal for both400A and400B is to generate a score410from the engagement record402. But the data-driven system400B uses historical records404, a regressor component408to extract features, and a suitable prediction model406without the need for an expert's belief, i.e. the process is more automatic than a combination of manual and automatic.

FIG. 5is a table500showing how the various factors influence each score component. After running a set of experiments with the parameter chosen, the BPS score for each interaction depends on the following factors: task completion, interacting turn, step, invalid response, response time, and system latency. The table500inFIG. 5represents how sensitive the BPS score is (i.e. how the BPS score reacts) to the changes in these factors. For example, system latency negatively impacts only the task efficiency score, and not the task completion and user engagement scores. But the overall BPS score is still negatively impacted (as shown in the last row).

The brand proximity score (BPS) of one engagement is a linear combination of three sub-scores: Task Completion(I) sun-score, User Engagement Score (UES) and Task Efficiency Score(TES):

The corresponding weight α and β are scaling parameters and reflect the relative importance.

Iiis a binary variable. It measure whether the task is completed or not. If the conversational flow goes through one of the pre-defined success nodes or modules, the score is 1, else the score is 0. For example, in a credit-card payment section, last question section of a survey etc. indicates task completion.

UESiis designed to evaluate the level of engagement per session. The score takes two aspects into account: the steps the flow has gone through and the user response time at each step.

The system can set a probability function pj[u] to represent the prior belief of whether the user will continue at the step j. The pj[u] will be influenced by a few categorical variables. e.g. Ti>j−1 total turns larger than previous turn, Ninumber of finite user-defined steps and Cjtypes of response for turn j. The p4[u] is the expected value of a Bernoulli distribution conditioned on several variables.

pj[u]can be estimated by regression over a training dataset. If we don't have sufficient data to get a robust estimate, an alternative way to compute pj[u]uses a prior distribution, e.g. Poisson's distribution (with varying values of lambda λ), which reflect the expert's belief on the distribution of the engaging turns. This is shown by the set of plots600inFIG. 6. Later on, the parameter lambda can be reset by the mean of the posterior distribution.

In general, g0(p) is a decreasing score function which assigns a high score to a lower p value. As it is compared with all the other candidates, this engagement is pushed further.

The selection of a scoring function is not an exact science. For example, the drop out rate g0(p)=1−p is a possible candidate which is naturally bounded within (0, 1), beta(α=2, β=0.98) can also be used for non-linearity assumption.

The total score for the steps will be a summation of each individual response score,

where, γ is a discounting factor and njis the repetitive times for the same module due to validation. The discounting factor was introduced for repetitive validation engagement because some module requires strict input format (MM/DD/YYYY) etc. Those user responses demonstrate that the user continue to engage with the system but those activities will lead to unbounded scoring. The discounting factor will ensure the engaging score will have an upper bound.

In some embodiments, g1(ttrim) is an exponential score function used in the score calculation, which takes in a trimmed average of user engagement response time for all messages in one session.

O is set to 1 and tkis ordered by value. The general concept is that the quicker the average response time, the more engaged the user is.

Another type of function, g2(maxk(tk[u])) is a step function to give the memorizing reward for a user to return back to engage with the system without a reminder. As the user might be in a situation that he was distracted by something else, and later he remember to continue to engage with the system and finish the flow.

FIG. 7shows the plot700of g1which is the quick response reward function andFIG. 8shows the plot800of g2which is the memorizing reward step function.

To sum up, the user engagement score for one session i:

Tiis the total user responses for session i. For the task efficiency score, g3(tj[s]) is an exponential function takes in the system response time at step j and yield a score. The longer the system responded back, the lower the score. However the penalization for a lagged system response at different stages and modules will be different. The relative importance the module plays in the full workflow can be applied here with a proper parameter setting. Another way is to utilize the prior probability pj[u]the user continue at step j. When the conversation starts, the user has a higher chance to continually engage as they want to explore, a lagged system response at an earlier stage will raise the probability of discontinuation. The discontinuation at an earlier stage can be penalized with more weights according to the following equation:.

K is a fixed parameter set to be higher than the length of all the engagements. If it is a completed session, padded the rest of the array with K-Tjdefault system responses and set each response time to 0. If the task flow is not completed, the padded system response time will be set to infinite.

The BPS for an enterprise at time window t is an average of the user engagements at that time window.

the BPI (brand proximity index) at time t will be a combination of latest BPS and historical stock value.

FIG. 9is a flow diagram of an example high-level method900of BPS generation as implemented by a component operating in accordance with some embodiments of the present disclosure. The method900can be performed by processing logic that can include hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions run or executed on a processing device), or a combination thereof. In some embodiments, the method900is performed by the BPS calculation component1013shown inFIG. 10. Although shown in a particular sequence or order, unless otherwise specified, the order of the operations can be modified. Thus, the illustrated embodiments should be understood only as examples, and the illustrated operations can be performed in a different order, while some operations can be performed in parallel. Additionally, one or more operations can be omitted in some embodiments. Thus, not all illustrated operations are required in every embodiment, and other process flows are possible.

At operation910, the enterprise engages with a customer to whom the enterprise provides a service. Note that the term “service” is broadly interpreted to encompass providing information about or delivery of tangible goods too.

At operation920, a first sub-score is obtained, as described above, the first sub-score being indicative of a degree of completion of a task that involves the enterprise providing the service to the customer.

At operation930, a second sub-score is obtained, as described above, the second sub-score being indicative of a level of user engagement between the enterprise and the customer.

At operation940, a third sub-score is obtained, as described above, the third sub-score being indicative of efficiency of the task that involves the enterprise providing the service to the customer.

At operation950, the processing device combines the first, second and the third sub-scores to determine a composite BPS indicative of the proximity of the enterprise's brand to the customer.

FIG. 10illustrates an example machine of a computer system1000within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, can be executed. In some embodiments, the computer system1000can correspond to a host system that includes, is coupled to, or utilizes a memory sub-system or can be used to perform the operations of a processor (e.g., to execute an operating system to perform operations corresponding to a BPS generation, also referred to as BPS calculation component1013). In alternative embodiments, the machine can be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, and/or the Internet. The machine can operate in the capacity of a server or a client machine in client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

The example computer system1000includes a processing device1002, a main memory1004(e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory1008(e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system1018, which communicate with each other via a bus1030.

The data storage system1018can include a machine-readable storage medium1024(also known as a computer-readable medium) on which is stored one or more sets of instructions1028or software embodying any one or more of the methodologies or functions described herein. The instructions1028can also reside, completely or at least partially, within the main memory1004and/or within the processing device1002during execution thereof by the computer system1000, the main memory1004and the processing device1002also constituting machine-readable storage media. The machine-readable storage medium1024, data storage system1018, and/or main memory1004can correspond to a memory sub-system.