Hierarchical interface for adaptive closed loop communication system

A communication system for processing a call includes control logic and at least one machine learning model generating call classifiers from outputs of an audio signal processor and a natural language processor operated on the call. Heuristic logic transforms the call classifiers into weighted sub-metrics for the call, and aggregate normalized Gaussian logic transforms the weighted sub-metrics into a metric control that may be applied as a feedback signal to adapt the operation of the control logic. The control logic in turn may adapt the behavior of an agent, automated voice attendant, or a template utilized in a call flow. The system includes a scorecard interface operable to select a target and an indication of the metric control to apply for the target, and to apply the metric control to generate and display a historical performance visualization and a performance feed of the metric for the target.

BACKGROUND

Conventional communication system metric controls include quality assurance (QA) metrics, customer satisfaction metrics (CSAT), and the net promoter score (NPS). These metric controls share two drawbacks when applied as feedback to adapt the system: rarity and uniformity. Conventional metric controls may rely on a random and/or low-frequency sampling of inputs from different processing agents in the system. This may result in a sparse signal for adapting agent and system behavior.

It is thus advantageous to generate adaptive controls from a greater percentage (or from all) inputs, and therefore provide a more responsive and precise feedback control for communication systems. Traditional metric controls may be noisy and bimodal, often limiting the usefulness and accuracy of singular measurements.

Call centers are increasingly utilized by organization for many reasons related to servicing customer inquiries and problems. Call centers are a key point of contact between large organizations and their customers, and therefor organizations are increasingly concerned with the quality of service provided to callers and to efficiency of call center operations.

A growing area of interest by organizations is therefor the improvement of call center service and efficiency, including the efficient and accurate allocation of resources such as agents to calls.

Many call systems have sparse data about as to which calls went well across their tracked metrics. Common metrics include quality metrics, such as was a successful outcome reached, was the call agent competent, etc., and matching human survey results, such as quality assurance audits, customer survey results, etc. In general, to determine these metrics, humans listen to and manually provide feedback, which may be cost ineffective and time consuming.

DETAILED DESCRIPTION

Embodiments of a communication system are disclosed utilizing metric controls generated using a combination of audio signal processing, natural language processor (NLP) transcription, machine learning models, and heuristic algorithms. The metric controls may be applied to adapt the system toward higher efficiency and accuracy when processing calls. The system may generate aggregate metric controls in the form of an automated ensemble of programmatic statistical models. The system provides adaptive feedback responsive to more and more frequent inputs than do conventional communication systems, so that corrective action may be applied for exceptional situations and so that processing agents and components operative in the system receive a continuous adaptive feedback control that enables more rapid correction and improvement of call processing. The system may provide more stable metric controls to more accurately compare performance between system agents, components, and/or groups and combinations thereof.

In another aspect the system may include global configuration settings for controlling a reference time frame for time series graphs of various operational metrics that are plotted against historical values on a corresponding time frame. Exemplary time frame settings may in one embodiment include:YesterdayA week agoThis day a month agoAverage for a particular day of the weekAverage for any day of the weekWhether to utilize raw values for metrics or utilize change rates for the metrics for control, reports, user interfaces, and visualizations.

In some aspects, a communication system for processing a call includes control logic and at least one machine learning model generating call classifiers from outputs of an audio signal processor and a natural language processor operated on the call. Heuristic logic transforms the call classifiers into weighted sub-metrics for the call, and aggregate normalized Gaussian logic transforms the weighted sub-metrics into a metric control that may be applied as a feedback signal to adapt the operation of the control logic. The control logic in turn may adapt the behavior of the automated voice attendant or a template utilized in a call flow.

In other aspects, an alert generator in a communication system for processing a call includes at least one machine learning model generating call classifiers from outputs of an audio signal processor and a natural language processor configure to operate on the call. Heuristic logic is configured to transform the call classifiers into a plurality of weighted sub-metrics for the call, and aggregate normalized Gaussian logic is configured to transform the weighted sub-metrics into a metric control. A threshold analyzer is configured to generate an alert signal to the communication system based on the metric control meeting a condition.

In other aspects, the alert generator includes an anomaly detector configured to identify anomalous calls. In some embodiments, the alert signal configures the communication system for priority response to the condition. In some embodiments, the alert signal is associated with portions of the call comprising content that contributed to activation of the alert signal. The call may be an active call or a recorded call. In some embodiments, the alert generator incorporates a learning function utilizing a call history and one or more of the weighted sub-metrics and the metric control.

In other aspects, a communication system for processing a call includes a scorecard user interface display, at least one machine learning model generating call classifiers from outputs of an audio signal processor and a natural language processor configured to operate on the call, heuristic logic configured to transform the call classifiers into a plurality of weighted sub-metrics for the call, and aggregate normalized Gaussian logic to transform the weighted sub-metrics into a metric control, the metric control applied as feedback to adapt control logic of the communication system. The scorecard interface is operable to select a target and an indication of the metric control to apply for the target, and to apply the metric control to generate and display a historical performance visualization and a performance feed of the metric for the target.

Disclosed herein are embodiments of a call flow manager that may be utilized in conjunction with aspects of said communication systems. In some aspects, the call flow manager includes a graph of connected nodes implementing a call center routing flow, and corresponding views for examining and modifying the nodes and graph. The call flow manager implements a set of fundamental node types, with complexity and behavior encapsulated within the nodes. Routes in the call flow, and hence the graphs, may be cyclic. Call flow is primarily defined by adding and removing child nodes to and from existing nodes.

Calls may be tagged with routing tags, either by outcomes determined by bot nodes, by a call classification system, or by agents. An exemplary routing tag is LANGUAGE: SPANISH for callers that speak Spanish. Agents may be tagged with agent tags. An exemplary agent tag is STATE: NEW MEXICO CAR INSURANCE for an agent qualified to handle New Mexico car insurance calls. Queues may also have tags. When a caller enters a queue, they temporarily receive all the tags of the queue. For instance, a queue may be tagged NEW ACCOUNT: TRUE if the caller has just created their account. However, once the call is routed out of the queue, the call loses that tag. In one embodiment, there is a default tag called QUEUE: <QUEUE-NAME> that is always applied.

In one embodiment, tags may have one or multiple values and a single key. Every call, agent, and queue may have zero, one, or multiple values selected. For instance, an agent may speak English and Spanish. In some cases only one or no value may be set for a given key.

If an agent or call has multiple values set for a tag key, then any one property may be sufficient for a match. For example, if an agent has LANGUAGE: SPANISH and LANGUAGE: ENGLISH values set, they may handle either Spanish or English calls. If a caller has INTENT: NEW POLICY and INTENT: REMOVE ACCOUNT set, an agent with either tag may take their call (and perhaps later clear that tag). In one embodiment, if no active agents (including busy agents) have all the tags needed to handle a particular call, the caller must be moved from their current queue to a failsafe route. If the failsafe route comprises a queue, the queue becomes a failsafe queue. Failsafe queues have the property that the call is stripped of all its tags (for routing purposes) and only retains the tags of the new queue. This increases the chances that the call is no longer over-constrained, but still enables control over which agents are permitted to handle failsafe calls.

The exemplary embodiments enable a routing management solution that reduces the complexity and management overhead of enterprise ACD systems. The exemplary embodiments may utilize a combination of heuristics and machine learning to match agents based on their historical performance on similar calls.

In one embodiment, if a call is rerouted by an agent, three options are:1. The agent de-matches with the caller, and the caller is re-matched to any other agent based on match score.2. The agent routes the caller to another node (e.g., an outlet node) or agent. The agent adds a new “hard” call tag that compels the new node or new agent to service the call.3. The agent routes the caller to another queue node again with a hard tag.

In the case that hard tags over-constrain matching, the caller may be routed to a failsafe queue node.

A routing history for a call may be generated and stored for later analysis, including 1) routing decisions, and why an agent and caller were matched, and 2) a list of agents and their performance scores for each tag, and for tags that lack sufficient coverage.

Examples of tag-based call routing include but are not limited to the processing of two types of calls, gold and bronze, utilizing two types of agents, gold and bronze. Gold agents can support gold and bronze calls. Bronze agents only support bronze calls. Calls of both types are routed into a single queue. Calls are tagged SUPPORT LEVEL: GOLD or SUPPORT LEVEL: BRONZE. Some agents are tagged with both SUPPORT LEVEL: GOLD and SUPPORT LEVEL: BRONZE. Other agents are only tagged SUPPORT LEVEL: BRONZE. The bootstrap score matches explicit agent tags for support level and callers that are escalated to the same support level. Over time, if the S model is enabled, some agents in the lower, bronze support level may occasionally need to handle gold calls. If they perform comparably to gold agents, they become de facto promoted to gold. If this behavior is undesirable then the user has the option to slide the S model control (in the exemplary embodiments utilizing slider controls) to an acceptably low level.

In some embodiments, a call center creation application may be invoked to configure new call flows or modify existing call flows. One example of actions to create a new call flow is:Name the call flowPick a first inbound phone numberIdentify type of call flow to start?Bot onlyBot with recorders and one outletBot and queues with surveyDispatcher with heavy fan-outEmptyIdentify next actionSetup call flow nodesAdd new nodes to flowTest call flow and add agentsInvoke the call flow manager

In some embodiments a flat list of nodes in a call flow may be provided, that can be filtered by type or by a search query. The list provides the raw settings of each node so that properties like outbound routes or bots may be enumerated and viewed.

In some embodiments, a system includes a communication interface configured to receive a call from a telephony carrier network. The system may be configured with a call flow between the communication interface and one or more of an outlet node and a call hangup node. The call flow may include an inlet node binding a communication address to one or more queue nodes, with at least one of the queue nodes coupled to one of the outlet node and the call hangup node. In one aspect, the call flow also includes at least one bot node. The bot node may configured to route the call to particular child nodes of the bot node according to particular outcomes of the bot node, and to apply tags to the call and route the call to the child nodes based on the applied tags.

In some embodiments, each queue node may include a state forwarding switch to enable or disable the propagation of state information from the queue node to a next node in the call flow, wherein the state includes tags placed on the call at the queue node. The queue node may be configured to perform state forwarding upon one or both of entry to the queue node and exit from the queue node and may be configured with configured with a state forwarding outlet type comprising one of HTTP GET, HTTP POST, email, and SMS, for example. The outlet node may be configured in some embodiments to operate a proxy to continue generating call analytics after routing of the call from the outlet node.

In some embodiments, the queue node may be configured with a priority and/or with tags associating the queue node with properties of a human agent, and/or with a failsafe child node, and/or with a control balancing the contributions of first-in, first-out priority and matching of the call to agent attributes to a service priority of calls in the queue node. In some embodiments, the control may be slide-configurable, and may set parameters α and β of the service priority algorithm set forth in Equation 1. The S model of the service priority algorithm may implement a machine learning model.

In some embodiments, one or more of the queue nodes may be configured to apply an inherent queue tag to the call upon the call entering the queue node, and to remove the queue tag upon exit of the call from the queue node. One or more of the queue nodes may also be configured in some embodiments to apply to the call an agent-provided tag provided by an agent servicing the queue node and retain the agent-provided tag on the call upon the exit of the call from the queue node. In some embodiments, one or more of the queue nodes may be further configured such that if no active agents are assigned agent tags matching tags applied to the call, the call is routed to a configured failsafe route, and on condition that the failsafe route comprises a queue, the queue of the failsafe route is configured as a failsafe queue to strip the call of any of the tags applied to the call that affect routing of the call. In one aspect of some embodiments, the agent-provided tag determines a child node of the queue node to which the call is routed upon the exit of the call from the queue node.

In another aspect, a system extracts useful metrics from spoken conversation, using call grading and call similarity. Call grading extracts important metrics from audio using a combination of direct audio content analysis (how things are said) and analysis of the speech content (words spoken). For the majority of the calls, there are enough examples of that exact type of call (subject, quality, outcome) to very effectively determine the audio and speech aspects of that type of call. For the minority of calls, there may be factors that are relatively rare among the dataset, and extraction of useful metrics may be difficult using call grading alone. In such cases, metrics are retrieved from a similar call to serve as a surrogate. The call grading and call similarity are then weighted, improving confidence in the scoring.

To ensure that the direct call grading and call similarity may be used together and averaged, multiple components of a communication system may utilize the same metrics on the same scale. The metrics may be utilized to track longitudinal histories for audio content by caller, agent, bot, etc. They may also be used in aggregate to track organizational metrics.

Compatibility may also be a concern of the system as the system may be continually trained over time. If the rubrics used to provide human labels change, there may be a temporal bias in the model control structure. Or in severe cases, the model control structure may start to return much less accurate predictions.

In the case that a small dataset may be available to retrain an existing model to a new set of metrics, the bulk of old system may be “freezed” and the final stage of the system may be trained. The original system, in later stages may have a bottleneck stage wherein the system may provide a rich encoding of the audio content before the final predictions. The bottleneck, typically a low-dimensional vector in the system, may be of limited size and, therefore, may efficiently and compactly describe the audio content before providing the final “human-readable” output.

That final transformation, from bottleneck stage to final predictions may often be <1% of the model control structure complexity (and free parameters). As a result, the conversion may be taught with a reduced number of examples. This enable the model to be reused and retrained across different system component and efficiently. In some cases, the bottleneck stage may be a useful output that may be used by organizations.

Call grading may be used in any application where audio content may be analyzed for both semantic and vocal content to measure some set of metrics. Ideal use cases include call centers, conference call systems, internal company meetings, fraud detection, employee training, sales, organizational or legal compliance, and education evaluation.

Human speech differs substantially across domains, cultures, and pretext and all applications may have systems be rebuilt while adhering to the same architecture. Depending the source of audio content, the preconditioning, transcription, and embedding systems may be rebuilt on data from the relevant domains. However, in some cases, metrics may be selected to be general enough to be useful across many different applications, either at the final or bottleneck stage.

A call processing system may physically transform received audio content into a display on a machine, such as a light-emitting device. The display may comprise a grade associated with the audio content received, the displayed grade being useful, concrete, and tangible result of the received audio content. The audio content may represent physical characteristics emitted by a sound producing device, such as a human emitting speech, the physical characteristics including the words spoken and how those words were spoken, and the grade of the machine display is a transformation representing those physical characteristics.

A call processing system may also improve the computerization of a technological process by determining an input for a metric-generating model from a received audio content wherein the audio content is split within the machine into at least two parts, the speech semantic content and the speech vocal content, each of which is then transformed into a vector that may be combined to provide an input, which may be a dense vector, to the metric-generating model.

Specifically, by utilizing multiple model control structures and weighting the results for identifying the speech semantic content from the audio content, identifying the speech vocal content from the audio content, and determining the model control structure from the combined message content, the accuracy, reliability, and quality of the resulting output may be increased. The processing speed of a large data set may be increased by selecting the audio content to which to determine similar audio content, which if applied may also improve the accuracy, reliability, and quality of the resulting computation.

In some embodiments, a slot of a neural network is configured to emphasize a portion of the received audio content, resulting in an enhanced analysis of portions of the audio content that are determined to be more important to a metric.

FIG.1depicts a communication system100in one embodiment. Calls are received by a call processing system102, analyzed and processed using audio signal processor104and a natural language processor106, and results of the analysis and processing are provided to heuristic algorithms108. The heuristic algorithms108apply weights110to call classifiers generated by machine learning models112utilizing one or more learning function114. The heuristic algorithms108may also operate on outputs from the audio signal processor104and natural language processor106. The machine learning models112may in one embodiment comprise an ensemble learning model.

The weighted sub-metrics are input to a GSAT algorithm116that generates aggregate metric controls, in particular normalized aggregate Gaussian metric controls. Herein “GSAT” refers to a normalized aggregate Gaussian metric. The GSAT metric controls are applied as a feedback signal to the call processing system102to adapt one or more of templates118, call processing control logic120, agent behavior, and the behavior of one or more automated voice attendant122. The GSAT metric controls and weighted sub-metrics may be provided in various form to a scorecard display interface124. The GSAT metric controls and possibly the weighted sub-metrics may also be utilized by an alert generator126that may raise an alert on the scorecard display interface124for anomalous calls.

The GSAT metric controls may also be utilized by the agents128and other components (e.g., automated voice attendant122) in real-time (existing call in progress) to adapt their behavior when processing and responding to calls. A call history repository130stores call transcripts, raw audio, weighted sub-metrics, and GSAT metric controls (as well as other information, potentially) for use by other components of the communication system100, for example for use in machine learning and reporting on agent, call, site, and team performance.

Exemplary sub-metrics that the heuristic algorithms108may generate are provided in Table 1 below.

TABLE 1Sub-metricDescriptionTargetcallback_signatureAn indication the caller must callback laterLow isgoodcaller_cross_talk_percentageHow much the caller interrupted the agentLow isgoodsilence_percentageHow much dead air the call containedLow isgoodagent_politenessHow polite the agent wasHigh isgoodagent_cross_talk_percentageHow much the agent interrupted the callerLow isgoodcaller_average_syllablesAverage syllables per word used by theEccentric iscallerbadcaller_word_countNumber of words used by the callerEccentric isbadword_countNumber of words in the callEccentric isbadagent_misunderstand_rateRate at which the agent asks the caller forLow isclarificationgoodagent_empathy_rateIndications the agent expresses sympathyHigh isfor the callergoodagent_valence_trendImprovement in emotional valence over theHigh iscourse of the call for the agentgoodagent_average_syllablesAverage syllables per word used by theEccentric isagentbadcaller_sentence_lengthAverage number of words per sentenceEccentric isused by the agentbadcaller_valence_trendImprovement in emotional valence over theHigh iscourse of the call for the callergoodfiller_word_rateRate of occurrence of filler words in theLow iscallgoodagent_discovery_questionsThe agent uses discovery questions to learnHigh ismore about the caller's situationgoodagent_talk_time_sThe duration of agent talk timeEccentric isbadcaller_gunning_fogA measure of language complexityEccentric isbadcaller_flesch_reading_easeA measure of language complexityEccentric isbadquestion_rateRate at which questions are asked in theHigh iscallgoodcaller_question_rateRate at which questions are asked by theHigh iscaller in the callgoodagent_talk_time_ratioThe ratio of agent to caller talk timeLow isgoodcaller_informalityThe use of informal language by the callerEccentric isbadcaller_discovery_questionsThe caller uses discovery questions to learnHigh ismoregoodcaller_misunderstand_rateThe rate at which the caller misunderstandsLow isthe agentgoodagent_average_emotion_valenceThe average emotional valence of the agentHigh isgoodagent_flesch_reading_easeA measure of language complexityEccentric isbadconfirmation_rateThe rate at which actions are confirmedHigh isgoodagent_informalityThe use of informal language by the agentEccentric isbadagent_question_rateThe rate at which the agent asks questionsHigh isgoodempathy_valence_correlationA measure of empathy measuring theHigh isrelatedness of the agent and callergoodemotional valenceagent_wpmThe rate at which the agent spokeEccentric isbadcross_talk_percentageHow much of the call had the partiesLos is goodspeaking over each otheragent_word_countThe number of words the agent spoke in theEccentric iscallbadresolution_signatureAn indication the call was resolvedHigh isgoodcaller_average_emotion_valenceThe average emotion of the callerHigh isgoodagent_filler_word_rateThe number of filler words used by theLow isagentgoodcaller_wpmThe rate at which the caller spokeEccentric isbadagent_complexityAn overall measure of complexity in theEccentric isagent's speechbadcaller_talk_time_sThe duration of caller talk timeEccentric isbadwpmThe overall rate of speech in the callEccentric isbadagent_competenceIndications the agent is competentHigh isgoodagent_sentence_lengthThe average number of words used in aEccentric issentence by the agentbadgreeting_signatureAn indication the agent properly greetedHigh isthe callergoodaverage_emotion_valenceThe average emotion of the callHigh isgoodcaller_empathy_rateIndication the caller showed empathy forHigh isthe agentgoodcaller_complexityAn overall measure of complexity in theEccentric iscaller's speechbadcaller_competenceIndications the caller showed awareness ofHigh istheir own situationgoodcaller_filler_word_rateUse of filler words by the callerLow isgoodvalence_trendThe overall trend in the emotional valenceHigh isin the callgoodagent_gunning_fogA measure of language complexityEccentric isbadcaller_politenessHow polite was the caller to the agentHigh isgood

One of ordinary skill in the art will appreciate that the sub-metrics in Table 1 may be computed using a number of techniques known in the art. For example, machine learning models (e.g., deep neural networks) may be utilized to predict metrics directly as classifiers, either per-utterance (a segment of an audio call) or over the full call. If computed per utterance, it is then summed and a maximum, minimum, mean, average, or some other descriptive statistic is computed. Statistical models may also be utilized downstream of one or more machine learning model, or on a time series output of a model. An example of this technique is computing the slope of the best fit curve of emotional valence (itself a model output). Statistical natural language processing techniques may also be utilized. For example, precomputed weights for different words and phrases may be implemented in a lookup table, and a word-trie data structure generated to efficiently count occurrences of words and phrases, weighted by configured coefficients. An example is counting all the filler words (“umm”, “you know”), with different penalties assigned per filler word/phrase based on rarity or severity.

Exemplary weights for the sub-metrics in the control metric calculations are given in Table 2 below. One of ordinary skill in the art will appreciate that these weights may be computed in multiple ways known in the art. One technique utilizes linear regression for a given metric against a different metric of call quality (ground truth sources such as human labelers, CSAT, NPS, or a custom QA score, or some combination of several ground truth sources). The linear regression produces an indication of how much each model should be weighted. Metrics may also be weighted more based on their accuracy. With regards to including accuracy and statistical independence, an ensemble model averaging and boosting technique may be utilized, in manners known in the art.

The distribution of each sub-metric may be independently determined over a large sample size of calls.

The alert generator126generates an alert to a system operator on condition that a set of one or more calls that have been detected by models that look for particularly alarming and/or anomalous situations that require special and possibly urgent handling. In one embodiment the alert generator126is configured with alert condition (condition settings202) by weighting a large set of empirically discovered call content patterns based on their historical predictiveness in labelled (training set) calls. Additional weight is assigned based on where in the call the pattern occurs, its rarity, and the outputs of emotion valence models.

While such calls may not always be truly urgent, they are anomalous and unusually likely to require escalation. Each call reported in the set may in one embodiment comprise the following attributes:State of the call (live or completed)Agent name and org chartExcerpts from the most anomalous parts of the callThe phone number and name of the callerThe ability to live listen or review recording snippets from the callA navigation control to the conversation viewDate/time of the call, if not liveA control to snooze or dismiss alerts that are non-emergent.

Certain systemic metrics may be determined and displayed to a system operator (e.g., on a system-wide view of the scorecard display interface124), such as:Number of calls processed in the present dayCall resolution rateNumber of active callsWhether the system is activeAverage GSAT metric control for the system

FIG.2depicts additional aspects of the communication system100in one embodiment. The call processing control logic120comprises an ensemble of machine learning models204utilized to control the behavior of agents and/or the automated voice attendant122, and/or to determine the content of templates118used thereby. Other embodiments may utilize a single machine learning model rather than an ensemble.

The machine learning models204receive the GSAT metric controls (for an agent, automated voice attendant, team, or site) from the GSAT algorithm116and the weighted sub-metrics from the heuristic algorithms108. The call processing control logic120identifies calls for which information is stored in the call history repository130that match characteristics of a particular call (either a completed call or an in-process call) and identifies those calls having more desirable GSAT metric controls and/or weighted sub-metrics. Such superior calls may indicate improved agent and/or automated voice attendant122performance as compared to the particular call. Differences between agent/automated voice attendant122behavior on those superior calls may be applied as a learning function to the machine learning models204to improve agent/automated voice attendant122/templates118performance on the particular call, if ongoing, or on future calls for a particular agent/automated voice attendant122/team/site.

The alert generator126may in one embodiment comprise a learning function206and a threshold detector208responsive to configured condition settings202. If one or more metrics meets the condition settings202, an alert is generated to the system, which may respond to the alert on a priority basis (meaning the alert receives a high priority for remediation over other tasks in the system). The threshold detector208may be implemented as or may utilize a learning function206to learn c/all content patterns, metrics, and sub-metrics, and/or combinations thereof, constituting an alert condition, over time and as more calls are processed.

FIG.3depicts a call flow process300in one embodiment. In block302, the call flow process300operates at least one machine learning model to transform outputs of an audio signal processor and a natural language processor into classifiers for a call. In block304, the call flow process300transforms the call classifiers into a plurality of weighted sub-metrics for the call. In block306, the call flow process300applies aggregate normalized Gaussian logic to the weighted sub-metrics to generate a metric control. In block308, the call flow process300applies the metric control to adapt control logic for a call flow. In block310, the call flow process300applies the metric control (e.g., via the control logic) to adapt a behavior of an automated voice attendant of the call flow. In block312, the call flow process300applies the metric control to adapt a template utilized in the call flow.

In block314, the call flow process300specifically applies the metric control to adapt a machine learning model of the control logic. In block316, the call flow process300applies a learning function for the machine learning model of the control logic utilizing a call history and one or more of the weighted sub-metrics.

FIG.4depicts an alert generation process400in one embodiment. In block402, the alert generation process400operates at least one machine learning model on outputs of an audio signal processor and a natural language processor to generate call classifiers. In block404, the alert generation process400operates heuristic logic to transform the call classifiers into a plurality of weighted sub-metrics for the call. In block406, the alert generation process400applies an aggregate normalized Gaussian transform to convert the weighted sub-metrics into a metric control. In block408, the alert generation process400operates a threshold analyzer to generate an alert signal to the communication system based on the metric control meeting a condition. In block410, the alert generation process400operates an anomaly detector to identify anomalous calls. In block412, the alert generation process400associates with the alert signal portions of the call comprising content that contributed to activation of the alert signal. In block414, the alert generation process400applies a learning function utilizing a call history and one or more of the weighted sub-metrics and the metric control to the alert generator.

Treated as Gaussian-distributed random variables, measured values of each sub-metric may be converted to a percentile (e.g., valued between 0 and 100). Exemplary percentiles are depicted in Table 3.

Each sub-metric for example from Table 1 may be converted to a percentile using a Gaussian cumulative distribution function500(CDF). Such a function is exemplified in the depiction inFIG.5. The percentiles may be weighted in accordance with the allocations depicted in Table 2. The weighted percentiles may summed to a single raw GSAT that resembles the percentile distribution for example as depicted in Table 3. The raw GSAT may then be renormalized and converted to a percentile (e.g., the raw GSAT has a mean of 49.49 and a STD of 8.32), for example as depicted by the renormalized metric control600inFIG.6.

The GSAT metric control, such as renormalized metric control600, may be applied as feedback into the communication system to modify call processing, component behavior, and templates. In some embodiments, templates comprise forms generated and displayed to callers by bot nodes (described below). For example, the metric control may be utilized to modify the audio behavior (questions and responses) of an automated attendant based on audio and semantic attributes of particular callers. The metric control may also be utilized to modify the content of forms generated and presented to callers by said automated attendants, and the processing of those forms.

For trending topics and saved searches, anomalies may control which types of system, agent, or agent group metrics are made prominent to the system operator. Some types of metrics may be mainstays (e.g., a non-dynamic set of configured call metrics to emphasize). Metric comparisons may be presented as time series graphs contrasted with the same metrics over a historical period, as for example depicted in the time series graph700depicted inFIG.7. Examples of time series metrics in one embodiment include:Call Metrics—A set of trends for call metrics. These metrics depict time dynamics and day over day performance. Metrics may be limited to metrics likely to change day to day, for instance, average emotion (which may be driven more directly by external factors than GSAT) and average handle time. Exemplary metrics include:Average call processing timeAverage emotion score for callsAverage emotion trend for callsCall resolutionAverage silence period in callsCall volume

Trending Topics—Words, phrases, or entities that are occurring anomalously frequently in a given time frame. Trending topics may also or additionally include a burst or cluster of calls relating to a topic or issue. Exemplary trending topics include:Saved Searches—A reduced set of saved searches, possibly curated by how anomalous they are for the time frame, and possibly customized from a larger set. If customized, a control to “add to daily briefing” may be generated in the saved search builder.

Time series may comprise the following attributes:The raw numeric value of a particular metric for the time frame (if selected or if the metric is not a global rate metric)The rate (percentage of calls) comprising some attribute for the time frame (e.g., for a binary metric).

Metrics and visualizations for call and/or system dynamics (metric change or change rate over time) may be generated and displayed, for example in the scorecard display interface124for a given call, agent, team (agent group), or system-wide. These metrics may in one embodiment only apply to binary categories (e.g., saved searches, whether a call contains a trending topic, but not, for instance, average handle time). The emotion associated with a topic may be identified as neutral, positive, or negative and whether it has become more negative than historical. This characteristic may be visualized by a pair of histograms over emotional content for calls. See for example the exemplary composite histogram display800depicted inFIG.8.

More generally, metrics and visualizations for dynamics may in one embodiment take the form of one or more of the following:A time series graph of the call property for a time frame compared periodically to a reference time frame. Whether to display raw values or rates for the value also may configurable.List of related topics that co-occur with the binary property. These are related topics to trending topics or to unresolved calls or to saved searches. For example, “burning” has related topics “fire” and “fire insurance”.Pull quotes from matching calls (binary) or extreme values (continuous). For continuous values like emotion or other call metrics, pull quotes may show if they come from high or low examples.An ability to navigate to a list of relevant calls, meaning a set of calls that match the target extreme, or for a related topic, a subset that also contain that related topic. This metric provides an indicator of how anomalous a given call is or calls in a time frame are.

In one embodiment, the system may generate reports in the form of site, team, and agent rankings comprising ranked lists of top-performing systems, teams, and agents by average GSAT.

In one embodiment, the system may pull quotes from calls that represent the nature of the call enabling listening at targeted locations in the call. This report may also depict the resolution of the call.

FIG.9depicts a scorecard display interface902implemented on an interactive machine display904in one embodiment. The scorecard display interface902may be configured for reporting metrics on an agent, a team of agents, or system-wide (a “site”). The scorecard display interface902may comprise these primary components:Scorecard Summary—A set of high level metrics (metric values906) that describe, in aggregate, how well an agent (or team etc.) is performing on configured metrics (e.g., globally configured metrics) over a selectable (interval selection control908) time frame. The metric values906of the scorecard summary may comprise moving averages of the globally configured scorecard metrics.Conversation Filter—A control (target control910) to filter the set of calls included in the aggregations.Performance History—A history (performance history912) for a selected metric (metric selection control914) by time interval, and/or a histogram (visualization selection controls916).Performance Feed (agent only)—A timeline (performance feed918) of a selectable agent performance events (event selection control920).Agent List (team or site only)—A list of agents in a site or team (target list922). Sortable by any metric (metric selection control924).Team List (site only)—A list of teams in the site (target list926). Sortable by any metric (metric selection control924).

The scorecard display interface902may further comprise a drill-down view for each item in the performance history. The drill-down view may pair a metric (e.g., “politeness”) and a target (e.g., “Agent Bob Smith”) and may in one embodiment comprise the following:Header—The metric name and target being drilled intoCoaching Examples—The best and worst calls list for that metric for that target.Coaching Article—A written description of advice for how to improve that metric (if available).

The metric values906of the scorecard display interface902may comprise objective, stable metrics to support agent coaching and may incorporate hysteresis such that reliable aggregate metrics of performance are presented and updated over time. The scorecard display interface902may enable the configuration of goals and monitoring of progress (e.g., via the performance feed918) to achieve those goals for an agent, team, and/or site.

In one embodiment the metric values906of the scorecard display interface902comprise holistic metrics as numeric moving averages. These metrics change slowly relative to a time interval of interest (metric selection control914), such as a day. The scorecard display interface902may in one embodiment comprise display of the following attributes for an agent. In some embodiments, some or all of the following attributes may be displayed in a modal or window when an agent is selected (e.g., by clicking on the agent or hovering on the agent) from the target list922.Agent/Team/Site NameAgent ID (if relevant)Start date (if relevant)Average GSATAverage evaluation score (if available)Average CSAT (if available)Average call handle timeAverage calls per day

The target control910enables filtering of a set of calls affecting the metric values906over which values are tracked and averaged. The set of metric values906that may be filtered may be pre-configured in the system global settings in one embodiment.

The performance history912may be responsive to configurable settings for tracking metrics historically and/or versus peers. Settings in one embodiment may include:Comparison configuration (agent only)—Selects a group to compare either over time or as a histogram. The options may include:Versus team (team comparison control928)Versus site (site comparison control930)Time series comparison configuration (agent only—see “TIME SERIES” control of the visualization selection controls916)—Selects whether the time series graph also graphs against one of the following:Number of time intervals active (average at the currently configured team/site). This may be applied to measure training and growth on newer agents.Average for configured team/site on the same time x-axis. This enables comparison against more mature agents and facilitates corrections for business-related causes of performance ups and downs (e.g., more negative calls across the team due to dissatisfaction with the product or service).

The set of available historical visualizations (performance history912) for a selected metric (metric selection control914) may enable multi-dimensional views of the target's strengths and weaknesses over time. The performance history912may in one embodiment include:Current Value—The current numeric value for the target for the selected metric.Time series (if not categorical)—An interval-binned time series graph of the target's performance on that metric over time. Graphed on the same graph is either a comparison of the average performance of an target with a comparable target with the same or similar time intervals of experience, or comparatively with a team or site overall. See for example the exemplary comparative visualization display1000depicted inFIG.10.Categorical Line Chart (if categorical)—If the selected metric is categorical (option based), the performance history912may comprise a series of line charts depicting an average percentage of responses in which a selected or each available option is selected (on one graph). See for example the categorical line chart display1100depicted inFIG.11.Current Goal (if a current goal is set)—A horizontal line on the time series and/or a cell on the histogram that depicts a goal set for the target.Histogram—A histogram or a plotted gaussian (that matches the mean and standard deviation of the target org). The target's actual value on the selected metric may be marked as a vertical line or shaded region. The target's goal on that metric may be marked with a second line. See for example the color coded distribution displays1200depicted inFIG.12.

The performance feed918may in one embodiment comprise a time-ordered feed depicting recent events of interest to the target. These may include in one embodiment:Annotations made on the target's callsEvaluationsCSAT'sGoals setGoals reachedWork anniversaries

The target list922may in one embodiment comprise a list of agents configured for a team or site. The list of agents in one embodiment may enable display of the following attributes when a particular agent is selected:Agent nameAgent IDAgent start dateAverage GSATAverage evaluationAverage CSATAverage handle timeAverage calls per daySort By—A dropdown of metrics to sort by. When something is sorted by a metric that's not in the default column set, it may be temporarily appended (e.g., filler words).Order—Increasing or decreasing

The target list926for sites may operate similar to the target list922for teams, but may display team attributes instead of agent attributes when a team is selected.

Metrics available for selection or configuration for use in the scorecard display interface902may in one embodiment comprise:Average GSAT, evaluation, and CSATAverage handle time and average calls per dayCall metrics including in one embodiment:Cross talkSilenceHold timeFiller wordsWords spoken per minuteAverage emotionEmotion trendAgent-to-caller talk ratioComplexityPolitenessQuestionsEvery CSAT question scoreEvery evaluation question score

In some embodiments, a metric drilldown 9detail) view may be activated from the scorecard display interface902(drill down view activation control932) and may in one embodiment display various attributes for the target and/or selected metric, as well as the following:Coaching examples (coaching examples934)—A set of recent calls that may be utilized to exemplify extreme examples of the metric for that target. These examples may be “best calls” or “worst calls” for that target sorted by that metric.Good list—N best calls under that metric for that agent, team, or site.Bad list—N worst calls under that metric for that agent, team, or site.Coaching article (coaching articles936)—Written content providing material about how to improve that metric. For some system generated metrics, this is a well-written article about the importance of improving x. For example, why it's important to control your speaking speed and exercises to improve it. For customer metrics (e.g., QA question score), this could be optionally authored by the org. By default, it's the question itself.Article and references

An embodiment of a call processing systems1300is depicted inFIG.13including a telephony carrier network1302, a call center1304, an external endpoint1306, an external endpoint1308, a call flow1310, a call flow1312, a call flow1314, an analog handset1316, a computing device1318, and a mobile phone1320.

Calls originate from sources such as the analog handset1316, mobile phones1320, or computing device1318(e.g., Skype call), for example. These calls are routed through one or more telephony carrier networks1302to a communication interface1322of a call center1304. From the communication interface1322the calls are routed to different call flows such as call flow1310and call flow1314. Call flows may process and forward, or terminate, the calls, or route them (e.g., via outlet nodes) to other call flows, such as call flow1312. Calls may be forwarded to external endpoints outside the call center1304such as external endpoint1306and external endpoint1308.

A call flow control structure1400in one embodiment is depicted inFIG.14, comprising a call flow graph1402, generic node attributes1404, specific node attributes1406, nodes1408, and edges1410.

Nodes1408are configured and joined with edges1410to form a call flow graph1402. The nodes1408each have generic node attributes1404common to all node types, and specific node attributes1406specific to particular types of nodes.

In one embodiment, each node type may include generic node attributes1404including:Name—The (mutable) name of the node. Conversations generated by the node use this name in the Node call metadata.Type—The (immutable) type of the node. Each type may be associated with a unique icon, badge, and/or color.InletQueueBotOutletRecorderHosted ScriptHangupDescription—A description of the node and/or its purpose.Parents—Links to nodes that can route to a node.Children—Links to nodes that a node can route to.State Forwarding—A mechanism to propagate state information from a node to a target endpoint. State includes tags associated with a call, bot data, and general call data. This provides a mechanism to support outbound task fulfillment without utilizing a hosted script.Enable—Turn state forwarding on or off.Trigger—Condition upon which the forwarding is triggeredOn Enter NodeOn Exit NodeBothOutlet TypeHTTP GETURLURL ParametersHTTP POSTURLEmailEmail addressSMSNumberRead only properties. Nodes may also have read only properties and statistics.

In one embodiment, the fundamental types of nodes include:Inlet—A phone number or address that can receive calls.Queue—A call queue serviced by human agents.Bot—A call queue serviced by machine agents.Ender—A hybrid of a queue node and bot node.Outlet—External (to the call flow manager) phone numbers and other communication endpoints.Recorder—A call recorder.Hosted Script—A flexible multipurpose node that implements custom call handling or business logic.Hangup—Terminates the call session.

FIG.15depicts a high-level structure of certain types of call flow control nodes1500, in one embodiment, including a queue node1502, a bot node1504, and an ender node1506. These types of call flow node1508each include a first-in-first-out structure (FIFO1510, FIFO1512, and FIFO1514) for queueing calls for service at the node. The bot node1504type shares other attributes in common with the queue node1502type, and the ender node1506has attributes common to both of the bot node1504and the queue node1502. Thus an ender node1506is a control structure that enables service by both bots and human agents, including the generation and presentation of forms (e.g., surveys or questionnaires) to callers.

Aspects of the communication systems disclosed herein may be utilized to provide adaptive feedback to modify the behavior of some node types. For example aspects of the communication system100depicted inFIG.1andFIG.2may be utilized to adapt the functioning of a queue node1502, bot node1504, and/or ender node1506in which automated voice attendants, agents, or templates are utilized, in manners previously described.

In one embodiment, a queue node such as queue node1502/queue node1602(seeFIG.16) may include these properties:Priority—A number indicating the priority of the queue. Higher numbers indicate higher priority. Zero (0) is the lowest priority possible.Tags—Properties of the queue that are applied match agents to calls. For example, “insurance” or “management”.Background BehaviorSilenceMusicCustom UploadMultiple licensed defaultsPeriodic AnnouncementNoneFixed Automated MessageApproximate Wait TimePosition in QueueFailsafe ChildIn a queue, one of the children may be designated the Failsafe Child (see Agent-Caller Matchmaking)Matchmaking SliderA slider between:FIFO—First in, first out routingSkill Matching—Utilize agent, queue, and caller tags to make a match, de-emphasizing wait times.

In one embodiment, a bot node such as bot node1504/bot node1702(seeFIG.17) may include these properties.Bot Name—Identifies the bot from a set of existing bots.Bot VersionVoiceAutomated messageAgent—Voice agent to use for this botInherit—Use the voice of the preceding node. If the node was a bot, inherit its voice. If it was a queue, inherit the voice of the agent who handled the call.Route mapping—Maps bot outcomes to child nodes. This deconvolves the set of possible bot outcomes from the bot's position in the call flow. It also facilitates bot reuse.Target child node—Determines how a bot outcome routes to child nodes.b. Target metadata field—Determines how bot outcomes are stored, including mapping to metadata, contact name, or CSAT result.Default route—A required field that is useful if the bot implements unhandled capabilities. Form-filling bots directly set call tags, which may be used for agent selection. For instance, a form-filling bot upstream may ask as a question “what language do you prefer?” and the “Spanish” tag is later used in agent selection when the caller is queued.

In one embodiment, an ender node such as ender node1506may include these properties:Bot Name—Selects the bot from a set of existing bots or lets you create a new bot.Bot VersionFreezes the bot to a particular version.LatestVoice (when bot-handled)Default automated voice attendantAgent—Defines an agent for the callInherit—Use the voice of the preceding node. If the node was a bot, inherit its voice. If it was a queue, inherit the voice of the agent who handled the call.Route mapping—Maps bot outcomes to child nodes. This deconvolves the set of possible bot outcomes from the bot's position in the call flow. It also facilitates bot reuse.Default route—A required field that is useful if the bot implements unhandled capabilities.Labor Pool—Default is Gridspace.Target Performance—% of human—Default is 80%Current Performance (Read only)9. Q&A Methodologya. GSAT (Default)b. Default QA FormDocument Set—The indexed documents available to Enders.Greeting—How the Ender should greet a caller.Outcome Names—This node type requires a descriptive name for every outcome, so Enders are configured with activations (e.g., buttons) for different call endings.Outcome Descriptions (Optional)—Additional information for some outcomes.Unhandled Outcomes—A link to the unhandled outcomes editor. The bot is set up with a user interface that is described in one embodiment below.

FIG.16depicts a queue node configuration1600in one embodiment comprising a queue node1602, a call1604, an outcome routing map1606, a call classification system1608, a router1610, a human agent1612, implicit tags1614, a slider1616, a call queue1618, a prioritizer1620, an S model1622, a failsafe1624route setting, a queue priority1626, and a clear tags1628setting.

A call1604is pulled from the call queue1618for the queue node1602and tags are applied including implicit tags1614, tags generated by a call classification system1608(such as described in U.S. application Ser. No. 15/653,411, “CALL CLASSIFICATION SYSTEM”, filed on Jul. 18, 2017), and tags applied by a human agent1612. The priority of the call for purposes of pulling it from the call queue1618may be determined by a prioritizer1620algorithm influenced by one or more of the call's position in the call queue1618and an S model1622. An exemplary prioritizer1620algorithm and S model1622are described in more detail below.

The implicit tags1614and a configured priority1626may affect which calls are routed into the call queue1618for the queue node1602.

The applied tags may affect the operation of the router1610for the queue node1602, such that a next node in a call flow is selected to receive the call based on matching agents or bots assigned to the next node with the tags on the call1604. Outcomes from the call classification system1608and human agent1612may also be applied to affect the routing, where outcomes are intentions derived from the call1604about the reason(s) the caller has for making the call1604. Intentions may be derived from the spoken content of the call1604, from forms presented to the caller, from historical data about the caller, or other means.

The router1610may also be influenced by an outcome routing map1606, which maps determined outcomes for the call1604to routes to downstream nodes of the call flow. The output of the router1610may take one of a number of forms as defined by output format settings1630, which may also configure the condition on which forwarding from the node is triggered.

A slider1616control may be operable by a human agent1612or by another means (e.g., automatically adjusted based on call volume/wait times, etc.) to balance between the influence of FIFO position/wait time and use of the S model1622on the priority of calls for servicing from the call queue1618.

The router1610may be configured (clear tags1628) to clear tags applied to the call, or not. The router1610may also be configured with a failsafe1624route for forwarding calls, in the event the call cannot be matched definitively to a downstream node using tags or outcomes.

Queue nodes operate to encode call state transitions, providing a singular queue caller tag that may be applied for call routing. A general pool of human agents and bots continuously undergoes a matchmaking process with existing callers. A type of node, herein referred to as a smart route node, may continuously select a best-match caller for an available agent, utilizing for example three parameters: 1) the hold time (which maybe represented by FIFO position), 2) the agent skill tags (if configured), and 3) the caller tags in coordination with historical agent performance.

In some embodiments, when selecting the best caller for an agent, the system evaluates a match score of this form.
M(a,ci)=αH(ci)+(1−α)(βB(a,ci)+εS(a,ci))   Equation 1

Where,M is the total match scoreH is the caller's hold timeB is the bootstrap score, which compares the similarity of the agent skill tags and the caller tagsS is a model that computes P(¬rla, ci), the probability a call will not reroute, given the caller tags and the agent's historical performance with those tags.α is a setting that interpolates between FIFO and the use of the S modelβ is a second weight parameter that controls how much the bootstrap score contributes relative to the S model outputε Is a setting to enable or disable use of the S model.

In one embodiment, B(a, ci) computes a minimum edit distance from each caller tag to each agent tag. This enables the system to reasonably understand that the caller tag “Speaks Spanish” and “Spanish” are related. One of ordinary skill in the art will appreciate that other fuzzy matching algorithms may also be utilized.

Here L(tc, ta) is the edit distance (for example, the character match error rate). This score sums up the best-case error for each caller tag, given the agent tags. If an agent has no tags defined, the character error rate for each tag is 1.0, so they are penalized the number of caller tags.

Callers with more tags are more constrained, and, with no agent tag information to use, the bootstrap score may be such that they are deferred for later processing (e.g., moved back in the FIFO).

The S model score may be a Bayesian estimate of the likelihood the call will not need to be rerouted. For each caller tag, for an agent, the model may compute:

This interpolates between the prior (likelihood of a reroute given the caller tags across all agents) and the posterior (number of reroutes that agent has encountered given the caller tag dividing by all callers with that tag the agent has seen). The posterior may be computed from one of many types of models, including statistical models, deep neural networks, decision forests, KNN or K-means clustering, larger Bayesian networks, or direct regression.

The total model score is the product of each of these Bayesian estimators.

Once these scores have been computed, the caller with the best score may be matched with the agent. Over time, the agent models learn about which agents handle which tags best. This allows upstream bots and agents (or CRM metadata) to be supplied blindly to the S model, to enable complex decisions such as matching agents to particular combinations of caller properties. New agents may be matched based on the Bayesian prior for the tags and the bootstrap measure.

FIG.17depicts a bot node configuration1700in one embodiment including a bot node1702, an outcome routing map1704, a call classification system1608, a form1706, a call1708, a router1710, an automated attendant1712, and a call queue1714. Other components of the502in common with a queue node1602are not depicted in the interest of clarity but will be understood to be present in some embodiments according to the following description.

The call1708is pulled from the call queue1714based on a priority determined for example in the manner described for a queue node1602(e.g., a balance setting between FIFO position and S model fit). Tags are applied to the call as determined by the call classification system1608, an automated attendant1712, and/or a form1706presented to the caller. Outcomes for the call1708may likewise be determined and assigned to downstream nodes by the router1710based on an outcome routing map1704. The tags and/or outcomes influence the router1710to select a downstream node for routing the call1708. The router1710may also be influenced by configured settings such as those described for the queue node1602(output format settings1630, failsafe1624route etc.).

FIG.18depicts a call prioritization process1800in one embodiment. In block1802, the call prioritization process1800directs a call along a directed graph of one or more call processing nodes, at least one of the nodes comprising a call queue. In block1804, the call prioritization process1800configures the node comprising the call queue with a control to balance (a) contributions of first-in, first-out priority, and (2) matching of the call to agent attributes, to a service priority of calls in the call queue. In block1806, the call prioritization process1800operates the control to set parameters α and β of a service priority algorithm in accordance with Equation 1.

FIG.19depicts a call flow1900in one embodiment. The call flow1900comprises an inlet node1902, a bot node (AVA)1904, a recorder node (leave a message)1906, a queue node (priority calls)1908, a queue node (sales calls)1910, a queue node (support calls)1912, a bot node (survey)1914, and a hangup node (survey)1916. The inlet node1902binds a communication address1918to the call flow1900. The call flow1900also includes a hosted script node1920with customized logic for handling calls that don't match to the capabilities provided by other nodes.

Calls to the communication address1918are received at the inlet node1902and from there directed to a bot node (AVA)1904with an automated voice attendant (AVA). Based on outcomes from the bot node (AVA)1904, the call is selectively routed to either the recorder node (leave a message)1906, the queue node (priority calls)1908, the queue node (sales calls)1910, the queue node (support calls)1912, or the hosted script node1920. Once processed at one of these nodes, the call is routed for a survey at bot node (survey)1914, and then to hangup node (survey)1916to terminate the call.

In one embodiment, an inlet node such as inlet node1902may include these properties:Inlet TypePhone numberSIP addressAddress—The phone number or SIP

The call flow1900exemplifies a call flow in a call center. The call flow1900results in one agent each serving one queue each, and a bot that routes to three queues and a recorder.

The inlet node1902is configured with a communication address1918(e.g., phone number) that binds the communication address1918to the call flow1900. Other type of communications addresses may also be utilized, such as IP addresses, email addresses, and so on.

A hosted script node allows custom scripting actions to be performed on a call. The call tags and call data are made available to the handler method of the node.

In one embodiment, a hosted script node such as hosted script node1920may include these properties:Script Name—The name of the hosted scriptScript Content—An editor (e.g., Javascript editor) for the hosted script.Save ScriptConsoleLink to documentation

A call flow control interface2000in one embodiment for transcripts and recordings from conversations in which an ender node (e.g., ender node1506) is not configured to handle the outcomes is depicted inFIG.20A,FIG.20B, andFIG.20C. For each call, an administrator may mark the correct outcome or type an answer to the question.

FIG.21depicts a call flow2100in one embodiment including an inlet node2102, a bot node (front desk)2104, a recorder node (file a complaint)2106, a queue node (technical support)2108, a queue node (manage bookings)2110, and a hangup node2112.

An inlet node2102receives calls into the call flow2100. All calls are routed first to a bot node (front desk)2104. A greeting and outcomes may be configured for the bot node (front desk)2104, such as:Technical supportManage bookingsFile a complaint

A different queue node is included in the call flow2100, each a child of the bot node (front desk)2104, and each for routing calls with a different determined outcome. The queue node (technical support)2108receives calls for callers expressing a desire for technical support. The queue node (manage bookings)2110receives calls for callers that want to book travel. From the queue nodes, the call flow2100proceeds to a hangup node2112for termination. A recorder node (file a complaint)2106receives and records complaint calls.

Human agents may be assigned to the queue node (technical support)2108and queue node (manage bookings)2110. The agent for the queue node (technical support)2108may be assigned a tag such as “queue: technical support”. If a the caller asks for technical support, the configured agent is connected to the caller via the queue node (technical support)2108. They have a conversation and then the agent operates a control to direct the call to the hangup node2112.

FIG.22depicts a call flow2200for a university in one embodiment including an inlet node2202, a bot node (front desk)2204, an outlet node (academic dean)2206, an outlet node (academic support)2208, a queue node (admissions)2210, a hangup node2212, an outlet node (advancement services)2214, a queue node (alumni support)2216, a queue node (anthro dept)2218, a queue node (catch all)2220, and a hangup node2222.

The call flow2200implements a broad fan out to reflect a confederation of university staff and employees from various departments, which may include a few regular off-duty agents. Phone numbers for these people and departments may be spread out across various websites and directories. Given the distributed nature of the organization, it would typically be challenging to provide call center analytics.

A catch-all number is assigned to an inlet node2202that routes to a bot node (front desk)2204that both answers common questions (e.g., admissions deadlines) and also routes to a wide array of departments (outlet node (academic dean)2206, outlet node (academic support)2208, queue node (admissions)2210, outlet node (advancement services)2214, queue node (alumni support)2216, queue node (anthro dept)2218), and to a default queue node when none of these departments are suitable for the caller (queue node (catch all)2220). Each routing branch eventually terminates at a hangup node (hangup node2222, hangup node2212). In one embodiment, university agents may mark themselves as “On-Call” to the call flow2200, and they receive SMS messages and browser notifications when a call is ready for service at a queue node they are assigned to.

In one embodiment, an outlet node such as outlet node (academic dean)2206and/or outlet node (academic support)2208and/or outlet node (advancement services)2214may include these properties:TypePhone numberSIP addressAddress—The phone number or SIP addressProxyTrue—Continue to record and process analytics about the call after routing the call from the outlet node.False—End the recording and processing upon routing.Ringback (only if Proxy is True)Traditional RingbackProprietary RingbackHold MusicCustom UploadMultiple licensed defaults

FIG.23depicts a call flow2300for a hotel front desk in one embodiment including an inlet node2302, a bot node (front desk)2304, a recorder node (guest feedback)2306, a bot node (bookings)2308, a queue node (hotel front desk)2310, a hangup node2312, and a hangup node2314.

The call flow2300may be less complicated than many other types of call flows, however, it provides substantial automation potential. Additionally, given the staff at a hotel may be often busy with other tasks, wait times may be long and bursty. The hotel has its main number assigned to an inlet node2302via their telephony provider. The inlet node routes to a bot node (front desk)2304that has a large number of intents (outcomes) configured. For example the AVA configured for the bot node (front desk)2304may answer questions about hours, hotel amenities, and upcoming events. The bot node (front desk)2304may additionally route to several departments, e.g., the front desk (queue node (hotel front desk)2310) and the reservations desk (bot node (bookings)2308).

The form filling bot node (bookings)2308may collect reservation information and route to the reservation department. If the reservation department is closed (and thus no matchmaking to an active agent can occur), a failsafe route from the queue node (hotel front desk)2310directs the call to recorder node (guest feedback)2306. The recorder node (guest feedback)2306may be configured with a prompt thanking the caller, and the recorder node (guest feedback)2306may be configured to output call information via emailed to the reservation email address to be processed at a later time. When the caller says they want to leave feedback, they may be routed to a guest feedback form filling bot node (not depicted) that performs a survey and sends it to management before directing the call to the hangup node2312.

A second call flow (not shown) may be implemented to handle internal calls, such as requests for room service, turn down service, Wi-Fi technical support, and valet service.

In one embodiment, a recorder node such as recorder node (guest feedback)2306may include these properties.Recording typeFixed DurationSeconds per recordingUntil speech stops2. Recording prompt or sound3. Enable NotificationsTriggerAll recordingsb. Email notificationEmail addressSMS notificationNumbersWebhookGET/POSTURI

FIG.24depicts a call flow2400for a tire retail store in one embodiment including an inlet node2402, a bot node (form fill)2404, a queue node (tire specialists)2406, and a hangup node2408.

In the call flow2400a main phone number is bound to inlet node2402and from there directed to bot node (form fill)2404which presents the caller with a form. The form comprises a survey that asks, for example:Customer nameCar makeCar modelDriving weather conditionsTire sizeAre you buying for a fleet or yourself?

The call is then routed into a single queue node (tire specialists)2406. The tag generated in response to the answer to the fleet question on the form is used to match with the available agents, some of whom are specialists in the lucrative fleet business. Initially, agents that handle fleet calls have the agent tag assigned “fleet: true” and the queue node will preferentially match a call indicating a fleet purchase to bootstrapped agents with that tag. Over time, if use of the S model (see S model1622) is enabled in the queue node (tire specialists)2406, the S model learns not only which agents are best at handling fleet calls, but also, which agents are most familiar with certain car makes and driving conditions. All of these tags may be applied for improved matching of calls to agents in the future.

If the caller is unmatchable, they are quickly routed to the hangup node2408. The form information may be emailed to the company's email ticketing system.

FIG.25Adepicts a call flow2500for a financial services organization in one embodiment including an inlet node2502, a bot node (front desk)2504, an outlet node (bank)2506, and an outlet node (insurance)2508.

The financial services organization in this example has two divisions: Bank and Insurance. The call centers for these divisions are independently operated, and therefore, agents only belong to one organization or the other. There may be three phone numbers to reach the call centers:Financial GeneralBank DirectInsurance Direct

The general number is assigned to an inlet node2502and from there directs to a single bot node (front desk)2504that routes callers to the distinct organizations via outlet node (bank)2506and outlet node (insurance)2508. In the case of Bank, there may be 10,000 agents, with over 1,000 discrete skills. Some of these skills are minor (FOOTBAL_TEAM:COWBOYS) and some are critical (ROLE:SUPERVISOR). One or both outlet nodes may have a proxy setting2510enabled to continue to record and process analytics about the call after routing the call from the outlet node. Agents are assigned many different skills via tags, however some map directly onto queues they are intended to serve (QUEUE:MORTGAGES). Initially, the bootstrapping may perform well, as the agents are well-segmented into queues they are trained to handle.

However, understaffing may begin to result in longer hold times. Over time, the agents start receiving calls from queues they were not initially assigned to. Due to the evolving knowledge base of call and agent information, cross-trained agents are discovered to be more than competent at handling calls from queues they were not initially assigned to.

In the Insurance division, things may work differently. Agent matchmaking may be more complicated, and in their previous ACD, over-constrained. They instead use a form-filling bot to gather information about the caller and then fluidly match agents based on skills and experience.

FIG.25Bdepicts the call flow2500in the bank division in additional aspects including an inlet node2512, a bot node (front desk)2514, a queue node (billing)2516, a queue node (mortgages)2518, and a queue node (credit card)2520. A call routed from the outlet node (bank)2506is received at inlet node2512, routed to bot node (front desk)2514, and from there to one of several queue nodes for different departments (queue node (billing)2516, queue node (mortgages)2518, or queue node (credit card)2520).

FIG.25Cdepicts the call flow2500in the insurance division in additional aspects including an inlet node2522, a bot node (form fill)2524, and a queue node (insurance)2526. A call routed from the outlet node (insurance)2508is received at the inlet node2522and routed from there to the bot node (form fill)2524to have the caller fill out a form identifying more details of the reason for the call. From there the call is routed to the general queue node (insurance)2526for service by an agent.

FIG.26-FIG.36depict embodiments of a call classification, metric generation, and anomalous call detection system and techniques. Aspects of these embodiments may be utilized for example to implement aspects of the machine learning models112, heuristic algorithms108, alert generator126, machine learning models204, and/or learning function206.

Referring toFIG.26, the audio environment2600comprises a first audio provider2602, a second audio provider2604, a third audio provider2606, a fourth audio provider2608, a first audio transmitting device2610, a second audio transmitting device2612, a third audio transmitting device2614, a fourth audio transmitting device2616, a telephone network2618, an internet2620, a server2622, an audio files control memory structure2624, a machine display2626, and an audio analysis system2700.

The first audio provider2602, the second audio provider2604, the third audio provider2606, and the fourth audio provider2608produce speech, which may be converted to audio. The first audio provider2602, the second audio provider2604, the third audio provider2606, and the fourth audio provider2608may be a human, a machine configured to produce speech, or other structure capable of producing speech.

The first audio transmitting device2610, the second audio transmitting device2612, the third audio transmitting device2614, and the fourth audio transmitting device2616receive the speech from the first audio provider2602, the second audio provider2604, the third audio provider2606, and the fourth audio provider2608, respectively. An audio transmitting device may receive speech from one or more audio providers. The first audio transmitting device2610, the second audio transmitting device2612, the third audio transmitting device2614, and the fourth audio transmitting device2616transform the speech into audio and send the audio to the telephone network2618. Each audio transmitting device may comprise a receiver to convert the sound wave associated with the speech to a electronic signal (i.e., the audio).

The telephone network2618receives the audio from each of the audio transmitting devices and sends the audio via the internet2620, to the server2622. Each audio may be associated with one or more other audio.

The server2622receives the audio and may send the audio, as recorded audio files, to the audio files control memory structure2624. The server2622may also send audio content to the audio analysis system2700.

The audio analysis system2700receives the audio content and generates an output that is sent to the machine display2626and/or a feedback control, which may be sent to the server2622. The feedback control may also be sent to one or more of the audio providers to alter the generation of the speech.

Referring toFIG.27, the audio analysis system2700comprises an audio content receiving component2702, a speech vocal content identifying component2704, a speech semantic content identifying component2706, a transformation component2708, a model control structure generating component2710, a weighting component2712, an idiosyncratic audio content identifying component2714, a similar audio content identifying component2716, a predictive metric control extraction component2718, and a model control structure sending component2720.

The audio content receiving component2702may receive an audio content of human speech as an input. The audio content is sent to the speech vocal content identifying component2704, the speech semantic content identifying component2706, and the idiosyncratic audio content identifying component2714.

The speech vocal content identifying component2704receives the audio content from the audio content receiving component2702. The speech vocal content identifying component2704may analyze speech patterns, cadences, and tone, which may imply confidence, empathy, kindness, or satisfaction, among many other metrics to generate speech vocal content. The speech vocal content identifying component2704sends the speech vocal content to the transformation component2708.

The speech semantic content identifying component2706receives the audio content from the audio content receiving component2702. The speech semantic content identifying component2706may analyze the speech semantic content (what words were spoken) for clues as to how the conversation went against a trained set of metrics. The speech semantic content identifying component2706sends the speech semantic content to the transformation component2708.

The transformation component2708merges and combines the outputs of the speech vocal content identifying component2704and the speech semantic content identifying component2706into a large vector. This vector may densely encode important features of both paths. The transformation component2708sends the combined vector to the model control structure generating component2710.

The model control structure generating component2710receives the combined vector from the transformation component2708. The model control structure generating component2710may be a dense neural network, or any other common machine learning technique. The combined information may be integrated into a model control structure. The model control structure may be a multi-modal model control structure. The model control structure is sent to the weighting component2712.

The idiosyncratic audio content identifying component2714receives the audio content from the audio content receiving component2702. The idiosyncratic audio content identifying component2714may utilize several methods when analyzing audio content to determine at how common or rare the audio content may be (e.g., does the audio content contain unusual words or phrases, is the audio content noteworthy). The idiosyncratic audio content identifying component2714sends the idiosyncratic audio content to the similar audio content identifying component2716.

The similar audio content identifying component2716receives the idiosyncratic audio content from the idiosyncratic audio content identifying component2714. In cases where the audio content may be designated to be a poor fit for the direct call grading in isolation, the similar audio content identifying component2716may utilize a matching technique may be used to compare the audio content against similar audio content. This may be performed by extracting the word embedded vectors of the audio content into a matrix, and optionally combining audio content features (e.g., special features, signal intensity, variance, etc.) along with the word embeddings. This forms a large matrix representing the audio content. The one or more stored audio content files may be stored as a matrix or some indexed set of features such that matching may be quickly performed. Matching algorithms include euclidean or cosine distance, minimum flow, or distance along a space filling curve (i.e., a Hilbert curve). These matching algorithms may have a low- and high-fidelity step such that the majority of audio content may be filtered, rather than performing a linear search. The similar audio content identifying component2716sends the similar audio content to the predictive metric control extraction component2718.

The predictive metric control extraction component2718receives the similar audio content from the similar audio content identifying component2716. When an audio content is matched with a similar audio content, the predictive metric control extraction component2718may extract the labels and annotations on the similar audio content (i.e., the predictive metric control). The predictive metric control is sent to the weighting component2712.

The weighting component2712receives the model control structure from the model control structure generating component2710and the predictive metric control from the predictive metric control extraction component2718. The weighting component2712may average the similar audio content into the predicted call grade generated by the model control structure. The weighting component2712sends the weighted model control structure to the model control structure sending component2720.

A noteworthiness metric may be used to decides the weighting in the average. This system may be primarily unsupervised, and improves with the number of examples. This increases the serendipitous similarity of the most-similar audio content.

Audio Content Types

Once direct call grading and call similarity have been trained with a sufficiently large dataset, they may accurately replicate human grading. The two systems complement each other, as they excel at opposite ends of the idiosyncratic spectrum. As both systems produce outputs on the same scale (the former producing a vector estimating the result of a human response and the latter finding a human response from similar audio content), they may be combined in a weighted average by the weighting component2712.

Where an audio content lies on the idiosyncratic spectrum may be estimated using call similarity. The similarity distance between the target audio content and a small set of randomly sampled audio content files may be computed. The higher the average match (lower distance to chosen audio content), the less idiosyncratic the audio content may be, and, therefore, the more weight may be placed on similarity.

If the average similarity match (higher is less distinct) over the random sample of audio content is:

Average Similarity

then the weighted average computed is:

Weighted Average

where g is the call grade, d is the result of sending the audio content and the transcript of the audio content through the direct call grading neural network, and s is the grade from the similar audio content in the similarity system.

While training the system may require large amounts of data and computation, running direct grading may typically be relatively quick. Call similarity may perform many matrix to matrix distance calculations between the audio content and the one or more stored audio content files and, therefore, may take longer to query than to train.

In one example implementation, direct call grading may be implemented using state of the art speech recognition, word embedding shallow neural networks, and a multi-modal long short-term memory (LSTM) recurrent neural network. Call similarity may be performed by embedding the sequence of words into a sequence of vectors, with several signal features (i.e., energy, variance, spectral coefficients) appended to the word embedding. The distance function between two similarity matrices may minimize the distance between paired word/signal vectors.

The model control structure sending component2720receives the weighted model control structure from the weighting component2712. The model control structure sending component2720may output the weighted model control structure as a vector of results, which may reproduce custom metrics (e.g., empathy, success, trust, competence), survey results (e.g., “On a scale of 1 to 5, how well did the agent resolve your issue?”), or other predictive analytics (e.g., future purchases, customer value, gender, demography). The weighted model control structure may also be sent as a feedback control to influence the creation of future audio records.

The audio analysis system2700may be operated in accordance with the processes describe inFIG.33andFIG.34.

Referring toFIG.28, the audio analysis system2800comprises an audio content receiving component2702, a speech vocal content identifying component2704, a speech semantic content identifying component2706, a transformation component2708, a model control structure generating component2710, and a model control structure sending component2720. The speech vocal content identifying component2704may further comprise a signal conditioning component2802, an audio spectral features component2804, and a CNN/RNN component2806. The speech vocal speech semantic content identifying component2706may further comprise a speech recognition engine component2808, a word vectorization and embedding component2810, and a DNN/RNN component2812.

The audio content receiving component2702receives the audio content and sends the audio content to the speech recognition engine component2808and the signal conditioning component2802.

The speech recognition engine component2808receives the audio content from the audio content receiving component2702. The speech recognition engine component2808transcribes the audio content. The transcribed audio content is sent to the word vectorization and embedding component2810.

The word vectorization and embedding component2810receives the transcribed audio content from the speech recognition engine component2808. The word vectorization and embedding component2810vectorizes the words in the transcribed audio content and embeds them into a lower dimensional vector space. Many different methods may be used to transform text to a denser vector space including matrix methods, logistic regression, and neural networks. The audio speech recognition step and vectorization steps are trained on larger corpuses of general speech or text. The transcribed and vectorized audio content is sent to the DNN/RNN component2812.

The DNN/RNN component2812receives the transcribed and vectorized audio content from the word vectorization and embedding component2810. The DNN/RNN component2812is the speech portion of call grading. The DNN/RNN component2812may include hand-designed heuristics, regression models, Bayesian models, latent Dirichlet allocation (LDA), latent semantic indexing (LSI), decision trees, decision forests, support vector machines, or a neural network, with or without recurrent units. The DNN/RNN component2812may have a slot to emphasize a portion of the audio content. The DNN/RNN component2812may be trained to emphasize this portion or may receive an input with instructions to do so. The portion may be fixed or based on the size of the audio content. For example, the portion emphasized may be the final thirty seconds of the audio content. The DNN/RNN component2812sends the speech semantic content to the transformation component2708.

The signal conditioning component2802receives the audio content from the audio content receiving component2702. The signal conditioning component2802conditions the audio content to better isolate or prepare the audio content. This may include de-reverberation, noise removal, normalization, distortion correction, beam-forming, mixing, frequency-depending filtering, or any other digital signal processing methods that prepares the audio content for the downstream processing. The signal conditioning component2802sends the conditioned audio content to the audio spectral features component2804.

The audio spectral features component2804receives the conditioned audio content from the signal conditioning component2802. The audio spectral features component2804processes the conditioned audio content with a feature extractor to generate spectrograms or some other spectral features. The features may be a 2D array of time-frequency data. The signal conditioning component2802sends the extracted features to the CNN/RNN component2806.

The CNN/RNN component2806receives the extracted features from the audio spectral features component2804. The CNN/RNN component2806may be a one- or two-dimensional convolutional neural network, an ordinary stacked neural network (with or without recurrent units), matrix models, hand-designed heuristics, decision trees, decision forests, support vector machines, or any other machine learning model that may examine a time and/or frequency domain signal. While the diagram here shows a choice of neural networks in the speech- and audio-path, one or several other statistical learning methods may be combined. The CNN/RNN component2806generates the speech vocal content and sends the speech vocal content to the transformation component2708.

The transformation component2708receives the speech semantic content from the DNN/RNN component2812and the speech vocal content from the CNN/RNN component2806. The transformation component2708concatenates the speech semantic content and the speech vocal content into a combined vector and sends the combined vector to the model control structure generating component2710.

The model control structure generating component2710receives the combined vector from the transformation component2708. The model control structure generating component2710may be a dense neural network, or any other common machine learning technique. At this stage, the combined information may be integrated into a model control structure. The model control structure is sent to the model control structure sending component2720.

The model control structure sending component2720receives the model control structure from the model control structure generating component2710, generates model prediction controls, and sends model prediction controls to drive a machine state of one or more machines. The model control structure may be applied as a feedback control to influence the creation of future audio records.

The audio analysis system2800may be operated in accordance with the processes describe inFIG.33andFIG.34.

Referring toFIG.29, the audio analysis system2900comprises an audio content receiving component2702, a model control structure sending component2720, a speech recognition engine component2808, a word vectorization and embedding component2810, a signal conditioning component2802, an audio spectral features component2804, and a call grading training component2902. The call grading training component2902may further comprise a transformation component2708, a model control structure generating component2710, a DNN/RNN component2812, and a CNN/RNN component2806.

The call grading training component2902may be trained in a supervised manner using labelled pairs of audio recordings and desired model outputs. Portions of the model are either hard coded or trained on generic data. Depending on the complexity of the model and the quality of the audio data, the model may take variously sized datasets to train. Complex models may take tens of thousands of conversations to reach target accuracy. Once the model is fully-trained, it can be used as a replacement for manual human grading of calls, in some cases at human-level accuracy.

The audio analysis system2900may be operated in accordance with the processes describe inFIG.33andFIG.34.

Referring toFIG.30, the audio analysis system3000comprises an audio content receiving component2702, a speech recognition engine component2808, a word vectorization and embedding component2810, a signal conditioning component2802, an audio spectral features component2804, a similarity matrix component3002, a one or more stored audio content files3004, and a distance function generating component3006.

The audio content receiving component2702, the speech recognition engine component2808, the word vectorization and embedding component2810, the signal conditioning component2802, and the audio spectral features component2804operate as described above. The word vectorization and embedding component2810sends the transcribed and vectorized audio content to the similarity matrix component3002. The audio spectral features component2804sends the extracted features to the similarity matrix component3002.

The similarity matrix component3002receives the transcribed and vectorized audio content from the word vectorization and embedding component2810and the extracted features from the audio spectral features component2804. The similarity matrix component3002concatenates the vectorized audio content and the extracted features into an audio content matrix that represents the audio content. The similarity matrix component3002sends the audio content matrix to the distance function generating component3006.

The one or more stored audio content files3004may be formatted as a series of matrices. Each of the one or more stored audio content files3004may represent previous audio content that has been indexed with a set of features.

The distance function generating component3006receives the audio content matrix. The distance function generating component3006compares the audio content matrix to the one or more stored audio content files3004. Matching algorithms include euclidean or cosine distance, minimum flow, or distance along a space filling curve (i.e., a Hilbert curve). These matching algorithms may have a low- and high-fidelity step so that the majority of calls may be filtered, rather than performing a linear search. When an audio content matrix is matched with one of the one or more stored audio content files3004, the labels and annotations on the matched one or more stored audio content files3004are transformed into a predictive metric control. The predictive metric control may be applied as a feedback control to influence the creation of future audio records.

The audio analysis system3000may be operated in accordance with the processes describe inFIG.33andFIG.34.

Referring toFIG.31, the audio analysis system3100comprises an audio content receiving component2702, a transformation component2708, a model control structure generating component2710, a model control structure sending component2720, a speech recognition engine component2808, a word vectorization and embedding component2810, a DNN/RNN component2812, a signal conditioning component2802, an audio spectral features component2804, a CNN/RNN component2806, a new model control structure generating component3102, and a new model control structure sending component3104.

The audio analysis system3100may be retrained to utilize the new model control structure generating component3102and the new model control structure sending component3104in place of the model control structure generating component2710and the model control structure sending component2720.

In some embodiments, the new model control structure generating component3102and the new model control structure sending component3104may be utilized with the model control structure generating component2710and the model control structure sending component2720, providing multiple output controls. The audio analysis system3100may utilized one or more of the new model control structure generating component3102and the new model control structure sending component3104, each generating a new model control structure. The new model control structures and the model control structure may be further combined into multi-modal model control structure. Each model control structure may be weighted prior to being combined. The multi-modal weight may be based on the correlation of each model control structure to the other model control structures. A model control structure with a high correlation with other model control structures may be weighted lower than a model control structure with a low correlation with other model control structures. The correlation, and thus the multi-modal weights, may be pre-determined based on operating the model control structures with training audio content.

In other embodiments, new DNN/RNN component2812and new CNN/RNN component2806may be utilized. These new DNN/RNN component2812and new CNN/RNN component2806may be similarly weighted by the transformation component2708to generate a multi-modal model control structure, where components with higher correlations to other components are weighted less than those with lower correlations to other components. The correlation, and thus the multi-modal weights, may be pre-determined based on operating the components with training audio content.

Audio analysis system3100may be operated in accordance with the processes describe inFIG.33andFIG.34.

Referring toFIG.32, the audio analysis system3200comprises an audio content receiving component2702, a transformation component2708, a model control structure generating component2710, a weighting component2712, a model control structure sending component2720, a speech recognition engine component2808, a word vectorization and embedding component2810, a DNN/RNN component2812, a signal conditioning component2802, an audio spectral features component2804, a CNN/RNN component2806, a similarity matrix component3002, a one or more stored audio content files3004, and a distance function generating component3006.

The audio content receiving component2702receives the audio content and sends the audio content to the speech recognition engine component2808and the signal conditioning component2802.

The speech recognition engine component2808, the word vectorization and embedding component2810, the signal conditioning component2802, and the audio spectral features component2804process the audio content and send to the direct call grading components (i.e., the DNN/RNN component2812, the CNN/RNN component2806, the transformation component2708, and the model control structure generating component2710) and the call similarity components (i.e., the similarity matrix component3002, the one or more stored audio content files3004, and the distance function generating component3006).

The direct call grading components generate a multi-modal model control structure and send the multi-modal model control structure to the weighting component2712.

The call similarity components generate a predictive metric control. The predictive metric control may comprise a measure of similarity and idiosyncrasy of the audio content. The predictive metric control is sent to the weighting component2712.

The weighting component2712generates a weighted model control structure from the multi-modal model control structure and the predictive metric control and sends the weighted model control structure to the model control structure sending component2720.

The model control structure sending component2720generates a model control structure. The model control structure may be applied as a feedback control to influence the creation of future audio records.

The audio analysis system3200may be operated in accordance with the processes describe inFIG.33andFIG.34.

Referring toFIG.33, the audio analysis process3300receives audio content (block3302). The speech semantic content is identified from the audio content (block3304). The speech vocal content is identified from the audio content (block3306). The combined message content id determined (block3308). The combined message content is a transformation of the speech semantic content and the speech vocal content. The model control structure is determined from the combined message content (block3310). The model control structure is applied as a feedback control to influence the creation of future audio records (block3312). The audio analysis process3300then ends (done block3314).

The audio analysis process3300receives audio signals and generates controls to drive the machine state of one or more machines. The model control structure may be a grade of the audio content, the one or more machines comprising a machine display, the machine display altered to display the grade.

The audio analysis process3300may perform a subroutine comprising determining the similar audio content, the similar audio content selected from one or more stored audio content files by comparing the audio content to the one or more stored audio content files; extracting a predictive metric control from the similar audio content; determining a weighted model control structure by combining the predictive metric control with the model control structure; and sending the weighted model control structure to affect the machine state of the one or more machines. During determining the similar audio content from the one or more stored audio content files, a tree structure may be utilized to reduce the number of comparisons between the audio content and the one or more stored audio content files. The tree structure may comprise a indication of the similarity among the one or more stored audio content files. As the audio content is compared to one of the one or more stored audio content files, the remaining one or more stored audio content files are filtered based on their relationship to the one compared to the audio content. The other one or more stored audio content files may be filtered if the comparison indicates similarity and the relationship indicates dissimilarity or the comparison indicates dissimilarity and the relationship indicates similarity. The unfiltered one or more stored audio content files may be compared with the audio content or may be further filtered based on further similarity and relationships. Additionally, other data culling techniques may be utilized. The subroutine may be performed for all data sets of one or more stored audio content files. The subroutine may also be performed for data sets below a pre-determined content files size. The subroutine may also determine the audio content to be idiosyncratic audio content and performing the other steps in response to the audio content being idiosyncratic audio content. The audio content may be compared to a pre-determined list of idiosyncratic terms comprising unusual words or phrases or other noteworthy characteristics, which if detected would activate the subroutine.

The audio analysis process3300may be operated multiple times. After each operation, a confidence value may be calculated that associated with the grade for each portion of the audio content (e.g., for each second). Each operation of the audio analysis process3300on the audio content may be averaged with the previous operations of the audio analysis process3300on the audio content. Once the confidence value is greater than a pre-determined threshold value, the audio analysis process3300is not operated on the audio content. The output controls may operate a machine display to display the plurality of grades for each of a plurality of segments of the audio content. The audio analysis process3300may then determine the speech semantic content and the speech vocal content associated with a change in the grade. The audio analysis process3300may determine those that exceed a threshold value of change.

Referring toFIG.34, the audio analysis process3400receives the audio content (block3402). The similar audio content is determined (block3404). The similar audio content is selected from one or more stored audio content files by comparing the audio content to the one or more stored audio content files. A predictive metric control is extracted from the similar audio content (block3406). The predictive metric control is sent as a feedback control to influence the creation of future audio records (block3408). The audio analysis process3400ends (done block3410).

The audio analysis process3400receives audio signals and generates controls to affect the machine state of one or more machines. The predictive metric control may be a grade of the audio content, the one or more machines comprising a machine display, the machine display altered to display the grade.

During determining the similar audio content from the one or more stored audio content files, the audio analysis process3400may utilize a tree structure to reduce the number of comparisons between the audio content and the one or more stored audio content files. The tree structure may comprise a indication of the similarity among the one or more stored audio content files. As the audio content is compared to one of the one or more stored audio content files, the remaining one or more stored audio content files are filtered based on their relationship to the one compared to the audio content. The other one or more stored audio content files may be filtered if the comparison indicates similarity and the relationship indicates dissimilarity or the comparison indicates dissimilarity and the relationship indicates similarity. The unfiltered one or more stored audio content files may be compared with the audio content or may be further filtered based on further similarity and relationships. Additionally, other data culling techniques may be utilized.

The audio analysis process3400may be performed for all data sets of one or more stored audio content files. The audio analysis process3400may also be performed for data sets below a pre-determined content files size. The audio analysis process3400may also determine the audio content to be idiosyncratic audio content and performing the other steps in response to the audio content being idiosyncratic audio content. The audio content may be compared to a pre-determined list of idiosyncratic terms comprising unusual words or phrases or other noteworthy characteristics, which if detected would activate the subroutine.

The audio analysis process3400may be operated multiple times. After each operation, a confidence value may be calculated that associated with the grade for each of a plurality of segments of the audio content (e.g., for each second). Each operation of the audio analysis process3400on the audio content may be averaged with the previous operations of the audio analysis process3400on the audio content. Once the confidence value is greater than a pre-determined threshold value, the audio analysis process3400is not operated on the audio content. The output controls may operate a machine display to display the plurality of grades for each of a plurality of segments of the audio content. The audio analysis process3400may then determine the speech semantic content and the speech vocal content associated with a change in the grade. The audio analysis process3400may determine those that exceed a threshold value of change.

Referring toFIG.35, the altered machine display3500comprises a first metric3502, a second metric3504, a third metric3506, a first scale3508, a second scale3510, a third scale3512, a first grade indication3514, a second grade indication3516, a third grade indication3518, a first grade3520, a second grade3522, and a third grade3524.

The altered machine display3500receives a model control structure (or weighted model control structure) and is altered to display one or more grades (i.e., the first grade3520, the second grade3522, and the third grade3524).

The first metric3502, the second metric3504, and the third metric3506indicate what the model control structure is measuring. The first scale3508, the second scale3510, and the third scale3512indicate the range of the grades. The scale may be 1-5, a percentage, a binary “yes or no”, etc. The first grade indication3514, the second grade indication3516, and the third grade indication3518depict where the grade is located on the respective scale. The first grade3520, the second grade3522, and the third grade3524depict the output of the audio analysis system2700based on an audio content.

Referring toFIG.36, the altered machine display3600comprises a grade indication3602, a fifth segment detail3604, and a tenth segment detail3606.

The altered machine display3600depicts a plurality of grades associated with a plurality of segments of an audio content. Each segment may be a time interval of the audio content, and each time interval may have the same or different duration. The grade indication3602shows the grade for each segment. The fifth segment detail3604and the tenth segment detail3606may be displayed either when activated by an input from an input device, such as a computer mouse, touch screen, audio control, etc. The fifth segment detail3604and the tenth segment detail3606may be automatically display in response to the change in the grade being greater than a pre-determined grade threshold value. The fifth segment detail3604and the tenth segment detail3606may depict the grade (e.g., 2.3 and 4.2, respectively) and semantic content associated with the segment (e.g., “You have the wrong widget.” and “Thank you for helping me.”, respectively). Other speech semantic or vocal content may be displayed.

Software Implementations

The systems disclosed herein, or particular components thereof, may in some embodiments be implemented as software comprising instructions executed on one or more programmable device. By way of example, components of the disclosed systems may be implemented as an application, an app, drivers, or services. In one particular embodiment, the system is implemented as a service that executes as one or more processes, modules, subroutines, or tasks on a server device so as to provide the described capabilities to one or more client devices over a network. However the system need not necessarily be accessed over a network and could, in some embodiments, be implemented by one or more app or applications on a single device or distributed between a mobile device and a computer, for example.

In a particular embodiment, the call flow and node components previously described are implemented within and by services of a cloud computer system.

Referring toFIG.37, a client server network configuration3700depicts various computer hardware devices and software modules coupled by a network3702in one embodiment. Each device includes a native operating system, typically pre-installed on its non-volatile RAM, and a variety of software applications or apps for performing various functions.

The mobile programmable device3704comprises a native operating system3706and various apps (e.g., app3708and app3710). A computer3712also includes an operating system3714that may include one or more library of native routines to run executable software on that device. The computer3712also includes various executable applications (e.g., application3716and application3718). The mobile programmable device3704and computer3712are configured as clients on the network3702. A server3720is also provided and includes an operating system3722with native routines specific to providing a service (e.g., service3724and service3726) available to the networked clients in this configuration.

As is well known in the art, an application, an app, or a service may be created by first writing computer code to form a computer program, which typically comprises one or more computer code sections or modules. Computer code may comprise instructions in many forms, including source code, assembly code, object code, executable code, and machine language. Computer programs often implement mathematical functions or algorithms and may implement or utilize one or more application program interfaces.

A compiler is typically used to transform source code into object code and thereafter a linker combines object code files into an executable application, recognized by those skilled in the art as an “executable”. The distinct file comprising the executable would then be available for use by the computer3712, mobile programmable device3704, and/or server3720. Any of these devices may employ a loader to place the executable and any associated library in memory for execution. The operating system executes the program by passing control to the loaded program code, creating a task or process. An alternate means of executing an application or app involves the use of an interpreter (e.g., interpreter3728).

In addition to executing applications (“apps”) and services, the operating system is also typically employed to execute drivers to perform common tasks such as connecting to third-party hardware devices (e.g., printers, displays, input devices), storing data, interpreting commands, and extending the capabilities of applications. For example, a driver3730or driver3732on the mobile programmable device3704or computer3712(e.g., driver3734and driver3736) might enable wireless headphones to be used for audio output(s) and a camera to be used for video inputs. Any of the devices may read and write data from and to files (e.g., file3738or file3740) and applications or apps may utilize one or more plug-in (e.g., plug-in3742) to extend their capabilities (e.g., to encode or decode video files).

The network3702in the client server network configuration3700can be of a type understood by those skilled in the art, including a Local Area Network (LAN), Wide Area Network (WAN), Transmission Communication Protocol/Internet Protocol (TCP/IP) network, and so forth. These protocols used by the network3702dictate the mechanisms by which data is exchanged between devices.

Machine Embodiments

FIG.38depicts a diagrammatic representation of a machine3800in the form of a computer system within which logic may be implemented to cause the machine to perform any one or more of the functions or methods disclosed herein, according to an example embodiment.

Specifically,FIG.38depicts a machine3800comprising instructions3802(e.g., a program, an application, an applet, an app, or other executable code) for causing the machine3800to perform any one or more of the functions or methods discussed herein. For example the instructions3802may cause the machine3800to implement the call flow control structures1400, call flow control nodes1500, call prioritization process1800, and particular call control nodes (queue node configuration1600, bot node configuration1700etc.). The instructions3802configure a general, non-programmed machine into a particular machine3800programmed to carry out said functions and/or methods.

In alternative embodiments, the machine3800operates as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine3800may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine3800may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions3802, sequentially or otherwise, that specify actions to be taken by the machine3800. Further, while only a single machine3800is depicted, the term “machine” shall also be taken to include a collection of machines that individually or jointly execute the instructions3802to perform any one or more of the methodologies or subsets thereof discussed herein.

The machine3800may include processors3804, memory3806, and I/O components3808, which may be configured to communicate with each other such as via one or more bus3810. In an example embodiment, the processors3804(e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an ASIC, a Radio-Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, one or more processor (e.g., processor3812and processor3814) to execute the instructions3802. The term “processor” is intended to include multi-core processors that may comprise two or more independent processors (sometimes referred to as “cores”) that may execute instructions contemporaneously. AlthoughFIG.38depicts multiple processors3804, the machine3800may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.

The memory3806may include one or more of a main memory3816, a static memory3818, and a storage unit3820, each accessible to the processors3804such as via the bus3810. The main memory3816, the static memory3818, and storage unit3820may be utilized, individually or in combination, to store the instructions3802embodying any one or more of the functionality described herein. The instructions3802may reside, completely or partially, within the main memory3816, within the static memory3818, within a machine-readable medium3822within the storage unit3820, within at least one of the processors3804(e.g., within the processor's cache memory), or any suitable combination thereof, during execution thereof by the machine3800.

The I/O components3808may include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I/O components3808that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones will likely include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I/O components3808may include many other components that are not shown inFIG.38. The I/O components3808are grouped according to functionality merely for simplifying the following discussion and the grouping is in no way limiting. In various example embodiments, the I/O components3808may include output components3824and input components3826. The output components3824may include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. The input components3826may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and/or force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), one or more cameras for capturing still images and video, and the like.

Communication may be implemented using a wide variety of technologies. The I/O components3808may include communication components3836operable to couple the machine3800to a network3838or devices3840via a coupling3842and a coupling3844, respectively. For example, the communication components3836may include a network interface component or another suitable device to interface with the network3838. In further examples, the communication components3836may include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components to provide communication via other modalities. The devices3840may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).

Instruction and Data Storage Medium Embodiments

The various memories (i.e., memory3806, main memory3816, static memory3818, and/or memory of the processors3804) and/or storage unit3820may store one or more sets of instructions and data structures (e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions3802), when executed by processors3804, cause various operations to implement the disclosed embodiments.

Some aspects of the described subject matter may in some embodiments be implemented as computer code or machine-useable instructions, including computer-executable instructions such as program modules, being executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program modules including routines, programs, objects, components, data structures, etc., refer to code that perform particular tasks or implement particular data structures in memory. The subject matter of this application may be practiced in a variety of system configurations, including hand-held devices, consumer electronics, general-purpose computers, more specialty computing devices, etc. The subject matter may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.

Communication Network Embodiments

The instructions3802and/or data generated by or received and processed by the instructions3802may be transmitted or received over the network3838using a transmission medium via a network interface device (e.g., a network interface component included in the communication components3836) and utilizing any one of a number of well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions3802may be transmitted or received using a transmission medium via the coupling3844(e.g., a peer-to-peer coupling) to the devices3840. The terms “transmission medium” and “signal medium” mean the same thing and may be used interchangeably in this disclosure. The terms “transmission medium” and “signal medium” shall be taken to include any intangible medium that is capable of storing, encoding, or carrying the instructions3802for execution by the machine3800, and/or data generated by execution of the instructions3802, and/or data to be operated on during execution of the instructions3802, and includes digital or analog communications signals or other intangible media to facilitate communication of such software. Hence, the terms “transmission medium” and “signal medium” shall be taken to include any form of modulated data signal, carrier wave, and so forth. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a matter as to encode information in the signal.

LISTING OF DRAWING ELEMENTS

“Active call” refers to a call that is in progress and has not yet ended.

“Agent” refers to a system component that monitors the content of calls and responds to that content by taking some action. Agents may be automated (chat bots, automated voice attendants), may be human, or a combination of automation and human (e.g., at ender nodes).

“Algorithm” refers to any set of instructions configured to cause a machine to carry out a particular function or process.

“Anomaly detector” refers to logic that analyzes call features and/or call metrics to identify patterns or values indicative of conditions that are out of a configured normal range (e.g., for one or more particular emotion metrics). This may include idiosyncratic call (e.g., audio) content.

“App” refers to a type of application with limited functionality, most commonly associated with applications executed on mobile devices. Apps tend to have a more limited feature set and simpler user interface than applications as those terms are commonly understood in the art.

“Application” refers to any software that is executed on a device above a level of the operating system. An application will typically be loaded by the operating system for execution and will make function calls to the operating system for lower-level services. An application often has a user interface but this is not always the case. Therefore, the term ‘application’ includes background processes that execute at a higher level than the operating system.

“Application program interface” refers to instructions implementing entry points and return values to a module.

“Assembly code” refers to a low-level source code language comprising a strong correspondence between the source code statements and machine language instructions. Assembly code is converted into executable code by an assembler. The conversion process is referred to as assembly. Assembly language usually has one statement per machine language instruction, but comments and statements that are assembler directives, macros, and symbolic labels may also be supported.

“Associator” refers to a Correlator (see the definition for Correlator).

“Audio content” refers to a sound signal or recording comprising speech.

“Automated voice attendant” refers to logic that generates audio to a caller to solicit caller actions, and interprets and acts upon caller actions such as spoken words or phrases or tones.

“Call” refers to any communication session conducted over machine networks. Calls can include analog voice calls, digital (e.g., IP) calls, chat sessions, and email conversations.

“Call flow” refers to a collection of linked control structures in a machine system. Communication sessions in the form of audio calls, data calls (e.g., chat) etc. are routed between the control structures (nodes) and for some types of nodes, are queued for processing within the nodes according to a call priority algorithm operated on the queue.

“Call flow node” refers to a call routing or call operation structure in a call flow.

“Call queue” refers to a structure to delay received calls while they await processing.

“Combiner” refers to a logic element that combines two or more inputs into fewer (often a single) output. Example hardware Combiners are arithmetic units (adders, multipliers, etc.), time-division multiplexers, and analog or digital modulators (these may also be implemented is software or firmware). Another type of Combiner builds an association table or structure (e.g., a data structure instance having members set to the input values) in memory for its inputs. For example: val1, val2, val3->Combiner logic->{val1, val2, val3} set.val1=val1; set.val2=val2; set.val3=val3; Other examples of Combiners will be evident to those of skill in the art without undo experimentation.

“Comparator” refers to a logic element that compares two or more inputs to produce one or more outputs that reflects similarity or difference of the inputs. An example of a hardware Comparator is an operational amplifier that outputs a signal indicating whether one input is greater, less than, or about equal to the other. An example software or firmware Comparator is: if (input1==input2) output=val1; else if (input1>input2) output=val2; else output=val3; Many other examples of Comparators will be evident to those of skill in the art, without undo experimentation.

“Compiled computer code” refers to object code or executable code derived by executing a source code compiler and/or subsequent tools such as a linker or loader.

“Compiler” refers to logic that transforms source code from a high-level programming language into object code or in some cases, into executable code.

“Computer code” refers to any of source code, object code, or executable code.

“Computer code section” refers to one or more instructions.

“Computer program” refers to another term for ‘application’ or ‘app’.

“Confidence value” refers to the frequency (i.e., the proportion) of confidence intervals that contain the true value of their corresponding parameter.

“Correlator” refers to a logic element that identifies a configured association between its inputs. One examples of a Correlator is a lookup table (LUT) configured in software or firmware. Correlators may be implemented as relational databases. An example LUT Correlator is: |low_alarm_condition|low_threshold_value|0∥safe_condition|safe_lower_bound|safe_upper_bound∥high_alarm_condition|high_threshold_value|0|Generally, a Correlator receives two or more inputs and produces an output indicative of a mutual relationship or connection between the inputs. Examples of Correlators that do not use LUTs include any of a broad class of statistical Correlators that identify dependence between input variables, often the extent to which two input variables have a linear relationship with each other. One commonly used statistical Correlator is one that computes Pearson's product-moment coefficient for two input variables (e.g., two digital or analog input signals). Other well-known Correlators compute a distance correlation, Spearman's rank correlation, a randomized dependence correlation, and Kendall's rank correlation. Many other examples of Correlators will be evident to those of skill in the art, without undo experimentation.

“Driver” refers to low-level logic, typically software, that controls components of a device. Drivers often control the interface between an operating system or application and input/output components or peripherals of a device, for example.

“Executable” refers to a file comprising executable code. If the executable code is not interpreted computer code, a loader is typically used to load the executable for execution by a programmable device.

“Executable code” refers to instructions in a ready-to-execute form by a programmable device. For example, source code instructions in non-interpreted execution environments are not executable code because they must usually first undergo compilation, linking, and loading by the operating system before they have the proper form for execution. Interpreted computer code may be considered executable code because it can be directly applied to a programmable device (an interpreter) for execution, even though the interpreter itself may further transform the interpreted computer code into machine language instructions.

“File” refers to a unitary package for storing, retrieving, and communicating data and/or instructions. A file is distinguished from other types of packaging by having associated management metadata utilized by the operating system to identify, characterize, and access the file.

“Grade” refers to a valuation of an aspect of an audio content. Call metrics are a type of grade.

“Idiosyncratic audio content” refers to audio content that is dissimilar to the content utilized to train a model.

“Inherent queue tag” refers to settings configured in a queue that are automatically applied as tags to calls that enter the queue.

“Instructions” refers to symbols representing commands for execution by a device using a processor, microprocessor, controller, interpreter, or other programmable logic. Broadly, ‘instructions’ can mean source code, object code, and executable code. ‘instructions’ herein is also meant to include commands embodied in programmable read-only memories (EPROM) or hard coded into hardware (e.g., ‘micro-code’) and like implementations wherein the instructions are configured into a machine memory or other hardware component at manufacturing time of a device.

“Interpreted computer code” refers to instructions in a form suitable for execution by an interpreter.

“Interpreter” refers to an interpreter is logic that directly executes instructions written in a source code scripting language, without requiring the instructions to a priori be compiled into machine language. An interpreter translates the instructions into another form, for example into machine language, or into calls to internal functions and/or calls to functions in other software modules.

“Library” refers to a collection of modules organized such that the functionality of all the modules may be included for use by software using references to the library in source code.

“Linker” refers to logic that inputs one or more object code files generated by a compiler or an assembler and combines them into a single executable, library, or other unified object code output. One implementation of a linker directs its output directly to machine memory as executable code (performing the function of a loader as well).

“Loader” refers to logic for loading programs and libraries. The loader is typically implemented by the operating system. A typical loader copies an executable into memory and prepares it for execution by performing certain transformations, such as on memory addresses.

“Machine language” refers to instructions in a form that is directly executable by a programmable device without further translation by a compiler, interpreter, or assembler. In digital devices, machine language instructions are typically sequences of ones and zeros.

“Metric control” refers to a signal generated as a metric and in which the metric value affects a type or amount of control applied to a system component, or if control is applied at all (e.g., binary or thresholded metric controls).

“Model control structure” refers to an output from a model for a specific audio content.

“Module” refers to a computer code section having defined entry and exit points. Examples of modules are any software comprising an application program interface, drivers, libraries, functions, and subroutines.

“Multi-modal weight” refers to a value applied to a model when utilized with other models.

“Object code” refers to the computer code output by a compiler or as an intermediate output of an interpreter. Object code often takes the form of machine language or an intermediate language such as register transfer language (RTL).

“Operating system” refers to logic, typically software, that supports a device's basic functions, such as scheduling tasks, managing files, executing applications, and interacting with peripheral devices. In normal parlance, an application is said to execute “above” the operating system, meaning that the operating system is necessary in order to load and execute the application and the application relies on modules of the operating system in most cases, not vice-versa. The operating system also typically intermediates between applications and drivers. Drivers are said to execute “below” the operating system because they intermediate between the operating system and hardware components or peripheral devices.

“Plug-in” refers to software that adds features to an existing computer program without rebuilding (e.g., changing or re-compiling) the computer program. Plug-ins are commonly used for example with Internet browser applications.

“Plurality of segments” refers to intervals of the audio content, each interval may or may not be equal in duration.

“Portion”, in the context of a call, refers to a sub-set (less than all) of the content of the call.

“Predictive metric control” refers to labels and annotations associated with a similar audio content.

“Priority response” refers to actions assigned an elevated priority in a priority hierarchy in a system.

“Process” refers to software that is in the process of being executed on a device.

“Programmable device” refers to any logic (including hardware and software logic) who's operational behavior is configurable with instructions.

“S model” refers to a fitting algorithm that determines one or more match metrics between calls and agents and/or nodes to service the calls. S models may include machine learning capability to improve the accuracy and/or efficiency of matching over time as more calls are processed. Specific S models are described herein, and one of ordinary skill in the art will appreciate that other models known in the art such as Support Vector Machine, perceptrons (neural networks), and statistical models may also be utilized.

“Selector” refers to a logic element that selects one of two or more inputs to its output as determined by one or more selection controls. Examples of hardware Selectors are multiplexers and demultiplexers. An example software or firmware Selector is: if (selection_control==true) output=input1; else output=input2; Many other examples of Selectors will be evident to those of skill in the art, without undo experimentation.

“Service” refers to a process configurable with one or more associated policies for use of the process. Services are commonly invoked on server devices by client devices, usually over a machine communication network such as the Internet. Many instances of a service may execute as different processes, each configured with a different or the same policies, each for a different client.

“Similar audio content” refers to audio content matching other audio content for some metric or vector or other measure of similarity.

“Software” refers to logic implemented as instructions for controlling a programmable device or component of a device (e.g., a programmable processor, controller). Software can be source code, object code, executable code, machine language code. Unless otherwise indicated by context, software shall be understood to mean the embodiment of said code in a machine memory or hardware component, including “firmware” and micro-code.

“Source code” refers to a high-level textual computer language that requires either interpretation or compilation in order to be executed by a device.

“Speech semantic content” refers to words spoken in an audio content.

“Speech vocal content” refers to characteristics, such as speech patterns, cadences, and tone, of an audio content.

“Sub-metric” refers to metrics used to generate other metrics.

“Subroutine” refers to a module configured to perform one or more calculations or other processes. In some contexts the term ‘subroutine’ refers to a module that does not return a value to the logic that invokes it, whereas a ‘function’ returns a value. However herein the term ‘subroutine’ is used synonymously with ‘function’.

“Tag” refers to a setting assigned to a call.

“Task” refers to one or more operations that a process performs.

“Template” refers to electronic forms, or configured action-response sequences or algorithms or models.

“Threshold analyzer” refers to logic to analyze metrics to determine if they meet a threshold value or range condition.

Various functional operations described herein may be implemented in logic that is referred to using a noun or noun phrase reflecting said operation or function. For example, an association operation may be carried out by an “Associator” or “Correlator”. Likewise, switching may be carried out by a “switch”, selection by a “Selector”, and so on.

Having thus described illustrative embodiments in detail, it will be apparent that modifications and variations are possible without departing from the scope of the invention as claimed. The scope of inventive subject matter is not limited to the depicted embodiments but is rather set forth in the following Claims.