Method for measuring quality of service in a telecommunications network

A method for assessing the quality of service of telephone calls on a trunk includes identifying and counting telephone call sequences including a plurality of telephone calls placed on the trunk. Each of the telephone calls in the sequence has the same originating telephone number and the same destination telephone number and a start time and a stop time and an elapsed time between the stop time of a telephone call and the start time of the next telephone call in a sequence being less than a predetermined amount. Identifying and counting unanswered reattempted telephone calls or answered reattempted telephone calls. A ratio is calculated between the number of identified unanswered or answered reattempted telephone calls to the number of identified telephone call sequences, such that the ratio provides a measure of the quality of service of telephone calls on the trunk.

TECHNICAL FIELD OF THE INVENTION

The present invention relates to telecommunications systems, and more particularly to a method for managing and measuring quality of service in telephony networks, such as those configured to transport and process cellular and conventional wireline calls.

BACKGROUND OF THE INVENTION

American and international telecommunications service providers today purchase termination services from a large number of carriers around the world. As previously centralized national telecommunications networks have opened to increasing competition, the number of network routes available for the termination of both domestic and international traffic has increased worldwide. This trend has been especially apparent since the mid-1990s during which time the deregulation of parts of American and other telecommunications networks engendered a dramatic increase in the number and quality of independent telecommunications carriers servicing the American market.

Additionally, foreign telecommunications markets have been deregulated and telecommunications markets have transitioned from the single government-owned incumbent carriers to a collection of multiple separate carriers.

This increased carrier penetration of different world telecommunications markets has given American telecommunications service providers a greater number of options for the termination of international telecommunications traffic than was previously available, along with lower costs of termination to many international destinations, especially in the developing world. Unfortunately, however, a growing number of the less costly options now available to American service providers for the termination of international traffic frequently fail to deliver a quality of service comparable to more costly established international carriers. Thus American and other telecommunications service providers interested in the cost savings associated with smaller, independent international carriers must guard against the potentially damaging impact of poorly qualified secondary routes to customer perceptions of quality.

Existing systems and methods to measure and manage quality of service are limited in several important ways. Some quality of service measurement and management methods equate network quality with a measure of Answer-Seize Ratio (ASR) either in part or in whole. However, experience has shown that small, independent carriers providing low-cost routes to the developing world may accidentally misrepresent or intentionally falsify release cause codes transmitted as part of the SS7 and ISDN telecommunications signaling protocols. For example, carrier substitution of a network congestion cause code in place of a user busy cause code may reclassify an unanswered call as a non-attempted call, thereby raising the measure of ASR for the carrier in question. Another limitation of quality of service measurements relates to ASR, PSC (percentage of short calls) and similar measures which express call quality as a ratio over total calls and can give inaccurate results over small samples. For example, a single caller repeatedly calling the same number to some low volume international destination may disproportionately raise or lower measures of ASR or PSC for the destination over a small sample of calls. As quality of service correction in realtime sometimes necessitates relatively small call samples, realtime quality correction may confound quality of service management systems predicated on ASR or PSC alone.

Thus, there exists a need to provide new and improved systems and methods that permit telecommunications service providers to measure and manage end-user perception of quality of service. A need has arisen to effectively address the needs of modem telecommunications service provides, in which such systems and methods must measure voice traffic quality of service to places in both the developed and developing worlds without reference to the special messaging parameters included in SS7, ISDN and other signaling protocols, and must correct for the skewing of repeated caller-to-caller dialing and must further afford comparison between different carriers terminating to the same destination.

SUMMARY OF THE INVENTION

A method for assessing the quality of service of telephone calls on a trunk in a telecommunications system is provided. The method includes identifying and counting telephone call sequences including a plurality of telephone calls placed on the trunk occurring at different times during a sample time period or in realtime. Each of the telephone calls in the sequence has the same originating telephone number and the same destination telephone number. Each of the telephone calls in the sequence has a start time and a stop time and an elapsed time between the stop time of a telephone call and the start time of the next telephone call in a sequence, the elapsed time being less than a predetermined amount. The method further includes identifying and counting unanswered reattempted telephone calls and answered reattempted telephone calls placed on the trunk occurring at different times during the sample time period or in realtime. Each of the telephone calls has the same originating telephone number and the same destination telephone number and placed in a sequence wherein the telephone call placed from the originating telephone number is not answered or answered at the destination number. A ratio between the number of identified unanswered reattempted telephone calls or identified answered reattempted telephone calls to the number of identified telephone call sequences is calculated, such that the ratio provides a measure of the quality of service of telephone calls on the trunk.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

GLOSSARYANI:the telephone number of the telephone originatingANRTS:a call reattempted, answered callsANRTS TKthe number of TK chains comprising at least oneChains:reattempted, answered callAnswer Supervisionequals the number of answered calls/callsRatio (ASR):Average Costequals the total outbound charges/total minutesPer Minute (ACM):Call Answerequals the number of answered reattempts/answeredReattempt Ratiocalls(ARR2):Calls a1different call recordsthrough an:Calls:a number of telephone callsCDR:call detail recordChain:an ordered sequence (a1, a2, . . . , an) of one ormore calls such that all calls share the sameoriginating telephone number (ANI); all callsshare the same destination telephone number(DEST); the start time of aiprecedes the starttime of i + 1, for all i; and the start timeof ai+ 1 minus the stop time of aiisless than the reattempt window, for all iDEST:the telephone number of the telephone at thedestination of a callDuration (DUR):a user-definable, number of seconds, such as,for example, 67 secondsINA5:the answered reattempt quality indicator and is equalto ANRTS TK chains/Known TK chainsINU5:the unanswered reattempt quality indicator and isequal to UNRTS TK chain/Known TK chainsKnown Pairs (KP):is a list of known (ANI, DEST) pairsKnow Targetis a list of chain Ids in which the target trunkChains (KTC):participatesPercent of Shortequals the number of calls of less than the specifiedCalls (PSC):duration, such as, for example 67 seconds orless/number of answered callsPSTN:public switched telephone networkQOS:quality of serviceQuality ofinclude, for example, PSC; INA5; INU5;Service Metrics:ARR2; and URR2Reattempt Windowis positive, integer number of seconds, and is user-(RW):definable, for calls of the same ANI and same DESTand separated by less than or equal to RWseconds to participate in the same call chainSample Size:is a variable, positive, integer representing thenumber of TK chainsTarget Place:is the geographic location of the destinationtelephone numberTarget Trunk:is the outgoing trunk under investigationTK Chains:is the number of elements in KTCTrunk Chainan ordered sequence (a1, a2, . . . , an) of one(TK Chain):or more telephone calls such that all callsbelong to the same chain and all callsshare the same outgoing trunkUnanswered Re-the unanswered reattempted calls/unanswered callsattempt Ratio(URR2):UNRTS:reattempted, unanswered callsUNRTS TK Chains:means the number of TK chains comprising atleast one reattempted, unanswered call

Referring toFIG. 1, a block diagram illustrating a telecommunications network for use with the present method for measuring quality of service is illustrated, and is generally identified by the numeral20. Network20includes a public switched telephone network22to which individual callers24are connected.

Network22is interconnected to a switching facility26. Facility26transmits call information in realtime to enable the present quality of service correction application28(to be subsequently described with respect toFIGS. 5-9) and writes call detail records to a call record database30for use with the present quality of service reporting application32(to be subsequently described with respect to FIGS.2-4). Call information is transmitted from switching facility26to correction application28and call record database30via a call control application34. Call control application34also interfaces to correction application28via a call control preferences36which allows for trunks on switching facility26to be put in and taken out of routing. Call control application34may comprise, for example, a Simplified OpenServices Platform sold by Simplified Development Corporation of Austin, TX. This platform is a call control package including a call control application, a call control preferences database and a call record database and includes application34, call record database30and call control preferences36. Call control application34functions to write CDRs to database30for purposes of billing and reconciliation. As previously stated, call control application34also provides stream CDRs in realtime, immediately upon completion of a call record, to application28.

Quality of service reporting application32is interconnected to a quality of service reporting preferences38which is connected to a reporting preferences user interface40. Reporting application32also connects to a quality of service report42which represents a visual report, either sent via email or published on a web site to provide quality of service metrics for trunks to destination numbers. A system administrator44utilizing reports42directs reporting preferences via interface40. Computer software for reporting application32, reporting preferences38, interface40and report42may reside on a single or separate server.

Quality of service correction application28interfaces to a quality of service correction preferences48which is connected to a user interface50for use by system administrator44. Correction application28, correction preferences48, and interface50may reside on a single or separate server. System administrator44also provides an input to a call preferences36for trunks on switching facility to be put in and taken out of routing.

Reporting application32at system configuration enters quality of service reporting preferences into reporting preferences38via interface40. Quality of service reporting preferences include report criteria, including, for example, destination numbers, number of trunks to be monitored and time periods to be monitored. Reporting preferences also include report meta data including, for example, run time, media (email, monitor html, archive html), delivery mechanism (email, web site) and recipients. At run time, reporting application32retrieves quality of service reporting preferences from reporting preference38, queries call record database30for call records as specified in reporting preference38and analyzes call records from call record database30for quality of service metrics. Reporting application32assembles quality of service metrics into a report42which then formats and delivers the report42as specified in reporting preferences38. At evaluation time, system administrator44evaluates the quality of service report42and optionally reconfigures reporting application32, control preferences36, or both.

Correction application28at configuration time, allows system administrator44to enter quality of service correction preferences into correction preferences48by interface50. Quality of service correction preferences include, for example, destination number(s) to monitor, trunk(s) to monitor, sample size, quality of service metric thresholds and corrective action(s). At run time, correction application28receives quality of service correction preferences from correction preferences48as entered by system administrator44. Correction application28collects call statistics from call control application34in realtime, as specified by correction preferences48. When correction application28collects a sample of calls sufficient in number according to the sample size specified in correction preferences48, correction application28analyzes collected call records from call control application34for quality of service metrics. The quality of service metrics include PCS, INA5 and INU5. If the quality of service metrics violate the quality of service metric threshold specified in correction preferences48, correction application28takes corrective action as specified in correction preferences48. These corrective actions include correction application28sending a notification email to system administrator44declaring quality violation of some trunk to some destination numbers and/or writing a routing exception to call control preferences36, thereby removing the offending trunk from routing to the violated destination numbers. At evaluation time, system administrator44reads notification email, if any, sent from correction application28and/or optionally reconfigures the quality of service correction subsystem, the quality of service reporting system, or both.

The present method defines action taken at configuration time, run time and evaluation time for reporting application32and correction application28. Reporting application32is further defined in connection withFIGS. 2-4. Correction application28is further defined with respect toFIGS. 5-9.

Referring now toFIGS. 2-4, the present method of identifying reattempted calls, assigning chain Ids and calculating the trunk chain statistics INA5 and INU5 at reporting time will now be described.FIG. 2illustrates the relationship betweenFIGS. 3 and 4.FIG. 3details the method of identifying all reattempted calls in a sample of CDRs and of assigning chain Ids to all chains in a sample of CDRs.FIG. 4details the method of calculating the measures INA5 and INU5 at reporting time once reattempts have been determined and chains Ids have been assigned. The functions Illustrated inFIGS. 2-4take place in reporting application32.

Referring toFIG. 2, two broad steps50and52are shown. At reporting time, which may or may not necessarily occur in realtime, but shortly after realtime or even weeks or months later, the quality performances of different telephone carriers is reported at the time in which the report is run. When reported, the first step performed is to derive the measurements INA5 and INU5 step52which are measures of caller reattempts. The two broad steps50and52first identify which calls in the sample were reattempts and then align different calls into chains (step50). After the reattempted calls are identified and assigned chain Ids, each call chain receives one unique Id, so the process of assigning chain Ids is the same as aligning calls into chains. Then at step52the measurements INA5 and INU5 are produced.

Referring toFIG. 3, the steps represented by step50, which is identifying reattempted calls and assigning chain Ids are illustrated. At step54, variables are initialized. The number of CDRs which are in the sample is denoted by the “n”. For example, n may have a value of 100,000 calls. Further in this example, the quantity of calls to Manila, Philippines will be determined. There are 100,000 calls in the sample that are being analyzed. A start counter i and the first available chain Id also are set at1. The steps inFIG. 3analyze each of the CDRs in the sample in order to find out which CDRs, if any, were reattempted. To begin that analysis, in step56all the CDRs in the sample are ordered by the four criteria given, (1) the originating number, (2) the destination number, (3) the start time and (4) stop time. The result of the step56is that all of the 100,000 calls going to Manila are ordered according to the four criteria in step56. An unordered sample is thereby ordered.

In step58, CDRi, is compared and with CDRi−1. The first CDR is compared and the CDR just before it in the sample. For the first time throughFIG. 3, looking at the first CDR there is nothing to compare it with. The second time throughFIG. 3, the second CDR is compared to the first CDR. The one hundredth time throughFIG. 3, the one hundredth CDR is compared to the 99thCDR, and so on until the 100,000th CDR is compared to the CDR just before it in the ordered sample. A decision is then made at step60to determine if the following three conditions all are true: does the CDRihave the same ANI as the CDRi−1just prior to the sample; does the CDRihave the same destination number as the CDRi−1just prior to the sample; and lastly does the CDRistart time follow the stop time of CDRiby fewer than the reattempt window (RW) number of seconds. The reattempt window is a positive integer value in seconds and is the amount of time after which one call terminates and another call begins. For example, party A calls party B and talk for a while and then for whatever reason get disconnected. Caller A calls Party B back and talk for a while longer. The second call, either counts as a reattempt call or it does not count as a reattempt call based on the number of seconds from the end of the first call to the start of the second call. That number of seconds is the reattempt window. A typical RW value is 150 seconds. The RW reflects that if two people are having a conversation and they reattempt the conversation within 2-½ minutes that there was a likely quality problem on the first call. Obviously, if two people were talking and they reattempt each other an hour later, the chances are considerably less that there is a quality problem.

After completing step60, if the two calls were sequential in the ordered sample of 100,000 CDRs, the CDR is counted as a reattempt and if so, that CDR is going to be that first CDR in the sample, that is CDRi−1CDR100, for example is compared to CDR99and if those CDRs share an ANI and share a DEST, and there are 150 seconds or fewer between them, then CDR99(CDRi−1) is labeled as a reattempt, the yes path is followed to step62and CDRi−1is marked as a reattempted call. CDRi−1is marked as a reattempted call, and then at step64, CDRichain Id is labeled the same as CDRi−1chain Id. CDRiand CDRi−1,99and100, will have the same chain Id which means that CDRs99and100are in the same call chain Being in the same call chain means that all CDRs in the same call chain have the same ANI and the same DEST and that consecutive CDRs in the call chain differ from each other at the start and stop times by a value less or equal to the reattempt window.

If the decision at step60is no, CDRs99and100are not in the same call chain, that is, CDR100does not indicate CDR99to be a reattempted call, then at step66, CDRi−1or CDR99is not marked as a reattempted call. At step68CDRior100is assigned the next available chain Id that has not yet been used. CDRiis assigned a different chain Id than CDRi−1such that CDRiandi−1are in two separate call chains.

Since CDRiandi−1do not have the same ANI or the same destination or differ from each other by greater by 150 seconds, CDRiand CDRi−1are not in the same call chain. The assignment of separate chain Ids to the different CDRs insures that differing CDRs enter into separate call chains.

In either case, whether the two CDRs in the sample are to be in a reattempt relationship (step64) or not (step68), the CDR is incremented at step70, the purpose of which is so that the next CDR in the ordered sample will be examined. At step72a decision is made to determine if i equals n, which is to see if the end of our sample has been reached. If i equals 100,000 in the example, then all 100,000 calls to the Philippines have been examined, no other calls are to be analyzed and the yes decision at step72is made to complete the process.

On the other hand, if not yet at the 100,000 sample size or the end CDR in the sample, the decision at step72is no and the next CDR is examined at step58. CDR101is compared with CDR100. The loop is repeated 100,000 times or n times, once for each of the CDRs in the sample. At the end ofFIG. 3, it is known that for every single CDR in the sample whether the CDR was reattempted later in time or not. Every CDR in the sample has been labeled as being part of exactly one chain. Every CDR will then have a chain Id and will then have a reattempt flag set to either true or false indicating whether the CDR is a reattempt or is not a reattempt, was reattempted or was not reattempted.

There can be any number of CDRs in a chain. A call chain may comprise but a single CDR. It is possible that a call chain will contain just CDR99, which means that CDR99was not reattempted. If different CDRs make up the links (metaphorical links) in the chain, it is perfectly possible to have a chain that has only 1 link. It is also possible to have a chain that has many links.

Referring now to FIG.4. When the steps ofFIG. 4are completed, the value of INA5 and INU5 is found for the sample of 100,000 calls to Manila, Philippines. At step80five different variables are initialized. The first three variables are lists, the last two variables are numbers. The first three variables are Known TK chains ANSRT TK chains, and UNSRT TK chains, known as trunk chains, answered reattempted trunk chains, and unanswered reattempted trunk chains. Known trunk chains are all of the trunk chains that have been analyzed after analyzing the sample of CDRs. The steps ofFIG. 3are completed until 100,000 CDRs to the Philippines have been analyzed. The steps ofFIG. 4analyze exactly the same 100,000 CDRs to the Philippines.FIGS. 3 and 4work together as a pair and all CDRs thatFIG. 3looks at,FIG. 4then turns around and also looks at it.FIG. 4takes the 100,000 CDRs thatFIG. 3has already looked at, looks through those CDRs again and this timeFIG. 4will go through and count each of the trunk chains thatFIG. 3identified. Every timeFIG. 3finds a trunk chain,FIG. 4knows about that trunk chain andFIG. 4adds the known trunk chain to the variable Known TK chain. Known TK chain is the process by whichFIG. 4tracks analyzed chains.

Referring again to step80, answered reattempt trunk chains and unanswered reattempt trunk chains means as follows: the answered reattempt trunk chain variables holds all the trunk chains in the sample, 100,000 calls, that have one answered reattempted call somewhere inside. That is to say, that if a chain that has 5 links in it, 5 CDRs,FIG. 4, will look at all 5 links in the chain and find out if any of the 5 links count as an answered reattempt. If any of the 5 links, 1 or more of the 5 links count as an answered reattempt, then the trunk chain whichFIG. 4is looking at will count as an answered. reattempted trunk chain. That answered reattempted trunk chain will go into a list “Answered reattempted trunk chain”. The variable unanswered reattempted trunk chain works similarly.FIG. 4will examine each of the trunk chains in the sample of 100,000 calls.FIG. 4will look at every link in each chain. Any link in the chain counts as an unanswered reattempted call then the containing trunk chain will count as an unanswered reattempted trunk chain and that trunk chain will enter into the variable unanswered reattempted trunk chain.

The last two variables in step80are i and n again which have the same meaning inFIG. 4as inFIG. 3. iis the counter that starts at 1, n is the number of CDRs in the sample, in this case 100,000. The target trunk Id is also input at step80.

At step82CDRiis examined. For example, if at CDR200, so i is equal to200,FIG. 4has passed through the loop 199 times and is now examining CDR200. A decision is made at step84whether the trunk Id of this CDR equals the trunk Id. The steps ofFIG. 4run on each outgoing trunk. That Is, if there is some trunk in the system, for example, trunk number610, and the administrator44wants to know how trunk610performed going to Manila, the steps ofFIG. 4are performed again. Finally, if the administrator44wants to see how a third trunk (trunk612) performed to Manila, the steps ofFIG. 4are performed a third time.

In the example of the 100,000 calls that went to Manila, the evaluation of trunk610is made. At step84the decision whether the trunk Id of CDR200equals the target trunk Id610is made. If the trunk Id of CDR200does not equal trunk Id610, in that case, the CDR is not of interest because the CDR is for a trunk that is not being analyzed. The next step is step86where 1 is incremental to201. At step88a decision is made to determine if i equals n. Since CDR201is not greater than 100,000, there are more CDRs to analyze, and the flow returns to step82to examine CDR201. CDR201is examined at step84to make the decision does the trunk Id of CDR201equal trunk610. If yes, the next decision is at step90, to decide if the chain Id of the CDR is in known trunk chains? CDR201has a chain Id because the CDR has already passed through flowchart ofFIG. 3, one of the purposes of which was to assign a chain Id to every CDR. In this example assume CDR201has a chain Id of1000. Step90determines if that chain Id1000is in the list Known TK chains which was initialized to be an empty list at the start of step80.

If200CDRs are passed through in the example it is possible that chain Id1000is already in the list of Known TK chains. If chain Id1000is in the list, the yes path is taken and i is incremented at step86. However, if that chain Id1000is not yet in the list of Known TK chains then at step92the CDR chain Id1000is inserted into Known trunk chains. Now the flowchart ofFIG. 4knows about chain Id1000and so chain Id1000goes to reside inside the variable, Known TK chain. At step94a decision is made whether CDR201was reattempted? It can be determined whether CDR201was reattempted or not, because of CDR201already passing through flowchartFIG. 3, one of the purposes of which was to mark each CDR as reattempted or not.

If the decision at step94is no, then i is incremented at step86. CDR201does not impact the measurement of INA5 or INU5. At step88the decision is made to see if CDR201is greater than 100,000, which it is not and, therefore the flow returns to step82to examine CDR202. Again, at step84the decision is made, does CDR202equal the target trunk of610? If yes, at step90a decision is made to determine if the chain Id of CDR202is in the Known TK chains. Assume that the chain Id is equal to1300and is inserted into the list of known chains at step92. At step94, assume that this CDR202was reattempted. In this case CDR202, which is part of chain1300was in fact reattempted. A decision is made at step96to decide whether the CDR202was answered or not.

CDRs are classified into classes as being either answered or unanswered. An answered CDR is an answered reattempted CDR and unanswered reattempted CDR is an unanswered unattempted CDR. Answered reattempted CDRs are used to compute INA5 which is a measure of the number of answered reattempted CDRs in the sample. Unanswered reattempted CDRs are used to produce the measure INU5 which is the measure of the number of unanswered reattempted CDRs in the sample. In this example for CDR202, the CDR202was answered, step96is yes and the next step is step98. At step98the decision is made to determine if the CDR chain Id is in the list of answered reattempted trunk chains. That is, does the identified chain1300contain at least 1 answered reattempt. If yes, i is incremental at step86, which means that CDR202does not cause the measure of answered reattempts to increase. On the other hand, at step98chain Id1300is not in the variable answered reattempted trunk chain, then that means for the first time chain1300has been identified as containing an answered reattempt and the decision at step98is no. Then at step100the CDR chain Id which is1300is inserted into the list variable answer reattempted trunk chain. Then at step86i is incremented to CDR203. CDR203is not the last call in the sample and the flow continues to step82to analyze CDR204.

Steps102and104perform similar functions to steps98and100to identify a CDR as an unanswered reattempt instead of an answered reattempt. The CDR is identified as an unanswered reattempt at step96, then a decision is made at step102to determine if the chain Id contains an unanswered reattempt. If not, it is entered into the chain Id of unanswered reattempted trunk chains at step104. In both cases i is incremented at step86and at step88and the decision is made to determine if the end of the sample is reached.

At the end ofFIG. 4in the example, the loop will have been made 100,000 times. The total number of trunk chains in the sample have been identified to obtain the total number of trunk chains in the sample which is the number of elements are in the variable Known TK chain. That is the length of variable Known TK chain. The length of the Known TK chains whenFIG. 4is completed equals the total number of trunk chains in the sample.

In the example of the 100,000 CDRs that have passed through to Manila for a time period of a month assume that there were 500 chains in which trunk610participated. Since the steps ofFIG. 4were performed only for the target trunk610, all other calls on the other trunks have not been analyzed. That would mean that there are 500 different numbers, 500 different chain Ids reside within the variable known TK chain. Answered reattempted trunk chains will have a length of some number of chain Ids inside of the variable answered reattempted trunk chain. The number of elements inside the variable answered trunk chain, which is the length of the variable answered reattempted trunk chain, tells how many of the 500 trunk chains contain at least one answered reattempt. There will be no more than 500 elements in the variable answered reattempted trunk chain. If there are only 500 trunk chains total, 500, or most likely much fewer than 500 of those chains, will contain an answered reattempt. In the example assume that answered reattempted trunk chain has 50 elements. That means that 50 of the 500 trunk chains has an answered reattempt. Lastly, the variable unanswered reattempted trunk chain will contain between 0 and 500 elements. Assume that unanswered reattempted trunk chains contain 100 elements. That means that 100 of the 500 trunk chains in the sample contained 1 unanswered reattempt. The calculation for the measures INA5 and INU5 can now be made. If known trunk chains is greater than 0, that means in the example that trunk610had 500 trunk chains, the length of known trunk chains definitely was greater than 0, in fact the length of known trunk chains was equal to 500. So since the length of known trunk chain in the example is greater than 0, then:INA5=length⁡(ANSRT⁢⁢TK⁢⁢chains)length⁡(Known⁢⁢TK⁢⁢chains)(1)
INA5 is equal to the length of the answered reattempted trunk chains divided by the length of Known TK chains. In the example, that is 50 answered reattempted trunk chains divided by 500 known trunk chains which results in INA5 equal to 10%.

If no single trunk chain for a trunk was found, INA5 cannot be determined and the result is not available, since the denominator of INA5 cannot be 0.

Lastly, the calculation for INU5 is done, assuming that there were some trunk chains, that is that the known trunk chain is grater than 0, then:INU5=length⁢⁢(UNSRT⁢⁢TK⁢⁢chains)length⁢⁢(Known⁢⁢TK⁢⁢chains)(2)
INU5 is equal to the length of the number of unanswered reattempted trunk chains divided by the total number of Known trunk chains. In the example, there are 100 unanswered reattempted trunk chains out of 500 total trunk chains, 100 divided by 500 or 20%. In a case where no trunk chains were found INU5 has no value. At the completion of the steps inFIG. 4, all 100,000 calls in the sample going to Manila will have been analyzed, calls in the sample that were for trunks other than the target trunk are disregarded and the number of chains in which the target trunk participated are found, the number of chains in which the trunk participated and contained an answered reattempt are found and the number of chains in which the trunk participated that contain an unanswered reattempt are found to produce the numbers INA5 and INU5 according to equations 1 and 2.

If there were 12 trunks that went to Manila in the sample month,FIG. 4would be processed 12 times, and INA5 and INU5 would be determined for all 12 trunks of the Philippines trunks in the sample month. The quality of each of the 12 trunks would be compared to each other to rank the 12 trunks relative to each other with regard to INA5 or INU5, or both.

FIGS. 5-9illustrate the realtime implementation of the method ofFIGS. 2-4. Referring toFIG. 5, correction application28(FIG. 1) is in a wait state110. If a new CDR is received, application28transitions from a wait state110into a process CDR state112, and the steps ofFIGS. 6-9are performed. Once application28has processed the new CDR, the CDR has been processed and correction application28transitions back into a wait state110. When correction application28is in a wait state110and correction application28receives a timer expiring, correction application28transitions into a process timer state114, which steps are included inFIG. 9Once the timer is processed correction application28transitions back into the wait state110. The correction application28spends most of its machine time waiting for a CDR to be received, or timer to expire. When either of those two event happen, application28begins the steps ofFIGS. 6-8. In realtime the system must not hold onto information longer than needed, clear timers are set in order to do housekeeping on memory management and remove elements from memory when they are no longer needed by application28. The timer ofFIG. 9expires to eliminate data in memory.

Referring toFIG. 6, variables are initialized including a variable Known Pairs (KP) which is a list in memory that holds all of the ANI, DEST pairs. Every CDR has an ANI and a DEST. The ANI is the originating telephone number and the DEST is the terminating telephone number. The combination of the originating telephone number and the destination telephone number is a pair of two telephone numbers which is observed in every CDR written. Different CDRs can have the same ANI/DEST pair. Therefore, an ANI/DEST paid does not uniquely identify a CDR, which is the basis for a chain having a series of CDRs not of the same ANI/DEST pair. When correction application28begins, it enters its wait state110. Correction application28does not know about any ANI/DEST pairs or about any CDRs that have passed through the switching system, but as the correction application28stays up and runs for hours and then days and then weeks, the correction application28learns about different ANI/DEST pair. Every time correction application28learns about a new ANI/DEST pair, correction application28writes that pair into the variable KP. KP is able to list all the ANI/DEST paid that correction application28knows about.

The first variable to be initialized is RW set at, for example 150 seconds. Variable, KTC, Known Trunk Chain lists all of the trunk chains that correction application28knows about in a way that is similar to how Known Pairs lists all of the ANI/DEST pairs that correction application28knows about.

FIG. 6also initializes three counters: answered reattempt counter, ANRTS, unanswered reattempt counters, UNRTS, and a trunk chain counter, TK Chains. The counter unanswered reattempts and trunk chains inFIG. 6have the similar purpose to the variables in FIG.4. The purpose of the three counters inFIG. 6is to count the number of answered reattempts, unanswered reattempts and trunk chains thatFIG. 6learns about as correction application28runs. At the conclusion of the realtime analysis, ratios are created equal to answered reattempts over trunk chains, and unanswered reattempts over trunk chains, which are exactly the INA5 and INU5 ratios. There are three flags used inFIG. 8, WI, WCand RR. WIis an internal warning flag, WCis a carrier warning flag and RRis a route removal flag. Each flag has a value that can be set either to zero or one, true or false. If the internal warning flag is set to true, when a trunk has had too many reattempts, correction application28can send email or text messaging to administrator44to identify a faulty trunk in the system. Also, correction application28can send an email to the carrier responsible for the faulty trunk and indicate the trunk is faulty and action is needed because application28intends to stop routing. Finally correction application28can remove the faulty trunk from routing. Whether or not the correction application28takes any of those three actions is determined by whether any of the three flags are set to true. If the internal warning flag is set to true, an email is sent to internal personnel to warn them; the the carrier warning flag is set to true, an email is sent to the carrier to warn them; and if the route removal flag is set to true, the faulty trunk is removed.

Referring toFIG. 6, the incoming CDR is denoted at step120as CDR1. A decision is made at step122whether there exists some CDR0that is already in KP. If a CDR has been received, and correction application28has been running for awhile, is there already another CDR and KP, the Known Pairs that are known, such that the ANI/DEST of CDR1are equal to the ANI/DEST of CDR0. If the decision is no, there is no CDR0in the data structure Known Pairs equals the ANI/DEST of CDR1, then at step124CDR1is assigned a chain Id. CDR1represents the start of brand new chain. The assignment is made because no calls are known about from the ANI/DEST pair. CDR1is assigned the next available chain Id at step124, for example, the chain ID here is set to 40000. It is the 4000thchain that correction application28has discovered since start up. After labeling CDR with chain Id4000, at step126, CDR1is inserted into the list of Known Pairs. At step126the timer114(FIG. 5) is attached to CDR1KP. The time is set at, for example, four hours and if a subsequent call is not made within that time period having the same ANI/DEST, CDR1is removed from KP.

After CDR1is inserted into KP and attach a timer to that CDR in KP, a decision is made at step128to determine if the outgoing trunk of CDR1is the target trunk, for example trunk610and if the outgoing destination is in the target place. If the trunk is the target trunk610, but the target place is not Manila, the answer is no and processing is over for CDR1. None of the counters are incremented, none of the variables show that there are reattempts going to the Philippines.

FIG. 6starts again, now with a call to Manila. The incoming CDR denoted by CDR1at step120; and at step122the decision is made to see if any calls form ANI/Manila are already known. Assuming not, at step124, CDR1is assigned a chain Id, for example4700. CDR1is inserted into KP and a timer is attached. At step128a decision is made to determine if the trunk of the CDR1is target trunk610, the target trunk and if the DEST is the target place of Manila. This time the result is yes and at step130CDR1's chain Id is inserted into the list of Known Trunk Chain. The chain Id was4700. Therefore, trunk610, which is the target trunks, participates in chain4700. At step130variable TK chain is incremented. At step132a decision is made to determine if the number of trunk chains that are Known, TK chains, is that equal to the sample size? The sample size is a predetermined number of trunk chains, for example 50. Correction application28waits until 50 trunk chains are received. In the present example, only one trunk chain has been received because the variable TK chains is equal to one, so there are not yet enough trunk chains in the sample to be confident in making a quality decision. The decision at step132is no and the process returns to step120.

Starting again at step120with the third call in the present example, a call to a different DEST in Manila at step122assumes the decision is no. At step124a new chain Id is assigned, for example chain Id5100. The chain Id is inserted into Known Pairs list and a timer is attached at step126. At step128the decision is made to determined if this call went out on target trunk610and if the call went to the target place, Manila. Assuming yes, at step130, the chain Id is inserted into a Known Trunk Chain, so Known Trunk Chains now has a chain Id4700, from our previous call and5100and other chains because there are other CDRs processed in between the second and third calls in the example. At step130trunk chains is incremented and trunk chains now increments from, for example, 49 to 50, so that correction application28now knows about 50 trunk chains. At step132the decision is made that trunk chain does equal the sample size and the process continues to FIG.8.

Referring now toFIG. 8, correction application28now has a sample of 50 trunk chains. At step140the system is going to set INA5 equal to the number of answered reattempts divided by the number of trunk chains and INU5 equal to the number of unanswered reattempts divided by the number of trunk chains.

Values for INA5 and INU5 are determined at the beginning ofFIG. 8because by passing throughFIG. 6over and over and over again, 50 trunk chains have been accumulated. Every time a trunk chain with an answered reattempt was found, the answered reattempts counter was incremented. Every time a trunk chain contained an unanswered reattempt, the counter unanswered reattempt was incremented. And, so by the time step140is reached, all three of the counters, answered reattempts, unanswered reattempts and trunk chains will have values and will be able to produce ratios for those values and assign those ratios to the values INA5 and INU5. INA5 and INU5 each have a quality threshold value for example 30% meaning that if any one sample trunk chain is greater than 30%, the trunk is faulty. At step142a decision is made to compare the computed value of INA5 for the sample 50 trunks chains to the quality threshold that is set up at configuration time for the system. If the decision is yes, INA5 computed for the sample of 50 chains, is greater than or equal to the INA5 threshold, then at step144, flat WIis checked. If WIis set, a quality of service warning email or text message is sent to personnel at step146. The message says trunk610going to Manila had 38% of its trunk chains reattempted in the sample. A faulty trunk exists because the value if INA5 is greater than 30%.

At step148flag WCis checked. If flag WCis set, a quality of service warning email is sent to the carrier responsible for trunk610at step150. This message to the carrier states that trunk610is not performing well going to Manila, and the system is either going to take it out, or it has already been taken out. At step152flag RRis checked. If the route removal flag is set at step154the target trunk at610is removed from service from routing to the target place, which is Manila.

The decision is then made at step156for this sample of 50 trunk chains, is INU5 greater than or equal to the INU5 threshold? If yes, the same sequence of events occurs. If an offending value is found, that INU5 surpasses the INU5 threshold, an internal email, send a carrier email, or remove this trunk from routing according to whether the WI, WCor RRare set at steps158to168. It is possible for one trunk, like trunk610for example, to generate two different warning emails, or multiple different warning emails, to the carrier, identifying that in the sample there were too many answered reattempts, and also too many unanswered reattempts.

At step170the counter variable are reset. Answered reattempts is set to zero. Unanswered reattempts is set to zero. Trunk chains is set to zero. The list of Known Trunk Chains, KTC, is cleared. At this point in the process, all 50 trunk chains were collected. A count of how many of those trunk chains had an answered reattempt, an unanswered reattempt was made, INA5 and INU5 were computed and compared to a threshold to see if those metrics were offensive. If so, correction action was taken by possibly talking to the carrier, and possibly by removing the trunk from routing.

Returning now toFIG. 6, CDR1now designates a call to a third DEST in Manila on trunk610. At decision step122the decision is now yes and a decision is now made at step172to find out whether CDR1's start value minus CDR0's stop value is less than or equal to our RW, which is the reattempt window. This decision is asking to determine whether this third call to Manila ended 150 seconds or less from the second call. If the answer is yes then the CDR0that was found in KP and the CDR1that has just now entered the system for processing are in a chain together. Both CDRs have the same ANI and same DEST and they are separated by fewer than 150 seconds or fewer than the configurable number of seconds listed as the reattempt window. If the decision is no, meaning the time difference between the two CDRs, CDR1and CDR0, were not close together in time, then step174is performed and the following steps124-132.

If the decision at step172is yes, the steps ofFIG. 7Step176are now performed to determine if CDR0was an answered reattempt or an unanswered reattempt. At step120(FIG. 6) the CDR that is received is denoted as CDR1. That is the CDR that is being analyzed. But in analyzing that CDR it was found out that a call had been made earlier, fewer than 150 second prior, and because of that discovery the focus of the analysis inFIG. 7shifts away from CDR1and toward CDR0.FIG. 7relates only to CDR0and not CDR1. When CDR1is received, the second call is the event that tells the system that the CDR0that was already present was reattempted. That is a crucial observation because the system cannot tell by analyzing one CDR in isolation, like CDR0if one looks at CDR0, it cannot be determined if CDR0was reattempted or not. The system must wait and see if another CDR going to the same place happens a short amount of time later. And, if that second CDR happens a short amount of time later, then one can be confident that CDR0was reattempted. CDRs are compared to each to find out whether any one CDR was, in fact, reattempted. That explains why in the example when CDR1is received, it is necessary to find out that an earlier call to the same DEST, represented by CDR0was made. Interest shifts to CDR0, because now for the first time it is known that CDR0was reattempted.

Referring toFIG. 7, decisions are made at step180. Is CDR0's outgoing trunk going out trunk610, the target trunk. Also, is CDR0's destination number in the target place, which is Manila? If the decision is no, even though this is a call to Manila, CDR0did not go out trunk610, it went out trunk777. In that case, CDR0did not go out the trunk being monitored, so there is no interest in CDR0. The flow therefore returns toFIG. 6, step176. CDR0was a reattempt, but not on one of the trunks being monitored, in this example trunk610.

If the decision at step180is yes, CDR0did go down trunk610, and CDR0also did go to the target place, which is Manila, then a decision is made at step182to determine if CDR0was answered. Was the call at the DEST, Manila, answered. If the caller did pick up, step184is processed to decide if the ANI, DEST and the chain Id of the CDR already counted as an answered reattempted call. If the decision is yes, meaning that the call is in a pattern of multiple calls to the same DEST which are each answered, and should not be counted as an answered reattempt, the increment counter at step186is bypassed and processing returns to step176in FIG.6. If the decision at step184is no, answered reattempted call counter is incremented at step186and the process returns to step176(FIG.6). In this manner, reattempted calls are not over-inflated due to calls placed over and over to the same DEST.

If the decision at step182is no, a decision is made at step188, similar to that at step184for unanswered reattempts to determine if the ANI, DEST and chain may be counted as an unanswered reattempted call. If that chain does not yet have an unanswered reattempt, the counter is incremented at step190. If the chain already has unanswered reattempts, the yes path is taken to return to step176(FIG.6). Therefore, a chain, no matter how long it is, even if it has several calls on a chain, the chain can only count as one bad mark against the trunk for answered reattempts and it can count as only one bad mark against the trunk for unanswered reattempts.

Referring again toFIG. 6, at step200, CDR1is assigned a chain Id, it would be the same as CDR0's chain Id. In moving fromFIG. 7back toFIG. 6the focus of the analysis leaves CDR0behind and goes back to CDR1, and what chain Id does CDR1have. CDR1is in the same chain as CDR0, so CDR1inherits CDR0's chain Id. Having already processed CDR0, at step202, CDR0is removed from Known Pairs. At step204CDR1is inserted into the Known Pairs and a timer is attached to CDR1so that CDR1cannot remain in the system for a time greater than for example four hours or for the configurable value of the time. A decision is then made at step206to determine if CDR1's out trunk is equal to the target trunk, does CDR1go out on610and did CDR1's destination number go to Manila. If not, processing is complete. If the decision at step206is yes,a decision is made at step208to determine if CDR1's chain Id is already in the Known Truck Chain. If it is, processing is complete. If CDR1's chain Id is not then the chain Id of CDR1it is inserted into the Known Trunk Chain at step130. That means that the number of truck chains that are known increments by one, and at step132the decision is made to determine whether the sample size of 50 trunk chains have been found. If 50 truck chains have been found the analysis previously described inFIG. 8is done.

Referring toFIG. 9, timer received steps220is performed at the completion of step170(FIG. 8) and the CDR in KP is removed corresponding to timer at step222to remove the timer for a CDR.

Other alteration and modification of the invention will likewise become apparent skill in the art upon reading the present disclosure, and it is intended that the scope of the invention disclosed herein be limited only by the broadest interpretation of the appended claims to which the inventor is legally entitled.