Patent Publication Number: US-11641419-B2

Title: Computer-generated speech detection

Description:
This application claims the benefit of UK patent application 2010427.9 filed 7 Jul. 2020 which is incorporated by reference herein. 
     This invention relates to managing incoming voice calls. Particular examples relate to the detection of computer-generated synthetic ‘voice’ calls to Interactive Voice Response (IVR) platforms using synthetic voice detection techniques. 
     BACKGROUND 
     Synthetic speech is audio data generated by computers and designed to sound to the human ear like a real human voice. Robotic speech is computer generated speech which is not designed to sound to the human ear like a real human voice, i.e. it is clearly computer generated or ‘robotic’ in nature. Robocalls are unsolicited computer-generated calls that may be marketing based, fraud based or simply nuisance calls. Robocalls are a type of voice call which may use either synthetic or ‘robotic’ speech. They are a major problem, with a US estimate putting Robocalls at over four billion per month in 2017. Current techniques rely on black-lists of blocked numbers, which become obsolete, and require updating, very quickly; or individual network detection of suspicious numbers which are also largely ineffective as well as being limited to single networks. These techniques are also designed for individual subscribers. 
     Interactive Voice Recognition (IVR) platforms allow users to interact with a host system via a telephone keypad or by speech recognition, allowing services to be inquired about through IVR dialogue. IVR platforms are designed to handle large volumes of incoming calls as well as having the capability to be used for outgoing calls. As such, computer-generated phone calls, often referred to as “robocalls” which use an auto-dialler to deliver a pre-recorded message can significantly slow down IVR platforms: a channel is tied up with the robocall, meaning it is unavailable for a genuine caller. Additionally, the robocall, being computer-generated, cannot follow the IVR prompts and therefore cannot choose correct workflow paths, for example, press or speak “one” for Sales. This in turn can lead to the IVR workflow having no choice but to hand the call over to a live agent. This is a problem because occupying a channel has an associated cost and occupying up an agent incurs even greater cost; robocalls waste both money and resources by occupying channels. Another problem is that robocalls skew abandonment rate statistics because automated calls are likely to finish with no result. It is therefore desirable to detect and terminate robocalls quickly, in order to limit the resources of the IVR platform which are used. 
     BRIEF SUMMARY OF THE DISCLOSURE 
     It is an aim of certain embodiments of the invention to solve, mitigate or obviate, at least partly, at least one of the problems and/or disadvantages associated with the prior art. Certain embodiments aim to provide at least one of the advantages described below. 
     According to a first aspect, there is provided a method for managing incoming voice calls, the method comprising:
         receiving audio of a voice call;   determining whether the audio of the voice call is computer-generated; and   terminating the voice call if it is determined that the audio of the voice call is computer-generated.       

     In an embodiment, the determining whether the audio of the voice call is computer-generated comprises calculating a score indicating a probability that the audio of the voice call is computer-generated; and
         wherein, when the score indicating the probability that the voice call is computer-generated is greater than a predefined threshold, the voice call is terminated.       

     In an embodiment, the determining whether the audio of the voice call is computer-generated comprises detecting silence in the audio indicating a human caller. 
     In an embodiment, the determining whether the audio of the voice call is computer-generated comprises detecting Dual Tone Multi-Frequency, DTMF, tones in the audio indicating a human caller. 
     In an embodiment, the method further comprises auto-dialling the number of the received voice call to determine whether the call is answered if the voice call is terminated;
         wherein the call being answered indicates that the voice call was made by a human caller.       

     According to a second aspect, there is provided an electronic device configured to:
         receive audio of a voice call;   determine whether the audio of the voice call is computer-generated; and   terminate the voice call if it is determined that the audio of the voice call is computer-generated.       

     In an embodiment, to determine whether the audio of the voice call is computer-generated the electronic device is further configured to:
         stream the received audio of the voice call to a server; and   receive an indication of whether the voice call is computer-generated from the server.       

     In an embodiment, the electronic device is an Interactive Voice Recognition, IVR, platform. 
     In an embodiment, the electronic device is further configured to open a channel to the server. 
     According to a third aspect, there is provided a system comprising:
         the electronic device according to the second aspect; and   a server configured to:   receive audio of a voice call from the electronic device;   determine whether the audio of the voice call is computer-generated; and   transmit the result of the determination of whether the audio of the voice call is computer-generated to the electronic device.       

     In an embodiment, to determine whether the audio of the voice call is computer-generated, the server is further configured to calculate a score indicating a probability that the voice call is computer-generated; and
         wherein, when the probability that the voice call is computer-generated is greater than a predefined threshold, the electronic device is further configured to terminate the voice call.       

     In an embodiment, the server is further configured to determine whether the audio of the voice call is computer-generated by detecting silence in the audio;
         wherein the detection of silence in the audio indicates a human caller.       

     In an embodiment, the server is further configured to determine whether the audio is computer generated by detecting silence of Dual Tone Multi-Frequency, DTMF, tones in the audio of the voice call;
         wherein the detection of Dual Tone Multi-Frequency, DTMF, tones in the audio indicates a human caller.       

     In an embodiment, if the call is terminated, the server is further configured to auto-dial the number of the received voice call, wherein the call being answered indicates that the voice call was made by a human caller. 
     Another aspect of the invention provides a computer program comprising instructions arranged, when executed, to implement a method in accordance with any one of the above-described aspects. A further aspect provides machine-readable storage storing such a program. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Embodiments of the invention are further described hereinafter with reference to the accompanying drawings, in which: 
         FIG.  1    is a flowchart showing an example of a method of managing incoming calls; 
         FIG.  2    is a flowchart showing another example of a method of managing incoming calls by an electronic device; 
         FIG.  3    is a diagram showing an example of a method of managing incoming calls by a system comprising an electronic device and a server; 
         FIG.  4    is a diagram showing another example of a method of managing incoming calls by system comprising an electronic device and a server; 
         FIG.  5    is a system diagram showing an example of an interaction between an electronic device, a calling device, and a network; and 
         FIG.  6    is a system diagram showing an example of an interaction between an electronic device, a calling device, a server, and a network. 
     
    
    
     DETAILED DESCRIPTION 
     Robocalls, hereinafter referred to as “computer-generated voice calls”, are not only computer-dialled; the actual spoken dialogue of the call is not a human caller speaking but is audio generated by text-to-speech software, hereinafter referred to as computer-generated audio (i.e. synthetic speech or robotic speech). The computer-generated audio could be ‘robotic’ speech, meaning it would be obvious to a human that it is not another human speaking, or it could use advanced synthetic speech tools to sound, to the human ear, like another human being. This latter case is referred to as synthetic speech. 
     Certain speech-based algorithms, normally found within voice biometric solutions, can detect a synthetic voice or a robotic voice from a human voice, even when a human would not be able to discern between the two. Synthetic or robotic voices can be detected by analysing frequency spectrum information, as well as detecting indicators of human callers such as silence and Dual Tone Multi-Frequency tones. It is this ability to detect a synthetic or robotic voice that forms the basis of this invention and allows IVR platforms to terminate computer-generated voice calls within seconds, thus quickly freeing valuable channels and avoiding expensive agent handovers. 
     To facilitate this, according to one example, upon an IVR answering the inbound voice call, a channel is opened to a detection engine and all audio, including silence, of the voice call coming from the caller is streamed to the detection engine. 
     Computer-generated calls generally start speaking immediately when they detect a sound from the called party, such as an IVR welcome script, so if computer-generated audio is not detected within a few seconds, the detection engine sends an indication that the voice call is legitimate to the IVR platform; alternatively, if computer-generated audio is detected, the detection engine sends a corresponding indication to the IVR platform which can terminate the voice call. Additionally, a legitimate call made by a human caller may include a period of silence and/or Dual Tone Multi-Frequency (DTMF) tones, both of which may indicate a human caller; and can therefore be used as indicators of computer-generated calls. 
     Where computer-generated audio is detected, the detection engine calculates a score indicating a probability that the audio of the voice call is computer-generated and provides the score to the IVR platform. The IVR platform would then terminate the call, provided the received score was above a predefined threshold. Alternatively, the detection engine may return a simple classification of the call as being human or computer generated. 
     Embodiments of the present invention will now be described. 
     In an embodiment of the present invention, a method of managing incoming voice calls is implemented wherein audio of a voice call is received, it is determined whether the audio of the voice call is computer-generated, and, if it is determined that the audio of the voice call is computer-generated, the voice call is terminated. It will be appreciated that this method may be implemented by an electronic device, a server, or may be shared between an electronic device and a server. For example, the method may be wholly carried out by an electronic device or may be wholly carried out by a server, or may be partially carried out by both an electronic device and a server, or the like. 
     The electronic device may be an IVR platform or server or may alternatively be a mobile phone or the like, capable of detecting computer-generated voice calls on an individual basis. The server may alternatively be a detection engine. 
     An example of this method of managing incoming voice calls is shown in  FIG.  1   . 
     In step S 100 , audio of a voice call is received. The audio of the voice call may be received directly by an electronic device answering the incoming voice call and thereby receiving the audio of the voice call directly. Alternatively, a server or the like may answer the incoming voice call and receive the audio of the voice call directly. Alternatively, the voice call, and therefore the audio of the voice call, may be received by an electronic device which may then stream the audio to a server; equally, the voice call may be received by the server which may stream audio of the voice call to the electronic device. That is, the remaining steps of  FIG.  1    may be performed by the device which receives the voice call or by a separate device to which audio of the voice call is streamed. 
     In step S 120 , it is determined whether the audio of the voice call is computer-generated. The determination of whether the audio of the voice call is computer-generated may be carried out by the electronic device or by the server. The determination of whether the audio of the voice call is computer-generated may comprise the detection of silence in the audio of the voice call; the presence of silence in the audio of the voice call being an indicator of a human caller. Alternatively, or additionally, the determination of whether the audio of the voice call is computer-generated may comprise the detection of Dual Tone Multi-Frequency (DTMF) tones; the presence of DTMF tones being a further indicator of a human caller. 
     In step S 140 , if it is determined that the audio of the voice call is computer-generated, the voice call is terminated. The voice call may be terminated by the device which received the voice call or may be terminated remotely by another device which determines that the audio of the voice call is computer-generated. 
     For example, an electronic device may receive the incoming voice call and may stream the audio of the voice call to a server which may perform the determination of whether the audio of the voice call is computer-generated. The server may then indicate to the electronic device that the voice call is computer-generated, and the electronic device may then terminate the voice call. Alternatively, the server, having determined that the voice call is computer-generated, may terminate the voice call directly. Alternatively, the electronic device, or the server, may receive the incoming voice call, determine whether the audio of the voice call is computer-generated, and terminate the voice call directly (without the involvement of another device in the processing). 
     It will be appreciated that the electronic device of the previous example may be an Interactive Voice Recognition (IVR) platform or the like, and the server may be a detection engine or the like. 
     Another example of this method of managing incoming voice calls is shown in  FIG.  2   . 
     In step S 200 , a voice call is received. The voice call may be received by an electronic device. Alternatively, the voice call may be received by a server or the like. 
     In step S 220 , a score is calculated which indicates a probability that the audio of the voice call is computer-generated. The score may be calculated by the electronic device or, alternatively, may be calculated by the server. Alternatively, the server may calculate the score indicating a probability that the audio of the voice call is computer-generated. The calculation may comprise the detection of silence in the audio of the voice call. Alternatively, or additionally, the calculation may comprise the detection of Dual Tone Multi-Frequency (DTMF) tones. 
     In step S 240 , when the score indicating a probability that the audio of the voice call is computer-generated is greater than a predefined threshold, the voice call is terminated. The voice call may be terminated by the device which received the voice call or may be terminated remotely by another device which determines that the audio of the voice call is computer-generated. 
     For example, the electronic device may receive the incoming voice call and may stream the audio of the voice call to the server which may calculate a score indicating the probability that the audio of the voice call is computer-generated. The server may then transmit the score indicating the probability that the voice call is computer-generated to the electronic device which may then terminate the voice call. Alternatively, the server, having calculated the score, may terminate the voice call directly. Alternatively, the electronic device, or the server, may receive the incoming voice call and calculate the score indicating the probability of the audio of the voice call being computer-generated, and terminate the voice call directly. 
     It will be appreciated that the electronic device of the previous example may be an Interactive Voice Recognition (IVR) platform or the like, and the server may be a detection engine or the like. 
     It will be appreciated that the above described examples may be carried out by a single device or may be performed using multiple devices. 
       FIG.  3    is a diagram showing a method of managing incoming voice calls as carried out by an electronic device  300  and a server  310 . In step S 320 , the electronic device  300  receives the voice call and, in step S 330 , the audio of the voice call is transmitted by the electronic device  300  to the server  310 . Alternatively, the server  310  may receive the voice call and may transmit the audio of the voice call to the electronic device  300 . Alternatively, the electronic device  300  or the server  310  may receive the voice call and may not transmit the audio of the voice call to another device. 
     In step S 340 , the server  310  determines whether the audio of the voice call is computer-generated. The determination may comprise the detection of silence in the audio of the voice call; the presence of silence in the audio of the voice call being an indicator of a human caller. Alternatively, or additionally, the determination of whether the audio of the voice call is computer-generated may comprise the detection of Dual Tone Multi-Frequency (DTMF) tones; the presence of DTMF tones in the audio being an indicator of a human caller. Alternatively, the electronic device  300  may perform the determination of whether the audio of the voice call is computer-generated. 
     In step S 350 , the server  310  transmits an indication of whether the audio of the voice call is computer-generated to the electronic device  300 . Alternatively, the electronic device may determine whether the audio of the voice call is computer-generated and may transmit a corresponding indication to the server  310 . Alternatively, in the case that the electronic device  300  or the server  310  has performed all the previous steps of the method, step S 350  may not be carried out at all. 
     In step S 360 , the electronic device  300  terminates the voice call if it has been determined that the audio of the voice call is computer-generated. Alternatively, the server  310  may terminate the voice call. In greater detail, the voice call may be terminated by the device which received the voice call, such as the electronic device, or may be terminated remotely by another device, such as the server, which determines that the audio of the voice call is computer-generated. 
     For example, the electronic device  300  may receive the incoming voice call and may stream the audio of the voice call to the server  310  which may perform the determination of whether the audio of the voice call is computer-generated and calculate a score indicating the probability that the audio of the voice call is computer-generated. The server  310  may then transmit the score to the electronic device  300  that the voice call is computer-generated and the electronic device  300  may then terminate the voice call. Alternatively, the server  310 , having calculated the score, may terminate the voice call directly. Alternatively, the electronic device  300 , or the server  310 , may receive the incoming voice call, calculate the score indicating the probability of the audio of the voice call being computer-generated, and terminate the voice call directly. 
     It will be appreciated that the electronic device  300  of the previous example may be an Interactive Voice Recognition (IVR) platform or the like, and the server  310  may be a detection engine or the like. 
       FIG.  4    is a diagram showing an alternative method of managing incoming voice calls as carried out by an electronic device  400  and a server  410 . In step S 420 , the electronic device  400  receives a voice call and transmits audio of the voice call to the server  410  in step S 430 . Alternatively, the server  410  may receive the voice call and may transmit the audio of the voice call to the electronic device  400 . Alternatively, the electronic device  400  or the server  410  may receive the voice call and may not transmit the audio of the voice call to another device. 
     In step S 440 , the server  410  calculates a score indicating a probability of the voice call being computer-generated and, in step S 450 , transmits the score to the electronic device  400 . Alternatively, the electronic device  400  or the server  410  may calculate the score and may not transmit the score to another device. 
     The calculation may comprise the detection of silence in the audio of the voice call; the presence of silence in the audio of the voice call being an indicator of a human caller. Alternatively, or additionally, the calculation of whether the audio of the voice call is computer-generated may comprise the detection of Dual Tone Multi-Frequency (DTMF) tones; the presence of DTMF tones in the audio of the voice call being an indicator of a human caller. Alternatively, the electronic device  400  may perform the calculation of whether the audio of the voice call is computer-generated. 
     In step S 460 , the electronic device  400  terminates the voice call if the score received from the server  410  indicating the probability of the audio of the voice call being computer-generated is greater than a predefined threshold. The predefined threshold may be defined such that the confidence that the audio of the voice call is computer-generated is very high, thereby minimising the likelihood that the determination is a mistake. Alternatively, the server  410  may terminate the voice call if the score indicating the probability of the audio of the voice call being computer-generated is greater than a predefined threshold. Additionally, the score indicating the probability of the audio of the voice call being computer-generated may be used to perform additional measures to ensure human callers are not mistaken as computer-generated voice calls. For example, if the score were approaching the threshold at which it is determined that the audio of the voice call is computer-generated, an additional check may be performed. An additional check may include auto-dialling the number of the voice call if the voice call is terminated to check whether the call is answered; the call being answered would indicate a human caller. 
     It will be appreciated that the electronic device of the previous example may be an Interactive Voice Recognition (IVR) platform or the like, and the server may be a detection engine or the like. 
       FIG.  5    is a system diagram showing the interaction between an electronic device  500 , a calling device  520 , and a network  510 . The calling device  520  connects with the network  510 , transmitting a voice call, and the network  510  relays the voice call to the electronic device  500 , which proceeds to determine whether the audio of the voice call is computer-generated. If it is determined that the voice call is computer-generated, the electronic device  500  terminates the voice call via the network  510 , which disconnects the calling device  520 . 
     It will be appreciated that the method performed by the electronic device  500  may be carried out by the server  520  or the like. 
       FIG.  6    is a system diagram showing the interaction between an electronic device  600 , a server  620 , a calling device  630  and a network  610 . The network  610  receives an incoming voice call from the calling device  630  and forwards the voice call to the electronic device  600 . The electronic device  600  receives the incoming voice call via the network  610  and streams the audio of the voice call to the server  620  via the network  610 . The server  620  determines whether the audio of the voice call is computer-generated and transmits a corresponding indication to the electronic device  600  via the network. Alternatively, the server  620  may determine whether the audio of the voice call is computer-generated and calculate a score indicating a probability that the voice call is computer-generated; the server  620  may then send the score to the electronic device  600  via the network  610 . The electronic device  600  may then terminate the voice call, if it is determined that the audio of the voice call is computer-generated. 
     It will be appreciated that the method performed by the electronic device  600  may be carried out by the server  620  or the like and that the method performed by the server  620  may be carried out by the electronic device  600  or the like. 
     It will be appreciated that embodiments of the present invention can be realized in the form of hardware, software or a combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage, for example a storage device like a ROM, whether erasable or rewritable or not, or in the form of memory, for example RAM, memory chips, device or integrated circuits or on an optically or magnetically readable medium, for example a CD, DVD, magnetic disk or magnetic tape or the like. It will be appreciated that the storage devices and storage media are embodiments of machine-readable storage that are suitable for storing a program or programs comprising instructions that, when executed, implement embodiments of the present invention. 
     Accordingly, embodiments provide a program comprising code for implementing apparatus or a method as claimed in any one of the claims of this specification and a machine-readable storage storing such a program. Still further, such programs may be conveyed electronically via any medium, for example a communication signal carried over a wired or wireless connection and embodiments suitably encompass the same. 
     Throughout the description and claims of this specification, the words “comprise” and “contain” and variations of them mean “including but not limited to”, and they are not intended to (and do not) exclude other components, integers or steps. Throughout the description and claims of this specification, the singular encompasses the plural unless the context otherwise requires. In particular, where the indefinite article is used, the specification is to be understood as contemplating plurality as well as singularity, unless the context requires otherwise. 
     Features, integers or characteristics described in conjunction with a particular aspect, embodiment or example of the invention are to be understood to be applicable to any other aspect, embodiment or example described herein unless incompatible therewith. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive. The invention is not restricted to the details of any foregoing embodiments. The invention extends to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed. It will be also be appreciated that, throughout the description and claims of this specification, language in the general form of “X for Y” (where Y is some action, activity or step and X is some means for carrying out that action, activity or step) encompasses means X adapted or arranged specifically, but not exclusively, to do Y. 
     The reader&#39;s attention is directed to all papers and documents which are filed concurrently with or previous to this specification in connection with this application and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference.