Patent Description:
Voice phishing, or vishing, is the use of telephony (often Voice over IP telephony) to conduct phishing attacks. Landline telephone services have traditionally been trustworthy; terminated in physical locations known to the telephone company, and associated with a bill-payer. Now however, phishing fraudsters often use modern Voice over IP (VoIP) features such as caller ID spoofing and automated systems (IVR) to impede detection by law enforcement agencies. Voice phishing is typically used to steal credit card numbers or other information used in identity theft schemes from individuals.

Usually, voice phishing attacks are conducted using automated text-to-speech systems that direct a victim to call a number controlled by the attacker, however some use live callers. Posing as an employee of a legitimate body such as the bank, police, telephone or internet provider, the fraudster attempts to obtain personal details and financial information regarding credit cards, bank accounts (e.g. the PIN), as well as personal information of the victim. With the received information, the fraudster might be able to access and empty the accounts or commit identity fraud. Some fraudsters may also try to persuade the victim to transfer money to another bank account or withdraw cash to be given to them directly. Callers also often pose as law enforcement or as an Internal Revenue Service employee. Scammers often target immigrants and the elderly, who are coerced to wire hundreds to thousands of dollars in response to threats of arrest or deportation.

Voice Phishing comes in various forms. There are various methods and various operation structures for the different types of phishing. Usually, scammers will employ social engineering to convince victims of a role they are playing and to create a sense of urgency to leverage against the victims.

Voice phishing has unique attributes that separate the attack method from similar alternatives such as email phishing. With the increased reach of mobile phones, phishing allows for the targeting of individuals without working knowledge of email but who possess a phone, such as the elderly. The historical prevalence of call centers that ask for personal and confidential information additionally allows for easier extraction of sensitive information from victims due to the trust many users have while speaking to someone on the phone. Through voice communication, phishing attacks can be personable and therefore more impactful than similar alternatives such as email. The faster response time to an attack attempt due to the increased accessibility to a phone is another unique aspect, in comparison to an email where the victim may take longer time to respond. A phone number is difficult to block and scammers can often simply change phone numbers if a specific number is blocked and often find ways to overcome rules and regulations.

Voice phishing attacks can be difficult for victims to identify because legitimate institutions such as banks sometimes ask for sensitive personal information over the phone. Phishing schemes may employ pre-recorded messages of notable, regional banks to make them indistinguishable from legitimate calls. Additionally, victims, particularly the elderly, may forget or do not know about scammers' ability to modify their caller ID, making them more vulnerable to voice phishing attacks.

In that context, the state-of-the-art describes techniques like.

In the view of above, it is an object of the present invention to provide techniques that enable an efficient protection against voice phishing of a telephone call.

According to a first aspect of the invention, a method is provided according to claim <NUM>.

It is further noted that
if the speech-analyzing-unit is implemented as software within the telecommunication device then the input data link is to be understood as a typical communication mean within the telecommunication device to provide data to an algorithm;.

This provides the advantage of detecting and preventing voice phishing crime by deeply inspecting the context of the telecommunication conversations in an automatic way that shifts the burden of detecting voice phishing from the user to the detecting algorithm. This is beneficial since it is hard to educate the telecommunication users whenever a new scam method is developed or evolved from the known scam methods. However, if a telecommunication service provider collects and spreads the new pattern of such scams in the form of the patterns of possible voice phishing to the databases of the speech-analyzing-unit, new patterns of such scams can be detected efficiently. Furthermore, those patterns of possible voice phishing can be automatically translated into other languages and being provided accordingly to the speech-analyzing-unit. This has the advantage, that typically scam first happens within a region that uses a single language and is then adapted to other countries that use different languages. The translation enables that protection is provided as fast as possible. Furthermore, the translation does not need not to be limited to spoken languages but can also include programming languages in which the speech patterns are written. Such languages can be XML, JSON, YMAL, etc. Hence, the invention will be especially beneficial for protecting elderly and children who are more prone to be victimized than others by such criminal acts.

In a practical way, the speech-analyzing-unit can be designed as a hardware and/or a software embodiment. If the data communication device is a smartphone, the speech-analyzing-unit can be loaded onto the smartphone as an app or can be integrated into the operating system of the smartphone. In these cases, the speech-analyzing-unit can be implemented as software within the smartphone, which enables an efficient standalone solution against voice phishing that could even work if the smartphone has no connection to the Internet. However, voice phishing also happens if the user uses a non-smartphone, for example at home. Such a non-smart-phone is typically not able to load and execute apps, in these cases, the speech-analyzing-unit can be designed as an external hardware unit that analyzes the telephone call accordingly. A possible implementation is within a residual gateway through which voice over IP telephone calls transverse so that it is possible to design the external speech-analyzing-unit in a way that it has access to the audio signals of the telephone call. Another scenario is that some users also like to make telephone calls, especially within the homes, using the telephone in a hands-free mode. In that case, the external speech-analyzing-unit could be placed anywhere within the room of the user and having a microphone as a input data link for the audio input and a sound speaker for the output data link that gives a warning to the user if voice phishing has been detected. If the speech-analyzing-unit is implemented as being software within the telecommunication device, it can have its own processor or use the processor of the data communication device to run the algorithm. If the speech-analyzing-unit is implemented as external hardware, typically has its own processor to run the algorithm.

Preferably, the algorithm transmits the text elements and/or other information about the telephone call via the output data link to a service of a service provider for further investigation.

This provides the advantage that the additional resources of the service provider can be used to analyze the text elements with regard to voice phishing. Typically, the resources of the service provider are more powerful and are regularly updated so that they provide the best detection results.

In an embodiment, the service provider investigates a geolocation of the telephone call and/or if the telephone call is listed on a blacklist.

This provides the advantage of providing a different approach in assessing if the telephone call is voice phishing. In particular, investigating a geolocation typically can only be performed by the service provider because the telecommunication device of the user cannot access the necessary information to identify the geolocation of a telephone call. The service provider can send the results of his analysis back to the telecommunication device. The results can be used by the telecommunication device along with its own results to determine if the telephone call is voice phishing or the results can tell the telecommunication device independently that voice phishing is happening.

In an embodiment, the speech-analyzing-unit performs the voice phishing detection for a predefined period of time, in particular for <NUM> seconds.

This provides the advantage of saving the computational resources of the telecommunication device and/or the service provider because typically the relevant voice phishing patterns are occurring at the beginning of a telephone call. The speech-analyzing-unit is configured to set the predefined period of time to different values.

In an embodiment, the different specific speech patterns are analyzed separately by the algorithm to avoid overfitting effects.

In mathematical theories, overfitting takes place if there are too many possible explanations parameters for a given set of data points. Mixing all the different speech patterns with the various parameters into one a single analysis could lead to the same effect, namely that the algorithm finds different patterns that are related to different speech parameter sets and concludes that voice phishing has occurred even if the matching of the text elements with those parameters happened only by chance. Therefore, the feature of analyzing the different speech separately avoids overfitting, which could detect voice phishing even if actually no phishing has occurred.

According to a second aspect of the invention, a speech-analyzing-unit is provided according to claim <NUM>.

The advantages of the speech-analyzing-unit correspond to the advantages described in the context of the method above except in cases where this is not technically applicable.

If the speech-analyzing-unit is not being implemented on the telecommunication device, wherein the telecommunication device can be a smartphone or a standard telephone, it could be implemented on a Residential Gateway for example a router of the user or could be designed as an adapter plug that is connected with the telephone line. In the latter case, the speech-analyzing-unit comprises appropriate input channels for the telephone line and outputs channels to signal if voice phishing was detected by its algorithm that runs on the processor of the adapter plug.

According to a third aspect of the invention, a telecommunication device according to claim <NUM> is provided.

The advantages of the telecommunication device correspond to the advantages described in the context of the method above except in cases where this is not technically applicable.

<FIG> shows an overview of a communication system <NUM> configured to enable protection against voice phishing of a telephone call.

The communication system <NUM> comprises a telecommunication device <NUM> of a user <NUM>, wherein the user <NUM> makes phone calls with the telecommunication device <NUM>. The telecommunication device <NUM> can be designed as a smart phone, a smart watch, a non-smart phone and/or other devices with a telephone functionality. The telecommunication device <NUM> comprises a speech-analyzing-unit 12a on which an algorithm is implemented to determine possible voice phishing. The speech-analyzing-unit 12a can be designed as a hardware or a software embodiment.

The phone calls of the user <NUM> are transmitted over a communication network <NUM>, in particular telephone communication network <NUM>, to other parties. In particular, the other parties can be a third person <NUM> or a voice phishing criminal <NUM>. By means of the communication network <NUM> the telecommunication device <NUM> can communicate with a server of a service provider <NUM>.

In the following, the steps of the method are described:
The user <NUM> or the third person <NUM> as a user's legal guardian <NUM> agrees on the service and installation of the anti-phishing software on the user's smart telecommunication device <NUM>. The service provider <NUM> transfers and/or periodically updates the software with patterns, in particular speech patterns, of known voice phishing scenarios. When the voice call is established, the software recognizes the words and sentences spoken by the caller/callee, and determines if the pattern of the voice call corresponds to known phishing voice patterns.

When a call is incoming and the caller's number is not unknown to the user <NUM> (e.g., contact list and/or call history), the speech-analyzing-unit 12a starts to listens to the conversation and can detect voice phishing by matching the spoken words with the one or more voice phishing patterns. If voice phishing is determined, the telecommunication device <NUM> warns the user about the possible voice phishing trial (with the background voice and text on the display). The speech-analyzing-unit 12a can create the corresponding data signals in a way that the telecommunication device warns the user <NUM>. Alternatively, and/or additionally, the user <NUM> may request for further analysis of the caller <NUM> (e.g. by pressing number on the dial pad or on the display screen).

If the user <NUM> requests the further analysis of the voice call, the telecommunication device <NUM> informs an anti-phishing service of the service provider <NUM>. The anti-phishing service further investigates the caller (e.g., geolocation, blacklist, etc). If the user has a legal guardian <NUM>, he can be informed with a SMS that voice phishing was determined.

The service provider <NUM> informs the legal guardian <NUM>, or the user <NUM>, if there's no legal guardian by playing background speech and/or showing text on the display, about the investigation result.

The user <NUM> and/or legal guardian can request for the termination of the voice call. Some of the processes described above can be performed with or without the explicit request from the user <NUM> (depending on the service policy, legal issue, and/or legal guardian's agreement).

<FIG> shows at least parts of an algorithm designed to determine voice phishing speech patterns.

Step <NUM> comprises voice recognition of the voice call, wherein the speech of the caller can be transcribed into text elements.

In step <NUM> keyword extraction from the conversation takes place for example for <NUM> seconds. Typically, the speech patterns related to voice phishing can be extracted within the first seconds of the voice call. Restricting the keyword extraction to a predefined time limit saves computational resources.

In step <NUM> pattern matching, in particular of the whole conversation, can be performed. If no matching patterns are found <NUM>, then the algorithm goes back in a loop-wise fashion into step <NUM>. If the matching pattern is found <NUM>, then the anti-phishing process continues in step <NUM>. In particular, the further analysis can be requested from the service provider <NUM>.

A speech pattern may dependent on one or multiple other patterns. For example, "pattern-of-the-telemarketing-fraud" pattern can require the matching result of "pattern-of-unknown-caller" pattern.

As an example, conversation patterns with an unknown person may include (but not limited to) following parameters described in <FIG>.

<FIG> shows a first table <NUM> of a speech pattern that is designed to detect a voice phishing pattern called "unknown- caller". The first table comprises a first column <NUM> with parameter names, wherein the parameters describe different categories of the content that can be extracted from the voice call and/or other properties like a contact list, a call history, keywords of the caller, keywords of the callee, politeness and a matching condition, wherein the matching condition describes how the patterns shall be evaluated to determine voice phishing. One or multiple entries of the first column <NUM> can be predefined by the service provider.

The invention describes three options to evaluate the voice call - these options are also called matching conditions. These options are, wherein the numbering starts with zero, <NUM>= threshold, <NUM> = complete, and <NUM> = tolerance. A speech pattern that corresponds to the first table <NUM> is evaluated with the option <NUM> = complete.

The second column <NUM> of the first table <NUM> comprises a description of the parameter names of the corresponding rows in the first column <NUM>. For example, the description of the pattern of the parameter name "contact_list" is: Is the number of the caller on the contact list?.

The third column <NUM> of the first table <NUM> comprises a data type of the parameter names of the corresponding rows in the first column <NUM>. For example, the data type of the parameter name "contact_list" is: Boolean. A possible definition can be that True corresponds to <NUM> and False corresponds to <NUM>.

The fourth column <NUM> of the first table <NUM> comprises a tolerance value of the parameter names of the corresponding rows in the first column <NUM>. For example, the tolerance value of the parameter name "contact_list" is: <NUM>.

The fifth column <NUM> of the first table <NUM> comprises a weight value of the parameter names of the corresponding rows in the first column <NUM>. For example, the weight value of the parameter name "contact-list" is: <NUM>.

The voice phishing patterns can be described in dynamic ways with various description languages such as XML, JSON, YAML, etc..

If matching_condition=<NUM>, the sum of values of the specified parameters (i.e., caller_keywords, callee_keywords, politeness) determines whether the conversation falls into the speech pattern described by the first table <NUM> or not.

If the sum of determined values is higher than the sum of the tolerances of the first table <NUM>, the pattern matching yields True, otherwise False. The weight doesn't play a role in this case. In this context, true means, that voice phishing is detected and false means that no voice phishing was detected.

In the following, the option matching_condition = <NUM> is being described in greater detail:
If matching_condition=<NUM>, the sum of values of the specified parameters determines whether the conversation falls into the pattern or not. If the sum of values is higher than the sum of their tolerances, the pattern matching yields True, otherwise False. The weight does not play a role in this case.

An equation used to determine whether the conversation pattern matches to the voice phishing pattern:.

<FIG> shows a second table <NUM> of a speech pattern that is designed to detect a voice phishing pattern called "IRS impersonation scam". IRS impersonation scams involve scammers targeting American taxpayers by pretending to be Internal Revenue Service (IRS) collection officers. The scammers operate by placing disturbing official-sounding calls to unsuspecting citizens, threatening them with arrest and frozen assets if thousands of dollars are not paid immediately, usually via gift cards or money orders. According to the IRS, over <NUM>,<NUM>,<NUM> Americans have received threatening calls, and $<NUM>,<NUM>,<NUM> have been reported lost to these call scams as of March, <NUM>. The problem has been assigned to the Treasury Inspector General for Tax Administration.

The second table <NUM> comprises a first column <NUM> with parameter names, wherein the parameters describe different categories of the content that can be extracted from the voice call and/or other properties like an acquaintance, keywords, critical keyword, matching condition and a threshold, wherein the matching condition describes how the patterns shall be evaluated to determine voice phishing. A speech pattern that corresponds to the second table <NUM> is evaluated with the matching condition = <NUM> = threshold.

The second column <NUM> of the second table <NUM> comprises a description of the parameter names of the corresponding rows in the first column <NUM>. For example, the description of the pattern of the parameter name "acquaintance" is: Does the user know the caller?.

The third column <NUM> of the second table <NUM> comprises a data type of the parameter names of the corresponding rows in the first column <NUM>. For example, the data type of the parameter name "acquaintance" is: Boolean.

The fourth column <NUM> of the second table <NUM> comprises a tolerance value of the parameter names of the corresponding rows in the first column <NUM>. For example, the tolerance value of the parameter name "acquaintance" is: <NUM>.

The fifth column <NUM> of the second table <NUM> comprises a weight value of the parameter names of the corresponding rows in the first column <NUM>. For example, the weight value of the parameter name "acquaintance" is: <NUM>.

One or multiple entries of the first column <NUM> can be predefined by the service provider.

In the following, the option matching_condition = <NUM> is being described in greater detail:
If matching_condition=<NUM>, the sum of evaluation points of specific parameters determines whether the conversation falls into the voice phishing pattern.

If the sum is higher than the pre-defined value of the threshold parameter, this pattern matching yields True (i.e., voice phishing found), False otherwise. The tolerance and weight do not play roles in this case.

An equation used to determine whether the conversation pattern matches to the voice phishing pattern:
True, the speech pattern of the voice call is determined as a voice phishing pattern: <MAT>, wherein i is the index number of the i-th parameter, n is the number of required parameters and Valuethreshold is the predefined value of the Threshold parameter.

False, the speech pattern of the voice call is not determined as a voice phishing pattern: <MAT>.

<FIG> shows a third table <NUM> of a speech pattern that is designed to detect a voice phishing pattern called "Hollywood Con Queen scam".

In this scam, gig workers in the entertainment industry are approached via phone or email by an individual claiming to be an executive, director, casting agent, or producer. The target of the scam is presented with an opportunity to work on a film or television project and is encouraged to travel to Jakarta, Indonesia to scout for locations and participate in meetings. When they arrive, the target is toured around various tourist destinations and cultural sites by a driver who speaks little English. Because of the heavy traffic in Jakarta, the target spends most of their time driving and is frequently told that their schedule has changed and meetings were cancelled. As the target is transported to various locations, they are forced to pay their driver fees with the promise of eventual reimbursement. When targets become upset or frustrated after days of cancelled meetings and driving, they often leave the country early. Throughout the process, the target communicates with the person who invited them to Jakarta, who becomes increasingly irate, demanding, or inappropriate. Targets may not realize they are the target of the scam until returning home and conducting research.

The third table <NUM> comprises a first column <NUM> with parameter names, wherein the parameters describe different categories of the content that can be extracted from the voice call and/or other properties like an acquaintance, keywords group <NUM>, keywords group <NUM>, keywords group <NUM>, matching condition and a threshold, wherein the matching condition describes how the patterns shall be evaluated to determine voice phishing. A speech pattern that corresponds to the second table <NUM> is evaluated with the matching condition = <NUM> = tolerance.

The second column <NUM> of the third table <NUM> comprises a description of the parameter names of the corresponding rows in the first column <NUM>. For example, the description of the pattern of the parameter name "acquaintance" is: Does the user know the caller?.

The third column <NUM> of the third table <NUM> comprises a data type of the parameter names of the corresponding rows in the first column <NUM>. For example, the data type of the parameter name "acquaintance" is: Boolean.

The fourth column <NUM> of the third table <NUM> comprises a tolerance value of the parameter names of the corresponding rows in the first column <NUM>. For example, the tolerance value of the parameter name "acquaintance" is: <NUM>.

The fifth column <NUM> of the third table <NUM> comprises a weight value of the parameter names of the corresponding rows in the first column <NUM>. For example, the weight value of the parameter name "acquaintance" is: <NUM>.

In the following, the option matching_condition = <NUM> is being described in greater detail:
When matching_condition=<NUM>, the total sum of values (i.e., the sum of values multiplied by their weights) of the specified parameters. If the total sum is greater than the tolerance of this parameter, the pattern matching will yield True (i.e. voice phishing detected), otherwise False (i.e. no voice phishing detected). The parameter with a value that is lower than its tolerance will be ignored in the equation.

True (the conversation pattern matches the scam pattern): <MAT> <MAT>, where Valuei > Tolerancei, i is the index number of the i-th parameter, n is the number of required parameters, and Valuetolerance is the pre-defined value of the Tolerance parameter.

False (the conversation pattern doesn't match the scam pattern): <MAT> <MAT>.

This provides the advantage that an evaluation can be tailored to different voice phishing methods.

The main idea of having multiple different matching conditions is to reduce the chance of false-positive errors. By using different condition of matching, one can increase chance to avoid the false-positive errors. For some cases, the weight is evenly distributed to multiple parameters. In such cases, there are no clear decisive parameters, and they would be equally important. In such cases, the judgement must be made by the overall number of matching appearances. That's why "matching_condition=<NUM> (threshold)" can be used.

For some cases, parameters of a pattern are too ambiguous to use them as decisive parameters. So, one can use matching_condition=<NUM> (complete) and set "tolerance" value on each of the parameters. For example, if a big data analysis finds out that telephone communications contain <NUM> times "May I" on average, then the tolerance for such keywords can be set to <NUM>.

Claim 1:
A method to enable protection against voice phishing of a telephone call performed by a speech-analyzing-unit (12a) that is part of a user's telecommunication device (<NUM>), wherein the speech-analyzing-unit (12a) is configured to perform the steps of:
∘ receiving, by an input data link, parts of the telephone call as audio input
∘ storing, by a database, speech patterns of possible voice phishing;
∘ converting, by an algorithm, the audio input to text elements and comparing the text elements to the patterns of possible voice phishing with regard to an occurrence of voice phishing, and triggering countermeasures if voice phishing is detected;
∘ transmitting, by at least an output data link, the countermeasures to the telecommunication device (<NUM>), characterized in that the countermeasures comprise a warning that is transmitted to the user of the telecommunication device as a background voice of the telecommunication device.