Using natural language processing for detection of intended or unexpected application behavior

Detection of unintended application behaviors, where natural language processing (NLP) techniques are used to analyze the application, and specifically its graphical user interface (GUI), and construct an acceptable (or expected) list per-context actions. Actions executed by the application in a given context that do not fall within the list are flagged as unexpected (or anomalous).

BACKGROUND

Aspects of the present invention generally relate to a method and computer program product for the detection of unintended computer application behaviors, whereby natural language processing (NLP) techniques are used to analyze the application (app), and specifically its graphical user interface (GUI), and construct an acceptable (or expected) list per-context actions. Actions executed by the app in a given context that do not fall within the list are flagged as unexpected (or anomalous).

SUMMARY

Aspects of the invention are method and computer program product for detecting unintended behaviors in a computer application where natural language processing (NLP) techniques are used to analyze the application, and specifically its GUI, and construct an acceptable (or expected) list per-context actions.

An exemplary method for using natural language processing (NLP) for detecting unintended computer application behavior to construct a statistical model to determine for an application, given the application's GUI and unlabeled text, whether the application is exhibiting unintended or abnormal behavior comprises: instrumenting the application such that UI transitions are recorded as an acceptable list per-context action and security-critical operations are monitored for training the system; capturing from the instrumentation as UI transitions occur for obtaining text associated with an enabled UI widget; forming a sequence of NLP textural content from interactions with the application and interleaving into the sequence security-relevant operations that occur between UI transitions; analyzing the suffix leading to the operation as a security-critical operation is about to execute to determine whether the operation is expected to occur at this point in the execution based upon NLP acceptable list and its GUI, and qualifying whether the application interaction is consistent with expected behavior based on identification of an intended semantic from the acceptable list per-context actions, and if not, raise a warning flag.

An exemplary non-transitory computer readable medium having computer readable program for using natural language processing (NLP) for detecting unintended computer application behavior to construct a statistical model to determine for an application, given the application's GUI and unlabeled text, whether the application is exhibiting unintended or abnormal behavior comprises: instrumenting the application such that UI transitions are recorded as an acceptable list per-context action and security-critical operations are monitored for training the system; capturing from the instrumentation as UI transitions occur for obtaining text associated with an enabled UI widget; forming a sequence of NLP textural content from interactions with the application and interleaving into the sequence security-relevant operations that occur between UI transitions;

analyzing the suffix leading to the operation as a security-critical operation is about to execute to determine whether the operation is expected to occur at this point in the execution based upon NLP acceptable list and its GUI, and qualifying whether the application interaction is consistent with expected behavior based on identification of an intended semantic from the acceptable list per-context actions, and if not, raise a warning flag.

The objects, features, and advantage of the present disclosure will become more clearly apparent when the following description is taken in conjunction with the accompanying drawings.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Abnormal mobile application behavior can result in inappropriate utilization of mobile resources—for example abnormal battery usage, unexpected talk and data usage, etc—that can compromise user privacy and/or lead to excess charges. Therefore, detection of malware, resource consumption abnormalities and other factors that contribute to exceptional behavior of the mobile application is a focus for many mobile security researchers.

Abnormal behavior is regarded as any behavior that deviates from what is considered normal or expected. The term “anomaly detection” generally means finding behavioral patterns that are not expected, such as data exchange, application crashes, personal information leakage, mobile function abuse, obtaining root privilege, and the like.

While basing anomaly detection on statistical analysis and understanding of common behaviors is effective, there are several limitations to this approach. First, there is the need to analyze offline a large set of executions. Second, this setting mandates the identification of useful features, or behaviors, with respect to which anomaly detection is performed. Third, as statistical analysis is agnostic to natural language meaning or context, its detection capabilities are limited to superficial-level “black-box” outlier and feature detection. Finally, if the training executions are heterogeneous, then the resulting model enables great freedom for malicious applications to apply undesired actions while still falling within what is considered the normal behavior range.

Aspects of the invention include a method for detection of unintended application behaviors, whereby natural language processing (NLP) techniques and in particular statistical NLP, which relies on stochastic, probabilistic or statistical methods to perform automatic summarization of a chunk of text or perform discourse analysis; or machine-learning-based NLP, which reduces the input data to a feature-vector representation to achieve these same goals, are used to analyze the application, and specifically its GUI, and construct an acceptable (or expected) list of expected operations per-context actions. The system is trained using a known properly functioning application for monitoring operation of the application to construct the list of expected operations. Actions executed by the application in a given context that do not fall within the list are flagged as unexpected (or anomalous). A warning flag is an indication that there is an anomaly encountered while performing the application and may be a dialog box visible to a user enabling the user to respond the unexpected application behavior. The method applies both online, as a means to enforce security/privacy considerations in real time, or offline, as a tool to test applications for security/privacy threats

Aspects of the invention are described in the following example. A mobile application that features location-aware services, enabling the user to search for certain attractions (or places of interest) in a given area, is sending a text message to a premium phone number whenever the search button is hit. Communication with the user, as reflected in user interface (UI) labels as well as messages displayed by the application, does not suggest that this would happen, and so the user will likely face unexpected charges. This is an anomalous pattern of behavior, as (well-behaved) applications that perform actions such as sending text messages normally secure user awareness before doing so. A warning will be issued to advise the user of an unexpected application behavior and potential malware or anomalous operation.

In the following example, using NLP, we construct a statistical model to determine for an application A—given A's GUI and unlabeled text—whether A is exhibiting unintended, or abnormal, behavior. Steps comprising an aspect of the method are described below. Referring toFIG. 1, there is shown a flow chart100of an aspect of the invention.

Steps: Given application A as input, an exemplary method comprises the following steps:

(step102) Instrument A and monitoring behavior of the application, such that (1) UI transitions are recorded (including changing the selection of the enabled UI widget), and (2) security-critical operations are monitored (e.g., sending text messages or HTTP messages, creating files, reading sensitive identifiers, etc). Instrumenting application A may be performed (assuming A is written in Java) by static/runtime bytecode editing (using e.g. java assist or ASM).

(step104) As UI transitions occur, and are captured by the instrumentation, utilize either OCR (i.e.: optical character recognition) or code analysis to obtain the text shown to the user associated with the enabled UI widget(s).

(step106) Form a sequence of textual content out of the user's interactions with the application. Interleaved into this sequence are security-relevant operations that occur in between UI transitions.

(step108) As a security-critical operation is about to execute, analyze some, or all, of the suffix leading to the operation to determine whether the operation is expected to occur at this point in execution based upon NLP acceptable list, and specifically its GUI, and in particular natural language understanding.

(step110) Based on identification of the intended semantic, what one would expect for operations is a properly functioning application, the system qualifies whether the application's interaction is consistent with expected behavior, or otherwise, raise a warning flag.

The following is an example where the method can be applied to detect intended or unexpected application behavior.

EXAMPLE

Consider a scenario where a user logs into a travel reservation application, then inputs a location (e.g., “New York”) and performs a search for available hotels at that location. The sequence of textual content would roughly be:

As such, we would expect the specified identifiers to be released, but not other identifiers, and similarly, we would expect the URL to point to a travel/hotel reservation website. Further, as the query involves finding a hotel, we would expect the information to be released via an HTTP request rather than, for example, an SMS message.

If other than these events are detected, then a flag is raised indicating a potential malware or anomalous event is occurring with the application.

The described method steps can be applied online, as a means to enforce security/privacy considerations in real time, or offline, as a tool to test applications for security/privacy threats.

The components of computer system may include, but are not limited to, one or more processors or processing units202, a system memory206, and a bus204that couples various system components including system memory206to processor202. The processor202may include a module200that performs the methods described herein. The module200may be programmed into the integrated circuits of the processor202, or loaded from memory206, storage device208, or network214or combinations thereof.

System memory206can include computer system readable media in the form of volatile memory, such as random access memory (RAM) and/or cache memory or others. Computer system may further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only, storage system208can be provided for reading from and writing to a non-removable, non-volatile magnetic media (e.g., a “hard drive”). Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a “floppy disk”), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media can be provided. In such instances, each can be connected to bus204by one or more data media interfaces.

Computer system may also communicate with one or more external devices216such as a keyboard, a pointing device, a display218, etc.; one or more devices that enable a user to interact with computer system; and/or any devices (e.g., network card, modem, etc.) that enable computer system to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces210.

Still yet, computer system can communicate with one or more networks214such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet) via network adapter212. As depicted, network adapter212communicates with the other components of computer system via bus204. It should be understood that although not shown, other hardware and/or software components could be used in conjunction with computer system. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.