Patent Description:
Electronic devices running applications manage permissions for applications to access the electronic devices' resources, such as the devices' cameras or microphones. The management of such permissions may be defined according to a permission set at the time of the application's installation on the electronic device or at a subsequent runtime. The initially set permission may create opportunities for the application to access resources under circumstances unforeseen or undesired by the user at the time the permission was initially set by a user. For example, a user of a mobile terminal may, at the time of installing a social media application, grant the application permission to access the mobile terminal's microphone, on the assumption that the application would only access the microphone when the social media application was open and actively being used by the user. However, in some cases, the application may abuse this permission and access the microphone under unwanted circumstances, such as while the application is only running as a background process, which can result in undesirable breaches of a user's privacy, for example, recording the user's conversations without her permission. Patent document <CIT> represents relevant prior art.

This disclosure provides apparatuses and methods for monitoring suspicious application access.

For a more complete understanding of this disclosure and its advantages, reference is now made to the following description, taken in conjunction with the accompanying drawings, in which:.

<FIG>, discussed below, and the various embodiments used to describe the principles of this disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of this disclosure may be implemented in any suitably arranged wireless communication system.

<FIG> illustrates an example of a device for implementing monitoring of suspicious application access according to this disclosure. The embodiment of device <NUM> illustrated in <FIG> is for illustration only, and other configurations are possible. However, suitable devices come in a wide variety of configurations, and <FIG> does not limit the scope of this disclosure to any particular implementation of a device.

As shown in <FIG>, the device <NUM> includes a communication unit <NUM> that can include, for example, a radio frequency (RF) transceiver, a Bluetooth® transceiver, or a Wi-Fi® transceiver, etc., transmit (TX) processing circuitry <NUM>, a microphone <NUM>, and receive (RX) processing circuitry <NUM>. The device <NUM> also includes a speaker <NUM>, a main processor <NUM>, an input/output (I/O) interface (IF) <NUM>, input/output device(s) <NUM>, and a memory <NUM>. The memory <NUM> includes an operating system (OS) program <NUM> and one or more applications <NUM>.

Applications <NUM> can include games, social media applications, applications for geotagging photographs and other items of digital content, virtual reality (VR) applications, augmented reality (AR) applications, operating systems, device security (e.g., anti-theft and device tracking) applications or any other applications which access resources of device <NUM>, the resources of device <NUM> including, without limitation, speaker <NUM>, microphone <NUM>, input/output devices <NUM>, and additional resources <NUM>. Further, applications <NUM> can include applications containing program code that when executed by a processor, such as main processor <NUM>, cause the processor to perform steps of methods for monitoring suspicious application access according to certain embodiments of the present disclosure.

The communication unit <NUM> can receive an incoming RF signal, for example, a near field communication signal such as a Bluetooth or Wi-Fi signal. The communication unit <NUM> can down-convert the incoming RF signal to generate an intermediate frequency (IF) or baseband signal. The IF or baseband signal is sent to the RX processing circuitry <NUM>, which generates a processed baseband signal by filtering, decoding, or digitizing the baseband or IF signal. The RX processing circuitry <NUM> transmits the processed baseband signal to the speaker <NUM> (such as for voice data) or to the main processor <NUM> for further processing (such as for web browsing data, online gameplay data, notification data, or other message data).

The TX processing circuitry <NUM> receives analog or digital voice data from the microphone <NUM> or other outgoing baseband data (such as web data, e-mail, or interactive video game data) from the main processor <NUM>. The TX processing circuitry <NUM> encodes, multiplexes, or digitizes the outgoing baseband data to generate a processed baseband or IF signal. The communication unit <NUM> receives the outgoing processed baseband or IF signal from the TX processing circuitry <NUM> and up-converts the baseband or IF signal to an RF signal for transmission.

The main processor <NUM> can include one or more processors or other processing devices and execute the OS program <NUM> stored in the memory <NUM> in order to control the overall operation of the device <NUM>. For example, the main processor <NUM> could control the reception of forward channel signals and the transmission of reverse channel signals by the communication unit <NUM>, the RX processing circuitry <NUM>, and the TX processing circuitry <NUM> in accordance with well-known principles. In some embodiments, the main processor <NUM> includes at least one microprocessor or microcontroller.

The main processor <NUM> is also capable of executing other processes and programs resident in the memory <NUM>. The main processor <NUM> can move data into or out of the memory <NUM> as required by an executing process. In some embodiments, the main processor <NUM> is configured to execute the applications <NUM> based on the OS program <NUM> or in response to inputs from a user or applications <NUM>. Applications <NUM> can include applications specifically developed for the platform of device <NUM>, or legacy applications developed for earlier platforms. Additionally, main processor <NUM> can be manufactured to include program logic for implementing methods for monitoring suspicious application access according to certain embodiments of the present disclosure. The main processor <NUM> is also coupled to the I/O interface <NUM>, which provides the device <NUM> with the ability to connect to other devices such as laptop computers and handheld computers. The I/O interface <NUM> is the communication path between these accessories and the main processor <NUM>.

The main processor <NUM> is also coupled to the input/output device(s) <NUM>. The operator of the device <NUM> can use the input/output device(s) <NUM> to enter data into the device <NUM>. Input/output device(s) <NUM> can include keyboards, touch screens, mouse(s), track balls or other devices capable of acting as a user interface to allow a user to interact with electronic device <NUM>. In some embodiments, input/output device(s) <NUM> can include a touch panel, a virtual reality headset, a (digital) pen sensor, a key, or an ultrasonic input device.

Input/output device(s) <NUM> can include one or more screens, which can be a liquid crystal display, light-emitting diode (LED) display, an optical LED (OLED), an active matrix OLED (AMOLED), or other screens capable of rendering graphics.

The memory <NUM> is coupled to the main processor <NUM>. According to certain embodiments, part of the memory <NUM> includes a random access memory (RAM), and another part of the memory <NUM> includes a Flash memory or other read-only memory (ROM). Although <FIG> illustrates one example of a device <NUM>. Various changes can be made to <FIG>.

For example, according to certain embodiments, device <NUM> can further include a separate graphics processing unit (GPU) <NUM>.

According to certain embodiments, electronic device <NUM> can include a variety of additional resources <NUM> which can, if permitted, be accessed by applications <NUM>. According to certain embodiments, additional resources <NUM> include an accelerometer or inertial motion unit <NUM>, which can detect movements of the electronic device along one or more degrees of freedom. Additional resources <NUM> include, in some embodiments, a user's phone book <NUM>, one or more cameras <NUM> of electronic device <NUM>, and a global positioning system <NUM>.

Although the communication unit <NUM>, the TX processing circuitry <NUM>, the microphone <NUM>, the RX processing circuitry <NUM>, the speaker <NUM>, the main processor <NUM>, the I/O IF <NUM>, the input/output device(s) <NUM>, the memory <NUM>, the GPU, and the additional resources are described as separate units in the device <NUM> in <FIG>, it is to be understood that the device <NUM> may be implemented with a form into which at least two of the communication unit <NUM>, the TX processing circuitry <NUM>, the microphone <NUM>, the RX processing circuitry <NUM>, the speaker <NUM>, the main processor <NUM>, the I/O IF <NUM>, the input/output device(s) <NUM>, the memory <NUM>, the GPU, and the additional resources are incorporated.

Although <FIG> illustrates one example of a device <NUM> for implementing monitoring of suspicious application access, various changes can be made to <FIG>. For example, the device <NUM> could include any number of components in any suitable arrangement. In general, devices including computing and communication systems come in a wide variety of configurations, and <FIG> does not limit the scope of this disclosure to any particular configuration. While <FIG> illustrates one operational environment in which various features disclosed in this patent document can be used, these features could be used in any other suitable system.

<FIG> illustrates an example of a block diagram of the components of a system for monitoring suspicious application access according to this disclosure.

According to some embodiments, the components of a system for monitoring suspicious application access include a monitor <NUM>, a permission monitor <NUM> and a configurator <NUM>. Each of monitor <NUM>, permission monitor <NUM> and configurator <NUM> can be implemented on a device (such as device <NUM> of <FIG>) via hardware or software. According to some embodiments, monitor <NUM> and permission monitor <NUM> can be implemented one or more layers <NUM> in the protocol stack of the device below the application layer. In the illustrative example of <FIG>, monitor <NUM> and permission monitor <NUM> operate at lower layers <NUM>, while configurator <NUM> operates on the application layer, above lower layers <NUM>.

In some embodiments, one or more applications <NUM> are running on an electronic device. Applications <NUM> include, without limitation, social media applications, games, photo and video editing applications, voice over internet protocol (VOIP) call applications, and map or navigational applications. Each of applications <NUM> can be permitted access to one or more resource <NUM> of the device. Resource <NUM> can include, without limitation, the device's camera, fingerprint sensor, global positioning system sensor, phone book containing a device user's contacts, and the device's microphone.

According to some embodiments, when an application <NUM> is first installed on the electronic device, the operating system of the device will prompt the user to grant or deny application <NUM> permission to one or more resources <NUM>. For example, a user of a mobile device, such as a smartphone, can install a social media application and grant it permission to access the camera the smartphone's camera. According to certain embodiments, each time the social media application attempts to access the camera, the operating system determines whether the social media application was, at the time of installation, granted permission to access the camera, and if so, permits the social media application to access the camera. According to other embodiments, the operating system determines whether the social media application requested permission to access the camera at a runtime after installation. According to certain embodiments, an ongoing monitoring functionality is provided, and the security risks associated with an application utilizing the user's permission for the application to access the camera in contexts beyond the intended scope of user's permission (for example, when the application is open, but the camera functionality has not been selected, or when the application is running as a background process) are reduced.

However, depending on the operating system of the electronic device, the social media application can retain its permission to access the smartphone's camera almost indefinitely, without requiring any further confirmation by the user as to the scope of the application's permission to access the camera. To revoke the social media application's permission to access the smartphone's camera, a user may, depending on the operating system of the electronic device, have to uninstall the application, or manually revoke the social media application's permission to access the camera. However, many users are not aware of the scope and/or duration of such applications' permission to access the camera. As such, social media applications and other applications can abuse the user's initial grant of permission to cameras and other device resources by, for example, accessing the resource without a user prompt, or as a background process. This can lead to, for example, the application recording the user's conversations without the user's knowledge or accessing the user's location information without the user's knowledge.

To decrease the risk of application <NUM> abusing the permissions to resource <NUM> set by the operating system of the device at the time of installation or at a post-installation runtime, monitor <NUM> resides on the one or more critical access paths between application <NUM> and resource <NUM>, thereby making resource <NUM> a monitored resource. According to certain embodiments, monitor <NUM> is configured to identify that application <NUM> is attempting to access resource <NUM>, and that the operating system has confirmed that application <NUM> was, at the time of its installation or at a post-installation runtime, given permission to access resource <NUM>, and granted the operating system's permission to access resource <NUM>. In response to identifying, subsequent to an initial access permission check performed by the operating system, an access attempt by application <NUM>, monitor <NUM> calls permission monitor <NUM>, to report the access attempt. Permission monitor <NUM> determines whether the access attempt by application <NUM> involves suspicious activity by evaluating the potential risk associated with the attempt by application <NUM> to access monitored resource <NUM>. According to certain embodiments of the present disclosure, and as will be described in greater detail herein, permission monitor <NUM> evaluates the potential risk associated with the access attempt based on, for example, predefined rules and contextual information regarding the access attempt. According to some embodiments, multiple monitors <NUM> can be implemented on a single device, with each monitor <NUM> corresponding to an access permission for a resource on the device.

According to some embodiments, permission monitor <NUM> is configured to interface with configurator <NUM>. Configurator <NUM> which can be implemented by software, hardware or a combination thereof, is configured to receive notice from permission monitor <NUM> that application <NUM>'s access attempt to resource <NUM> involves suspicious activity, to provide a graphical user interface (GUI) to a display (such as a display among input/output devices <NUM> shown in <FIG>) providing a notification of the access attempt. According to some embodiments, the GUI provided by configurator <NUM> also is configured to receive inputs selecting one or more courses of action (e.g., allowing access to the resource, denying access to the resource, reporting the access attempt, etc.).

Although <FIG> illustrates one example of a block diagram of a system for monitoring suspicious application access, various changes can be made to <FIG>. For example, monitor <NUM> and/or permission monitor <NUM> may operate on other levels of the device's protocol stack. Alternatively, monitor <NUM> and permission monitor <NUM> can be combined in a single component of the system.

<FIG> illustrates, in flowchart form, operations of an exemplary method <NUM> for monitoring suspicious application access on an electronic device according to this disclosure.

In the example of <FIG>, at operation <NUM>, a device, (such as device <NUM> shown in <FIG>) performs an initial determination of whether an application running on the device has been granted permission to access a resource on the device, such as a phone book, location information, camera, etc. According to some embodiments, operation <NUM> is performed by the operating system of the device at runtime for the application, and the determination is based on permissions set by the user at the time the application was installed, or in response to a post-installation request by the application. According to certain embodiments, the determination performed in operation <NUM> is based on permissions updated by the user through a "settings" menu at some time after the application was installed. If, at operation <NUM>, it is determined that the application does not have a pre-set permission to access the monitored resource, the method proceeds to operation <NUM>, and the application is denied access to the monitored resource.

According to certain embodiments, if it is determined at operation <NUM> that the application has previously been granted permission to access the monitored resource, the method proceeds to operation <NUM>. At operation <NUM>, the application attempting to access the monitored resource is identified. According to some embodiments, operation <NUM> is performed by a monitor operating on the device (such as monitor <NUM> shown in <FIG>). According to certain embodiments, the monitor is implemented as a code snippet on each of the critical access paths to the monitored resource. Further, according to some embodiments, the identification of the application at operation <NUM> further comprises collecting information, such as a unique user ID (UID) or process ID (PID) for the application, permission information related to the application, a time stamp associated with the access attempt, or information regarding the monitored resource's and/or application's access history.

According to certain embodiments, following operation <NUM>, the method proceeds to operation <NUM>, where a determination is performed to determine whether the access attempt involves suspicious activity by evaluating a potential risk associated with the application. In some embodiments, operation <NUM> is performed by a permission monitor (for example, permission monitor <NUM> in <FIG>) using information generated or collected by the monitor as part of operation <NUM>. The evaluation of the potential risk can be based on, without limitation, a unique user ID (UID) or process ID (PID) for the application, permission information related to the application, a time stamp associated with the access attempt, information regarding the monitored resource's and/or application's access history or other contextual information regarding the access attempt.

If, at operation <NUM>, it is determined that there is no potential risk associated with the access request, according to certain embodiments, the method proceeds to operation <NUM>, wherein the access request is granted and the application is permitted to access the monitored resource.

If, at operation <NUM>, it is determined that there is a potential risk associated with the access attempt, the method proceeds to operation <NUM>, wherein responsive action is taken. According to certain embodiments, at operation <NUM>, the permission monitor calls a configurator (such as configurator <NUM> shown in <FIG>), notifying the configurator of the risk associated with the access attempt. According to some embodiments, as part of operation <NUM>, the configurator provides a GUI to the display, wherein the GUI provides a notification of the access attempt. According to certain embodiments, as part of operation <NUM>, the configurator provides a GUI to the display, and the GUI is configured to receive an input corresponding to a response to the access attempt, such as permitting or denying the application access to the monitored resource. Alternatively, at operation <NUM>, instead of presenting a GUI notifying a user of the access attempt, the configurator may respond to the detected access attempt through the application of one or more predetermined rules. For example, the configurator may automatically block the application's access to the monitored resource.

Although <FIG> illustrates operations of one exemplary method for monitoring suspicious application access on an electronic device according to this disclosure, various changes may be made to <FIG>. For example, operation <NUM> may be removed from the method in cases where the operating system on the device does not provide any mechanism for a user to revoke or adjust an application's permissions to access a resource apart from uninstalling and reinstalling the application. In such cases, where there is little to no user control over applications' permissions to access resources, it may be reasonable to assume a potential risk attached to all access attempts. Further, for example, while shown as a series of steps, various steps in <FIG> could overlap, occur in parallel, occur in a different order, or occur multiple times.

<FIG> illustrates, in block diagram form, an example of a monitor <NUM> included as part of a system for monitoring suspicious application access according to certain embodiments of this disclosure. According to certain embodiments, monitor <NUM> includes access detector <NUM>, contextual information collector <NUM>, local information processor <NUM>, and communicator <NUM>. Certain embodiments of monitor <NUM> are possible, including embodiments adding or subtracting certain of the components shown in <FIG>.

Additionally, according to some embodiments, monitor <NUM> is configured along the critical access path(s) between an application <NUM> and a monitored resource <NUM>. According to certain embodiments, monitor <NUM> is one of a plurality of monitors running on a device along critical access paths between applications and monitored resources. Further, according to some embodiments, monitor <NUM> resides on the critical access path(s) between application <NUM> and a plurality of monitored resources <NUM>. The present disclosure contemplates a wide range of mappings between application(s), monitor(s) and monitored resource(s).

According to certain embodiments, monitor <NUM> is provided with access to system information <NUM>, which, in the non-limiting example of <FIG>, includes system properties information. According to certain embodiments, monitor <NUM> has access to read permission information for an application stored as part of system information <NUM>. In certain embodiments, system information <NUM> is stored as a persistent bit map recording the user's choices regarding an application's <NUM> permissions to access monitored resource(s) <NUM>. As will be discussed in greater detail elsewhere in this disclosure, monitor <NUM> and a permission monitor (for example permission monitor <NUM> shown in <FIG>) may have permission to read system information <NUM>, while a configurator (for example, configurator <NUM> shown in <FIG>) is configured to write to system information <NUM>. Monitor <NUM> can be configured to communicate with a permission monitor <NUM>.

According to certain embodiments, access detector <NUM> is implemented as software which monitors application activity or application behavior on the critical access path(s) between application <NUM> and monitored resource <NUM>. According to certain embodiments, access detector <NUM> is implemented as one or more code snippets on each of the critical access path(s) to the monitored resource <NUM>. According to still certain embodiments, access detector <NUM> is implemented as hardware.

According to certain embodiments, access detector <NUM> performs an initial query to determine whether monitoring application <NUM>'s access to monitored resource <NUM> has been enabled or disabled. For example, a user may be able to selectively enable and disable such monitoring through by adjusting settings through a GUI provided by a configurator, such as configurator <NUM> shown in <FIG>. As noted above, access detector <NUM> detects access attempts by application <NUM> to monitored resource <NUM>. In some embodiments, the detection of access attempts is implemented by detecting the operation of processes of application <NUM> associated with monitored resource <NUM>. According to some embodiments, when access detector <NUM> detects the operation of a process associated with monitored resource <NUM>, access detector <NUM> routes the detected process to contextual information collector <NUM>.

According to some embodiments, contextual information collector <NUM> collects information indicating the context of the access attempt. Such information includes, without limitation, the identity of the application <NUM> (for example, the application's UID or PID) attempting to access to monitored resource <NUM>, permission information related to the access, which according to some embodiments can be read from system information <NUM>, and a time stamp for the access attempt.

In some embodiments, the contextual information collected by contextual information collector <NUM> is passed to local information processor <NUM>. In the example of <FIG>, local information processor <NUM> processes the information collected by contextual information collector <NUM> to determine whether the access attempt involves suspicious activity. In some embodiments, this processing comprises the application of rules or permutations of sets of rules to the contextual information. For example, local information processor <NUM> can apply a rule flagging an access attempt as suspicious if there is contextual information showing a particular UID associated with the application. Similarly, local information processor <NUM> can apply a rule flagging an access attempt as suspicious in cases where there is contextual information showing that application <NUM> has not been opened by a user, and the access attempt is being performed as a background process.

If the processing by local information processor <NUM> determines that the access attempt is not suspicious, monitor <NUM> can permit application <NUM> to access monitored resource <NUM>. According to certain embodiments, this permission is performed by returning the process associated with the access attempt which was detected by access detector <NUM> to the critical access path for the process. In this way, monitor <NUM> acts as a checkpoint along the critical access path between application <NUM> and monitored resource <NUM>.

If, however, the processing by local information processor <NUM> determines that the access attempt is suspicious, local information processor <NUM> will, according to certain embodiments, notify communicator <NUM> and provide the collected contextual information regarding the access attempt and the determination by local information processor that the access attempt was suspicious. In response to receiving a notification from local information processor <NUM>, communicator <NUM> will call permission monitor <NUM> to report the access attempt.

Although <FIG> illustrates operations of one example of a monitor included as part of a system for monitoring suspicious application access on an electronic device according to this disclosure, various changes may be made to <FIG>. For example, monitor <NUM> may be configured to be "always on," thereby eliminating the operation of determining by access detector <NUM> whether monitor <NUM> is enabled.

<FIG> illustrates, in block diagram form, an example of a permission monitor <NUM> included as part of a system for monitoring suspicious application access according to certain embodiments of the present disclosure.

According to certain embodiments, permission monitor <NUM> is implemented as software, specifically as a system service launched by the device at boot time. Additionally, permission monitor is continuously running and able to accept calls and communications from monitor(s) <NUM>. According to certain embodiments, permission monitor <NUM> is implemented as hardware, including as part of a system on a chip (SoC) system.

According to the non-limiting example of <FIG>, the components of permission monitor <NUM> include caller authenticator <NUM>, permission monitor activator <NUM>, whitelist <NUM>, access risk evaluator <NUM>, rate limiter <NUM> and communicator <NUM>. Additionally, permission monitor <NUM> is communicatively connected to configurator <NUM> and has read access to system information <NUM>.

According to certain embodiments, caller authenticator <NUM> is configured to perform an analysis as to verify the identity of the caller to permission monitor <NUM>. Depending on the device and operating system, a malicious application may attempt to "spoof" systems for monitoring suspicious application access according to this disclosure by presenting a call to a permission monitor regarding access request, which appears to originate from an actor other than the malicious application. According to certain embodiments, caller authenticator <NUM> authenticates the call to permission monitor <NUM> received from monitor <NUM> and confirms that the call to permission monitor <NUM> was received from an actor authorized to call permission monitor <NUM>. This authentication can be based on a UID associated with the application. Depending on the operating system used by the device, it may be difficult or impossible to "spoof" a UID, or UIDs having certain characteristics have indicia of trustworthiness. For example, in devices using the Android operating system, the UID of a caller may be checked via a binder, such as "getCallingUid. " Where the UID has a value of less than <NUM>,<NUM>, this indicates that the caller is an application pre-loaded onto the device, such as a monitor or an application associated with a monitor, and thus inherently more trustworthy than an application having a UID of <NUM>,<NUM> or greater.

In cases where caller authenticator <NUM> is unable to authenticate the caller, the call is ignored, and the application will not be able to access the monitored resource. However, when caller authenticator <NUM> is able to authenticate the caller to permission monitor <NUM>, according to some embodiments, permission monitor activator <NUM> performs a determination as to whether permission monitoring of the application's access attempts to the given resource has been activated. To perform this determination, permission monitor activator <NUM> can read the system information <NUM> for the application to determine whether monitoring has been activated. In the non-limiting example of <FIG>, system information <NUM> includes system properties information.

According to certain embodiments, where permission monitor activator <NUM> confirms that permission monitoring for the given application has been activated, whitelist <NUM> performs a further analysis of the access attempt based on the information provided to process monitor <NUM> as part of an authenticated call from monitor <NUM>. According to certain embodiments, whitelist <NUM> operates to allow the application to access the monitored resource if certain criteria are satisfied. Such criteria include whether the application seeking access to the monitored resource is a pre-loaded application or whether the application belongs to a predefined list of trusted applications. According to certain embodiments, the predefined list of trusted applications is set according to a policy of the device manufacturer. Alternatively, according to some embodiments, after an access attempt pass review by whitelist <NUM>, the access attempt is further reviewed by access risk evaluator <NUM>.

According to some embodiments, access risk evaluator <NUM> operates as a process of permission monitor <NUM> to determine, based on the context and/or states of the application attempting access to a monitored resource, the potential risk of the access being requested. For example, access risk evaluator <NUM> can check whether the application has any threads running in the foreground of the device, whether the screen of the device is off or whether the screen or device itself is locked. In cases where access risk evaluator <NUM> determines the extent to which the context of the access request presents a potential risk. For example, access risk evaluator <NUM> can determine that an application's request to access the device's camera while the device itself is in a locked state presents a significant potential risk.

According to certain embodiments, rate limiter <NUM> operates to deny an application access to a monitored resource if the access attempt satisfies certain criteria. For example, rate limiter <NUM> can operate to deny or ignore an application's access attempt if the same type of access has been reported within a certain period of time. For example, rate limiter <NUM> can permit a social media application to occasionally access a device's location information, but to deny repeated accesses to this information, lest the application unwantedly operate as a tracker of a user's location.

According to certain embodiments, communicator <NUM> operates to send the outputs from access risk evaluator <NUM> and rate limiter <NUM> to configurator <NUM>. Depending on the operating system utilized by the device, communicator <NUM> can communicate with configurator <NUM> by sending configurator <NUM> an intent containing information about the application attempting access, information about the permissions for the application and/or the monitored resource, and contextual information regarding the access intent. Further, according to certain embodiments, the intent sent by communicator <NUM> to configurator <NUM> is an explicit intent, restricting the configurator as the receiver of the intent.

Although <FIG> illustrates components of one example of a permission monitor included as part of a system for monitoring suspicious application access on an electronic device according to this disclosure, various changes may be made to <FIG>. For example, whitelist <NUM> may be omitted or incorporated within access risk evaluator <NUM>.

<FIG> illustrates, in flowchart form, operations performed by a permission monitor in a method <NUM> for monitoring suspicious application access according to this disclosure.

In the non-limiting example of <FIG>, method <NUM> begins with operation <NUM>, wherein a permission monitor, such as permission monitor <NUM> shown in <FIG>, receives, from a monitor (such as monitor <NUM> shown in <FIG>) a call reporting an application attempting to access one or more monitored resources.

According to certain embodiments, method <NUM> then proceeds to operation <NUM>, in which a determination of whether the received call is authentic, or from an application "spoofing," or pretending to be, an application authorized to call the permission monitor, such as a monitor. The determination performed at operation <NUM> can be based on criteria, which depending on the operating system of the device, may be difficult to "spoof," such as an application's unique ID ("UID"). Certain embodiments are possible, wherein authentication is performed based on a different indicia, such as a process ID ("PID") associated with the call.

If, at operation <NUM>, the caller to the permission monitor cannot be authenticated, the process moves to operation <NUM>, and the permission monitor ignores the call.

If, however, the caller to the permission monitor is authenticated, according to certain embodiments, method <NUM> advances to operation <NUM>, wherein the permission monitor reads system property information from a configuration file to confirm the current permissions for the application attempting access to the monitored resource. According to certain embodiments, system information are stored locally with the permission monitor, and method <NUM> proceeds to operation <NUM>.

According to certain embodiments, at operation <NUM>, a determination is performed as to whether access attempt involves an application or a resource for which permission is monitored. According to certain embodiments, monitoring is persistent and universal, and operation <NUM> may be avoided.

If, at operation <NUM>, a determination is made that the application is not an application whose permission to access a resource is monitored, or that the resource is not a resource for which permission to access is monitored, or that the combination of application and resource presented by the access attempt is not one for which permission is monitored, method <NUM> may proceed to operation <NUM>, wherein the call to the permission monitor is ignored. According to some embodiments and depending on configuration, this results in the application being denied access to the resource.

If, at operation <NUM>, a determination is made that the combination of application and/or resource presented by the access attempt is one for which permission is monitored, method <NUM> advances to operation <NUM>, wherein a determination is performed as to whether the application attempting to access the monitored resource is a whitelisted application.

If, at operation <NUM>, it is determined that the application is not a whitelisted application, method <NUM> advances to operation <NUM>, wherein the call to the permission monitor is ignored. According to some embodiments and depending on configuration, this results in the application being denied access to the resource. If, at operation <NUM> it is determined that the application is a whitelisted application, method <NUM> proceeds to operation <NUM>.

According to certain embodiments, at operation <NUM>, the permission monitor, or a component thereof (such as access risk evaluator <NUM> shown in <FIG>), evaluates the risk to the device user's privacy or security associated with the access attempt based on, without limitation, contextual information regarding the access attempt. Contextual information includes, without limitation, the current status of the device (e.g., "is the screen on?", "Is the device locked?", or "Is Wi-Fi connectivity turned on?") and one or more user privacy preferences. Further examples of contextual information include device integrity status (for example, whether preloaded official software, including the OS, has been modified), battery level, device location, type of wireless connectivity (e.g., cellular network, Bluetooth, Wi-Fi, or NFC), currently running applications and services, user interactions, current time, a usage history for the device, information from built-in sensors of the device (for example, accelerometers and gyroscopes) and user privacy preferences stored in a settings file. According to certain embodiments, user privacy preferences can be set in response to a notification from one or more components of a system for monitoring suspicious application access (such as configurator <NUM> shown in <FIG>). According to some embodiments, the relevant user privacy preferences can be set at the time the application is installed on the device, or in response to a request from the application at a post-installation runtime. At operation <NUM>, the permission monitor and/or a component thereof, outputs its determination of the potential risk associated with the access attempt. According to some embodiments, the output can be represented as a number, a string and/or a bitmap.

According to certain embodiments, subsequent to operation <NUM>, method <NUM> proceeds to operation <NUM>. At operation <NUM>, a permission monitor or one or more components thereof performs rate limiting on the application. According to some embodiments, at operation <NUM>, a permission monitor determines the time interval since the application last attempted to access the monitored resource, and if the time interval is less than a threshold value, causes the permission monitor to ignore the application's access attempt or otherwise denies the application's access attempt. By ignoring reports within a specified time period, the performance of a device may be optimized, in that processing and battery resources are not wasted generating redundant access reports for multiple instances of the same suspicious access attempt.

According to certain embodiments, subsequent to operation <NUM>, method <NUM> proceeds to operation <NUM>, wherein the permission monitor or one or more components thereof (such as communicator <NUM> shown in <FIG>) generates an intent reporting the access attempt. The generated intent can, depending on embodiments, include additional information associated with the access attempt, including without limitation, information regarding the application performing the access attempt, information regarding the permissions for the application and/or the monitored resource, and contextual information regarding the access attempt. By ignoring a.

Finally, according to certain embodiments, method <NUM> proceeds to operation <NUM>, wherein the permission monitor or one or more components thereof (such as communicator <NUM> shown in <FIG>) calls a configurator (such as configurator <NUM> shown in <FIG>) and sends the generated intent to the configurator. According to some embodiments and depending on the operating system used by the device, the generated intent is an explicit intent to be received only by a configurator.

Although <FIG> illustrates operations of one example of method for monitoring suspicious application access on an electronic device according to this disclosure, various changes may be made to <FIG>. For example, the sequence of operations shown in <FIG> may be reordered and steps added or omitted. As previously noted, operation <NUM> may, depending on embodiments, be omitted. Additionally, for example, operations <NUM>, <NUM> and <NUM> may be performed in a different order than shown in <FIG>. Numerous variations are possible. Further, for example, while shown as a series of steps, various steps in <FIG> could overlap, occur in parallel, occur in a different order, or occur multiple times.

<FIG> illustrates, in block diagram format, an example of a configurator <NUM> included as part of a system for monitoring suspicious application access according to this disclosure.

In the non-limiting example of <FIG>, the components of configurator <NUM> include access notification generator <NUM>, access history manager <NUM> and configuration manager <NUM>. Further, as shown in <FIG>, configurator <NUM> can be connected to a permission monitor <NUM>, a monitor <NUM>, system information <NUM> and a user <NUM>.

According to certain embodiments, configurator <NUM> is configured to receive calls, from permission monitor <NUM> reporting attempts by applications to access monitored resources, which permission monitor <NUM> has determined to involve suspicious activity and/or present a potential risk associated with the application accessing the monitored resource. According to some embodiments, upon receipt of a call reporting a suspicious access attempt, access notification generator <NUM> presents a visible notification to a user <NUM> notifying them that a suspicious access attempt has been performed. According to certain embodiments, presenting a visible notification to user <NUM> comprises providing a graphical user interface (GUI) to the user. In addition to reporting the fact of the suspicious access attempt, the GUI also can be configured to receive, from user <NUM> an input selecting a response to the suspicious access attempt. Selected responses can include providing the application with a one-time permission to access the resource, performing a one-time denial of access, or updating a permission setting associated with the application and/or resource.

According to certain embodiments, access notification generator <NUM> provides an audible notification of the suspicious access attempt, such as a "beep. " Other forms of notifications, such as haptic notifications, (e.g., a vibration), or a transmission to a connected device (such as a computer or smart watch).

According to certain embodiments, access notification generator <NUM> interfaces with access history manager <NUM>. According to certain embodiments, access history manager <NUM> stores and maintains a history of an application's access attempts to a particular resource as reported to configurator <NUM> by permission monitor <NUM>. According to some embodiments, a GUI provided by access notification generator <NUM> includes information showing access history information maintained by access history manager <NUM>. For example, access history manager <NUM> can maintain information showing that a particular application has repeatedly attempted to access a device's microphone while the device is in a locked state. Such contextual information can be useful to a user trying to understand the significance and risk associated with an access attempt.

Further, according to some embodiments, configurator <NUM> includes configuration manager <NUM>. Configuration manager <NUM> can, depending on embodiments, interface with monitor <NUM> and/or write to system information <NUM> to allow a user to turn a suspicious application access monitor on or off for the whole device, for a particular permission, for a particular resource, or for an application. According to some embodiments, configuration manager <NUM> receives inputs turning a monitor on or off from a GUI provided by access notification generator <NUM>.

Although <FIG> illustrates of one example of a configurator included in a system for monitoring suspicious application access on an electronic device according to this disclosure, various changes may be made to <FIG>. For example, configuration manager <NUM> may be configured to turn monitors on according to predefined rules or user-indicated logic, rather than in response to specific instructions turning a particular monitor on or off. For example, a user may indicate a general preference for a high level of security, and configuration manager may, based on predefined rules or application logic, configure and activate a set of particular monitors.

<FIG> illustrates, in block diagram format, management of application permission monitoring settings using system information according to this disclosure. In the non-limiting example of <FIG>, the components a system for monitoring suspicious application access include a plurality of monitors <NUM> and <NUM> (such as monitor <NUM> shown in <FIG>), a permission monitor <NUM> (such as permission monitor <NUM> shown in <FIG>) and a configurator <NUM> (such as configurator <NUM> shown in <FIG>). In the example of <FIG>, each of monitor <NUM>, monitor <NUM> and permission monitor <NUM> has permission to read system information <NUM>. By the same token, configurator <NUM> has permission to write to system information <NUM>. According to certain embodiments, system information <NUM> includes system properties information.

According to certain embodiments, configuration information (including, without limitation, permission settings, rules, and monitor activation settings for the system for monitoring suspicious application access) is stored within system information <NUM> as a bitmap designated "Persist. permission. " According to certain embodiments, system information <NUM> is configured such that, for each application, a flag is set, the flag indicating whether the application's attempt to access a particular resource is to be reported.

According to an embodiment of the disclosure provides a non-transitory computer-readable medium comprising program code, that when executed by a processor, causes a system to: identify an access attempt to a monitored resource by an application, the identification occurring after an access permission check is performed; determine whether the access attempt involves suspicious activity by evaluating a potential risk associated with the application accessing the monitored resource; and in response to determining that the access attempt involves suspicious activity, provide a graphical user interface (GUI) to a display, the GUI providing a notification of the access attempt.

Herein, the non-transitory computer-readable medium further comprises code, that when executed by the processor, causes the system to: collect context information associated with the access attempt; and determine whether the access attempt involves suspicious activity based in part on the collected context information.

Herein, the non-transitory computer-readable medium further comprises code, that when executed by the processor, causes the system to: implement a monitor on a critical access path to the monitored resource, the monitor configured to identify the access attempt to the monitored resource by the application; implement a permission monitor, the permission monitor configured to receive, via the monitor, a call reporting the access attempt to the monitored resource by the application; and authenticate the call to the permission monitor.

Herein, the permission monitor is configured to implement at least one of a whitelist, an access risk evaluator, a rate limiter or a communicator.

Herein, the non-transitory computer-readable medium further comprises code, that when executed by the processor, causes the system to: implement a configurator, the configurator configured to receive, from the permission monitor, a notification of the access attempt by the application, wherein the configurator is configured, in response to receiving the notification, to perform at least one of permitting the application to access to the monitored resource, blocking the application from accessing the monitored resource or generating the GUI providing a notification of the access attempt.

Herein, the non-transitory computer-readable medium further comprises code, that when executed by the processor, causes the system to: implement a second monitor on a critical access path to a second monitored resource; identify, after a second access permission check is performed, a second access attempt to a second monitored resource by a second application, transmit, from the second monitor to the permission monitor, a second call reporting a second access attempt, the second access attempt performed by the second application attempting to access the second monitored resource; and determine whether the second access attempt involves suspicious activity by evaluating a potential risk associated with the second application accessing the monitored resource.

Certain aspects of the present disclosure may also be embodied as computer readable code on a non-transitory computer readable recording medium. A non-transitory computer readable recording medium is any data storage device that can store data, which can be thereafter read by a computer system. Examples of the non-transitory computer readable recording medium include ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and carrier waves (such as data transmission through the Internet). The non-transitory computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, code, and code segments for accomplishing the present disclosure can be easily construed by programmers skilled in the art to which the present disclosure pertains.

It can be appreciated that a method and apparatus according to an embodiment of the present disclosure may be implemented by hardware, software and/or a combination thereof. The software may be stored in a non-volatile storage, for example, an erasable or re-writable ROM, a memory, for example, a RAM, a memory chip, a memory device, or a memory integrated circuit (IC), or an optically or magnetically recordable non-transitory machine-readable (e.g., computer-readable), storage medium (e.g., a compact disk (CD), a digital video disc (DVD), a magnetic disk, a magnetic tape, and/or the like). A method and apparatus according to an embodiment of the present disclosure may be implemented by a computer or a mobile terminal that includes a controller and a memory, and the memory may be an example of a non-transitory machine-readable (e.g., computer-readable), storage medium suitable to store a program or programs including instructions for implementing various embodiments of the present disclosure.

The present disclosure may include a program including code for implementing the apparatus and method as defined by the appended claims, and a non-transitory machine-readable (e.g., computer-readable), storage medium storing the program. The program may be electronically transferred via any media, such as communication signals, which are transmitted through wired and/or wireless connections, and the present disclosure may include their equivalents.

An apparatus according to an embodiment of the present disclosure may receive the program from a program providing device which is connected to the apparatus via a wire or a wireless and store the program. The program providing device may include a memory for storing instructions which instruct to perform a content protect method which has been already installed, information necessary for the content protect method, and the like, a communication unit for performing a wired or a wireless communication with a graphic processing device, and a controller for transmitting a related program to a transmitting/receiving device based on a request of the graphic processing device or automatically transmitting the related program to the transmitting/receiving device.

While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present disclosure as defined by the appended claims.

Although the present disclosure has been described with exemplary embodiments, various changes and modifications may be suggest to one skilled in the art. It is intended that the present disclosure encompass such changes and modifications as fall within the scope of the appended claims.

Claim 1:
An apparatus for monitoring access to resources of the apparatus comprising:
a display;
a memory for storing instructions; and
one or more processors configured to execute the stored instructions to:
grant, to an application, an access permission for a resource of the apparatus during a runtime of the application after installation of the application at the apparatus;
identify an access attempt by the application for the resource, the identification occurring after the access permission for the resource of the apparatus is granted to the application during the runtime of the application after installation of the application at the apparatus;
based on the access permission granted to the application, determine that the access attempt is related to a suspicious activity by evaluating the access attempt from the application accessing the resource of the apparatus; and
based on determining that the access attempt is related to the suspicious activity, display, on the display, a graphical user interface, GUI, including a notification of the access attempt related to the suspicious activity,
based on a user input, perform at least one of permitting the application to access the resource or blocking the application from accessing the resource.