Systems and methods for detecting usage anomalies based on environmental sensor data

Disclosed herein are systems and method for detecting usage anomalies based on environmental sensor data. A method may include: receiving a physical user input at a computing device located in an environment; determining whether the physical user input was received from an authorized user of the computing device by: retrieving environmental sensor data from at least one sensor located in the environment; identifying a window of time during which the physical user input was received; and verifying a presence of the authorized user at the environment during the window of time based on the environmental sensor data; and in response to determining that the authorized user was not present in the environment during the window of time, detecting a usage anomaly and not executing the physical user input.

FIELD OF TECHNOLOGY

The present disclosure relates to the field of data security, and, more specifically, to systems and methods for detecting usage anomalies based on environmental sensor data.

BACKGROUND

Conventional data security systems often rely on rudimentary authentication procedures (e.g., a typed password, a fingerprint, a face match, etc.) to provide access to protected data. In some cases, there may be two-step authentication in which the user confirms his/her identity using a verification code emailed or texted to a trusted device registered in the security system. However, in either case, the authentication can be easily forged.

Moreover, existing security solutions are limited to software-based traces and attacks. For example, applications such as ZeroTrust and user and entity behavior analytics (UEBA) commonly focus on information technology (IT) software events such as logins, program starts, etc. Such solutions are not very efficient and have a high false-positive rate with insider attacks.

There is an increasing trend in compromised credentials and attacks in the physical environment (e.g., unlocked machine in the office being used by cleaning staff to do malicious activity). There thus exists a need to detect usage anomalies that cannot be determined using software events alone.

SUMMARY

In one exemplary aspect, the techniques described herein relate to a method for detecting usage anomalies based on environmental sensor data, the method including: receiving a physical user input at a computing device located in an environment; determining whether the physical user input was received from an authorized user of the computing device by: retrieving environmental sensor data from at least one sensor located in the environment; identifying a window of time during which the physical user input was received; and verifying a presence of the authorized user at the environment during the window of time based on the environmental sensor data; and in response to determining that the authorized user was not present in the environment during the window of time, detecting a usage anomaly and not executing the physical user input.

In some aspects, the techniques described herein relate to a method, further including: notifying the authorized user of the usage anomaly.

In some aspects, the techniques described herein relate to a method, further including: in response to determining that the authorized user is present in the environment, correlating the user input with the environmental sensor data by time to verify whether the authorized user had physical access to the computing device; and in response to determining that the authorized user had physical access to the computing device, executing the physical user input.

In some aspects, the techniques described herein relate to a method, further including: in response to determining that the authorized user did not have physical access to the computing device, not executing the physical user input and notifying the authorized user of the usage anomaly.

In some aspects, the techniques described herein relate to a method, wherein correlating the user input with the environmental sensor data by time further includes: determining whether the environmental sensor data indicates whether the authorized user left the environment before the physical user input was received.

In some aspects, the techniques described herein relate to a method, wherein the at least one sensor includes one or more of: (1) a camera installed at the environment, (2) a door badge scanner at the environment, (3) a light sensor at the environment, (4) a microphone at the environment, (5) a personal computing device of the authorized user.

In some aspects, the techniques described herein relate to a method, wherein determining whether the physical user input was received from the authorized user of the computing device further includes: transmitting, via the computing device, a discovery message to all computing devices in the environment in the vicinity of the computing device; receiving a confirmation message from another computing device, wherein confirmation message includes an identifier of the another computing device; determining whether the identifier of the another computing device is present in a database of authorized identifiers; and in response to determining that the identifier is not present in the database, determining that the physical user input was not received from the authorized user.

In some aspects, the techniques described herein relate to a method, wherein the physical user input includes accessing data that is restricted to the authorized user.

It should be noted that the methods described above may be implemented in a system comprising a hardware processor. Alternatively, the methods may be implemented using computer executable instructions of a non-transitory computer readable medium.

DETAILED DESCRIPTION

The present disclosure thus presents systems and methods for detecting usage anomalies. Techniques of the present disclosure use data sources that are not directly related to the software or hardware on which malicious activity was performed. Exemplary data sources include as CCTV, door badges, light sensors, voice assistants, etc., as additional trackers of an anomaly detection system (e.g., to determine if a user is indeed in front of their office computer or whether they in fact arrived to the office on that day). The techniques further train an artificial intelligence (AI) decision model with this data to improve the accuracy of the decisions made on software based indicators. More specifically, an implementation may involve gathering real-time data from related environment systems (e.g., Internet of Things (IoT) devices), correlating these log files with user activity (e.g., logins and unlocks), and adding them as additional data source to the AI decision engine.

FIG.1is a block diagram illustrating system100for detecting usage anomalies based on environmental sensor data. Consider an environment101that comprises computing device106. Environment101may be, for example, an office and computing device may be, for example, a work laptop provided by an employer. Security component110may be a data security software installed on computing device106for detecting usage anomalies (i.e., user inputs coming from an unknown entity). In some aspects, security component110may be installed on a different device (e.g., remote server) that is connected to computing device106and each sensor108in environment101. In some aspects, the application of security component110on the different device may be a thick client application and the application of security component110on computing device106may be a thin client application. The thin client application may transmit information to the thick client application about received user inputs and the thick client application may determine whether the user inputs should be executed.

Security component110may comprise communication engine112that retrieves data from sensors108aand108bin environment101and sensor108cthat is in the vicinity of environment101. Security component110may further comprise authorization credentials120, which is a database of identifiers of authorized users of computing device106. Security component110may further comprise verification module116, which determines whether an authorized user provided a user input received by computing device106. Security component110may further comprise a correlation engine114, which temporally aligns a received user input and environmental sensor data retrieved by communication engine112. Security component110may also comprise a notification module118that alerts authorized user(s) and/or an administrator of usage anomalies.

InFIG.1, user102may be physically present in environment101. In an exemplary aspect, computing device106may receive a physical user input (e.g., a login attempt via a keyboard or touchpad of computing device106) from user102. In some aspects, the physical user input is a request to access data that is restricted to an authorized user (e.g., only the authorized user may access the data). Security component110may intercept the user input in order to determine whether the physical user input was received from an authorized user of computing device106. Communication engine112may retrieve environmental sensor data from at least one sensor (e.g.,108a,108b) located in the environment101or at least one sensor located near environment101(e.g.,108c). For example, environment101may be an office of user104. Examples of sensors include, but are not limited to, a camera, a door badge scanner, a light sensor, a microphone at the environment, and a personal computing device of an authorized user. These sensors may be placed inside the office, or in the building where the office is located, or outside the building where the office is located. For example, sensor108amay be a smart speaker and sensor108bmay be a facial/ID scanner.

Security component110may further identify a window of time during which the physical user input was received. The length of the window of time may be predetermined (e.g., 10 minutes). For example, the physical user input may be received at 4:30 pm. The window of time may thus be between 4:25 pm and 4:35 pm.

Verification module116may then verify a presence of an authorized user at environment101during the window of time based on the environmental sensor data. For example, the environmental sensor data originating from sensor108amay be an audio clip comprising a voice of user102and the environmental sensor data originating from sensor108bmay be a captured facial image of user102when he/she entered environment101.

Authorization credentials120may be a database that comprises voice patterns and facial images of all authorized users of computing device106. In some aspects, verification module116may compare the retrieved environmental sensor data with the data in the authorization credentials120to determine if there is a match. More specifically, security component110determines whether the authorized user was present in environment101during the window of time. Suppose that the authorized user is user104, who used computing device106briefly before stepping out of his/her office. User102may be an intruder that has attempted to gain access to computing device106. In response to determining that authorized user104was not present in environment101during the window of time, security component110detects a usage anomaly.

As stated, sensors108a-cmay collect information such as audio clips, images, badge scans, biometrics (e.g., retinal or fingerprint scans), etc. Security component110parses this information to produce identifiers that can be compared with entries in authorization credentials120. For example, sensor108bmay capture video and security component110may analyze each frame to extract a group of pixels that represent a facial image. Verification module116may compare the group of pixels to facial images in authorization credentials120. Likewise, if sensor108bcollects audio clips, security component110may isolate portions of the audio clips that include human speech. Verification module116may compare the human speech with vocal information (e.g., temporal and frequency information in authorization credentials120) to determine whether an authorized user's speech shares characteristics of the extracted speech.

Preventing anomaly false positives is a difficult task because there are many ways the required information may not be collected adequately. For example, depending on the placement of each sensor and the angle at which a user approaches computing device106, the user's face may not be captured at an angle that is comparable to the facial images in authorization credentials120. Likewise, if the user is speaking at a low volume and his/her voice is drowned out by noise, the comparison of human speech may be ineffective. Because sensor placement is a factor in the quality of the environmental sensor data, security component110may utilize a weighting system that gives greater weight to sensor data from sensors that are closer to computing device106. Referring toFIG.1, the information captured by sensors108band108awill be given greater weight than sensor108c. In some aspects, security component110may track all sensors in the vicinity of a particular computing device and only consider environmental sensor data from those sensors. The list of sensors may be predetermined and may be manually updated whenever new sensors are introduced, old sensors are physically removed from an area, and/or sensor placement or computing device placement is changed.

Another device that security component110may consider is a device that is physically with a user. For example, users often carry a smartphone with them. Security component110may determine whether a physical user input on computing device106was received from an authorized user of the computing device by transmitting, via computing device106, a discovery message (e.g., via Simple Service Discovery Protocol or an equivalent) to all computing devices in environment101—specifically in the vicinity of computing device106(e.g., within a range at which the discovery message may be received by the device). The discovery message may query an identifier of the device (e.g., a MAC address, IP address, device name, etc.). In response to the discovery message, security component110may receive a confirmation message from another computing device (e.g., a smartphone of a user). The confirmation message may include the queried identifier of the another computing device. This identifier is also treated as part of the environmental sensor data.

Verification module116may determine whether the identifier of the another computing device is present in a database of authorized identifiers (e.g., authorization credentials120). For example, an authorized user may register an identifier of a device that the authorized user carries with him/her in authorization credentials120. If an unauthorized user's device is detected (e.g., verification module116determines that a MAC address is not in authorization credentials120), it is likely that the physical user input is not originating from an authorized user. Thus, in response to determining that the identifier is not present in the database, verification module116determines that the physical user input was not received from the authorized user.

This differs from techniques involving two-step authorization in which a code is sent via email or text that can potentially be viewed on an authorized device because here, the presence of the device itself is important. Email and text are mediums of communication that are vulnerable to interception and theft. Having the physical device in the presence of computing device106proves that the authorized user is present at the given location.

To prevent false positives in which an authorized user introduces a new device that is not identified in authorization credentials120(thus causing verification module116to assume that the user is not present in environment101), an authorized user may request to register new devices in authorization credentials120. In this case, the registration request must be accompanied with identifiers of the authorized user (e.g., biometrics) to prevent an unauthorized user from adding unauthorized device identifiers to authorization credentials120.

Security component110uses a combination of sensors to evaluate whether an authorized user is present in environment101and subsequently whether the authorized user is accessing computing device106. For example, when security component110sends a discovery message, depending on the amount of devices in the vicinity of computing device106, security component110may receive several confirmation messages in which some devices are registered in authorization credentials120and others are not. Unless there is only one person present at a time near computing device106, this makes it difficult to come to a verdict on whether the authorized user is present or not. That is why security component110also considers factors such as whether sensor108bcaptured a facial image of the authorized user, whether sensor108acaptured an audio clip of the authorized user, and whether sensor108creceived a badge scan from a physical identity card of the authorized user. Again, only three sensors are shown inFIG.1for simplicity. One skilled in the art will appreciate that there may be more or fewer sensors in environment101. For example, security component110may query whether a device registered as belong to an authorized user in authorization credentials120(e.g., a smartphone) connected to a wireless sensor such as a router in environment101.

If the authorized user104is not present in environment101, security component110does not execute the physical user input, and notification module118may notify the authorized user of the usage anomaly. For example, authorization credentials120may also include emergency contact information of authorized user104(e.g., an email address, phone number, etc.) where usage anomaly notifications may be sent. Notification module118may thus retrieve the information and alert authorized user104. In some aspects, the associated environmental sensor data may be included in the notification (e.g., a facial image of user102and/or an audio clip).

The window of time is a simple way to verify whether an authorized user is present without performing considerable amounts of processing. The window of time limits the amount of environmental sensor data that needs to be parsed. For example, if a physical user input was intercepted at computing device106at 1:00 pm, environmental sensor data from 7:00 am may not be relevant to verifying the presence of authorized user104. The most relevant time period may be immediately before and after the interception. Accordingly, security component110may use an iterative time window. For example, security component110may first set the time window to 10 minutes and analyze the data within 12:55 pm and 1:05 pm. If verification module116cannot identify authorized user104, security component110may increase the time window from 12:50 pm to 1:10 pm. If verification module116still cannot identify authorized user104in environment101after a threshold number of time window expansions (e.g.,3expansions), security component110may determine that authorized user104is not present in environment101. This approach is used to quickly dismiss user inputs received when a user was not physically present without expending processor power/memory. If the time window is too large (e.g., 12:30 pm to 1:30 pm) and does not need to be because authorized user104is readily identified at the 12:59 pm mark, the amount of resources taken to scan starting from 12:30 pm are wasted. This is prevented by starting with a small time window and increasing the window size whenever the authorized user cannot be identified (i.e., only as necessary).

In some aspects, the increase in window size may be predetermined. For example, security component110may set the first window size to 10 minutes, the second window size to 20 minutes, the third window size to 30 minutes, etc. In some aspects, the increase in window size is based on a difference between a confidence level (e.g., a quantitative value) of verification module116and a threshold confidence level. For example, when comparing parsed environmental data to entries in authorization credentials120, verification module116may generate a confidence level that is a function of the similarity values between the compared attributes (e.g., the amount of pixels or facial features that match between two facial images), how close two voices are in audio clips, etc. Suppose that a confidence level is given on a scale of 1 to 100. If authorized user104is not looking directly at a sensor that captures video during the first time window (e.g., in a 10 minute span), the image comparison value may not be at a minimum value that indicates a match. For example, if a match requires the comparison value to be at least 75 out of 100, and the comparison value is 70, verification module116may not indicate that the match exists. Likewise, if the audio clip captured by the sensor has background noise, a voice clip match may indicate a 60 out of 100 resemblance.

To generate a confidence level, verification module116may take the average of both values (e.g., (70+60)/2=65). Suppose that the threshold confidence level is 75. The difference between the confidence level and the threshold confidence level is thus10. Generally, the larger the difference, the more information is needed to verify the identity of authorized user104. The smaller the difference, the less information is needed. For example, in the latter, verification module116may simply need a few more frames in which authorized user is looking directly at the sensor capturing video. Accordingly, the time window expansion does not have to be large. Security component110may therefore increase the time window size based on this difference. Security component110may use, for example, a table that converts a difference into additional time (e.g., a difference of 10 needs an increase in window size by 5 minutes whereas a difference of 20 needs an increase in window size by 15 minutes).

If the authorized user104was present during the window of time, a second verification process is initiated. More specifically, in response to determining that authorized user104is present in environment101, correlation engine114correlates the user input with the environmental sensor data temporally to verify whether the authorized user had physical access to computing device106. The correlation process divides the environmental sensor data into a plurality of events. Examples of events include, but are not limited to, “user entered room,” “user exited room,” “second user entered room,” “system failure,” “location check-in,” etc. Correlation engine114then determines when each event took place. For example, sensor108cmay be a security camera that shows authorized user104entering environment101at 4:00 pm. Correlation engine114interprets this as an event in which authorized user104entered the room. Subsequently, correlation engine114may determine that the physical user input was received at 4:01 pm. Because authorized user104is the sole person in environment101and was present in environment101, correlation engine114may determine that authorized user104had physical access to computing device106. Accordingly, security component110may enable the execution of the physical user input. This logic may be governed by a plurality of predetermined rules. For example, a first rule may query whether a video of the user entering environment101was detected prior to the physical user input interception. Another rule may query whether a video of the user exiting environment101was detected prior to the physical user input interception.

In contrast, consider a scenario in which correlation engine114detects that user102entered environment101after authorized user104at 4:05 pm. Correlation engine114may further determine an event based on the environmental sensor data that indicates that authorized user104exited the environment101at 4:06 pm. Subsequently, correlation engine114may determine that the physical user input was received at 4:07 pm. In this case, authorized user104was present in environment101during the window of time (e.g., between 4:02 pm and 4:12 pm) and security component110will trigger the correlation process. However, correlation engine114would determine that based on the timeline of events, authorized user104did not have physical access to computing device106when the physical user input was received (i.e., because a combination of sensors indicate that the user104has left). In response to determining that authorized user104did not have physical access to computing device106, security component110does not execute (e.g., blocks) the physical user input and notification module118may notify authorized user104of the usage anomaly.

FIG.2illustrates a flow diagram of method200for detecting usage anomalies based on environmental sensor data. At202, security component110receives a physical user input at a computing device (e.g., device106) located in an environment (e.g., environment101). At204, security component110retrieves environmental sensor data from at least one sensor (e.g., sensor108b) located in the environment. At206, security component110identifies a window of time during which the physical user input was received. At208, security component110determines whether an authorized user was present in the environment during the window of time. In response to determining that the authorized user was present, method200advances to210, where security component110correlates the user input with the environmental sensor data by time. At212, security component110determines whether the authorized user had physical access to the computing device at the time the physical user input was received. In response to determining that the authorized user did have physical access, at214, security component110executes the physical user input.

However, if security component110determines that the authorized user did not have physical access (at212) or was not present in the environment (at208), method200advances to216, where security component110detects a usage anomaly. At218, security component110does not execute (e.g., blocks) the physical user input. At220, security component110notifies the authorized user of the usage anomaly.

FIG.3is a block diagram illustrating a computer system20on which aspects of systems and methods for detecting usage anomalies based on environmental sensor data may be implemented in accordance with an exemplary aspect. The computer system20can be in the form of multiple computing devices, or in the form of a single computing device, for example, a desktop computer, a notebook computer, a laptop computer, a mobile computing device, a smart phone, a tablet computer, a server, a mainframe, an embedded device, and other forms of computing devices.