System, method, and computer program for predictive autoscaling for faster searches of event logs in a cybersecurity system

The present disclosure describes a system, method, and computer program for predictive autoscaling for faster searches of event logs in a cybersecurity system. In one embodiment, the system receives search-related signals from a plurality of signal sources. The signals are indicative of: (1) a user's intent to perform a search for event logs in a cybersecurity database, (2) how computationally intensive the potential search is likely to be, and (3) the currently available computational resources. The signals are evaluated, and an autoscale prediction score is calculated. The autoscale prediction score reflects the likelihood of a user to submit search, the computational resources required for the potential search, and the currently available computational resources. The system scales computational resources in accordance with the autoscale prediction score. These steps are performed before any search is submitted by the user in a search user interface.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates generally to cybersecurity systems, and, more specifically, to predictive autoscaling for query searches on a database of event logs in a cybersecurity system.

2. Description of the Background Art

Cybersecurity systems monitor entity behavior in a network in order to detect cybersecurity threats. As entities interact with the network, various systems generate raw logs related to the entity behavior. For example, a cybersecurity system may obtain raw data logs related to a user's interactions with the IT infrastructure, such as user logon events, server access events, application access events, and data access events. Cybersecurity system will typically take these raw data logs and generate event logs from the raw data logs.

Cybersecurity analysists within an enterprise perform searches on the event logs to provide a better understanding of cybersecurity risks. Event log databases can be large, and search queries on the event log databases are often compute intensive processes. A query life cycle typically has the following stages:

Many cybersecurity systems store event logs in cloud-based databases, and scale compute resources for searches on demand. The scaling is performed after the query is submitted and parsed. Dynamically scaling compute resources on demand is more cost efficient than maintaining a high amount of compute resources at all times. Unfortunately, the time required to autoscale compute resources creates a delay in query execution, which results in slow queries. Therefore, there is demand for a solution that can autoscale without slowing down searches.

SUMMARY OF THE DISCLOSURE

The present disclosure describes a system, method, and computer program for predictive autoscaling for faster searches of event logs in a cybersecurity system. The system receives one or more signals indicative of a user's intent to perform a search for event logs in a cybersecurity database. Examples of the signals include a user clicking on a query builder window within a search interface for the event logs database or selecting a previous search. The system calculates an autoscale prediction score for a potential search based on the one or more signals. The system then makes an autoscale decision based on the autoscale prediction score. These steps are all performed prior to a user submitting a search query. In other words, the system makes an autoscale decision based on the likelihood that a user will submit a search. This way, if and when the user does submit a search, the computational resources for the search are available and there is no delay associated with provisioning additional compute resources.

In a preferred embodiment, the system receives signals from a plurality of different sources that collectively indicate not only the user's intent to perform a search, but also indicate how computationally intensive the search is likely to be and the currently available computational resources. For example, a signal indicating that a user has selected a query time domain within the search user interface provides an indication of the size of the search domain and hence the amount of computational resources required.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present disclosure describes a system, method, and computer program for predictive autoscaling for faster searches of event logs in a cybersecurity system. The methods disclosed herein are performed by a computer system (“the system”), such as cybersecurity system that detects cybersecurity threats in a network.

FIG. 1 illustrates a method for predictive autoscaling for event log searches in accordance with one embodiment. The system receives one or more signals indicative of a user's intent to perform a search for event logs in a cybersecurity database (step 110). Examples of the signals include a user clicking on a query builder window within a search interface for the event logs database or selecting a previous search. The system calculates an autoscale prediction score for a potential search based on the one or more signals (step 120). The system then makes an autoscale decision based on the autoscale prediction score (step 130). These steps are all performed prior to a user submitting a search query. In other words, the system makes an autoscale decision based on the likelihood that a user will submit an event log search. This way, if and when the user does submit a search, the computational resources for the search are available, and there is no delay associated with provisioning additional compute resources.

FIG. 2 illustrates an example implementation of the method of FIG. 1. The system receives signals from a plurality of sources that are collectively indicative of: (1) user intent to perform a search, (2) likely search domain size, and (3) currently available computational resources (step 210). The signals related to the user intent and to the search domain size include signals about a user's actions on a search user interface before the user submits a search. Examples of the signals are set forth below in Table 1.

The system evaluates the signals and assigns a value to each signal (steps 220-230). The table below lists examples of the signals received and the values which may be assigned to each signal:

What the signal is
Example Range of

Signal
indicative of
Values

1
User is using a
User intent to search
0 (no usage),

query builder on a

search user interface

(e.g., user clicks on

a query bar).

2
User selects a query
User intent to search;
0 (no selection of a

time window (e.g.,
Size of search
window),

time range selection)
domain (i.e.,
0.1 (selection of a

resources required 
0.2 (selection of a

and 0.3 (selection of

large are defined by

time window

3
User selection of a
User intent
0 (no selection),

recent query

4
Pattern of usage
User intent
0 (current time is

hours for the given

outside the user's

peak search

1 (current time is

within the user's

peak search

5
Pattern of usage
User intent
0 (current time is

hours for the given

outside the entity's

customer entity

peak usage window

for searches),

1 (current time is

within the entity's

peak usage window

for searches)

6
Adding indexed
User intent to search
0 (an index field is

fields to the query
and computational
not yet included

builder
resources required
within the search

indexed field has

been added to the

search criteria)

7
Percentage of
Available compute
0.3 (no compute

compute resources
resources
engines currently

available

amount of compute

amount of compute

0 (large amount of

compute engines

medium, and large

values may be

predefined in

configuring the

8
Size of customer
Size of search 
0.1 (small),

event log database
domain
0.2 (medium),

medium, and large

values are defined

by data size ranges.

The system calculates an autoscale prediction score as a weighted sum of the values assigned to the signals (step 240). The score reflects the probability of a user submitting a search, the size of the search domain, and currently available computational resources. The autoscale prediction score may be expressed mathematically as follows:

In one embodiment, the signal values and the weights are assigned such that the autoscale prediction score is in the range between 0 and 1. The example signal values in Table 1 are designed for a prediction score between 0 and 1.

The system adjusts computational resources available for searches as a function of the autoscale prediction score (step 250). The autoscale prediction score is used to decide whether to autoscale and the capacity desired. This may be expressed mathematically as followed:

Steps 210-250 are performed before a user submits a search.

FIG. 3 illustrates an example architecture of the system. A Prediction Service module 320 receives signals from a plurality of signal sources 310. The Prediction Service Module 320 calculates an autoscale prediction score in accordance with the methods described herein. The Autoscaler 330 provisions compute engines 340 for searching as a function of the autoscale prediction score.

The methods described with respect to FIGS. 1-3 are embodied in software and performed by a computer system (comprising one or more computing devices) executing the software. A person skilled in the art would understand that a computer system has one or more memory units, disks, or other physical, computer-readable storage media for storing software instructions, as well as one or more processors for executing the software instructions.

As will be understood by those familiar with the art, the invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the above disclosure is intended to be illustrative, but not limiting, of the scope of the invention.