Dynamically determining a trust level of an end-to-end link

A system and method for dynamically determining a trust level of an end-to-end link of a computer database, including: in a preparation stage: capturing a first set of messages of an end-to-end link; compressing a skeleton of each message of the first set of messages to generate a construct of each message of the first set of messages, the skeleton includes the message without a value field; creating a characteristic histogram of the constructs of the first set of messages; and during an operation stage: capturing a second set of messages of the end-to-end link; compressing a skeleton of each message of the second set of messages to generate a construct of each of the second set of messages; creating a work histogram of the constructs of the second set of messages; and determining a trust level of the end-to-end link by comparing the work histogram with the characteristic histogram.

FIELD OF THE INVENTION

The present invention relates generally to dynamic whiten sting, and specifically, to dynamically determining a trust level of an end-to-end link from a user to a computer database.

BACKGROUND

A data firewall typically captures or sniffs data accesses to a database (e.g., requests and responses) in real-time and analyzes the data according to policy rules. The data firewall may include a data activity monitor (DAM) and/or file activity monitor (FAM). The requests and responses sniffed by the data firewall may include a request, e.g., a structured query language (SQL) statement, or a response, and associated header information. The header may include metadata such as machine information, network information, user information, client information, etc.

The canonical solution for DAM is to install an agent on the data-source server (e.g., the database server). The agent may capture, mirror or sniff requests and responses and send requests and responses to a security server. The security server may parse the data, perform policy-enforcement, and then audit, analyze, alert or block the requests and responses as required. One of the main challenges of DAMs, is the huge amount of data that have to be captured and analyzed. For example, in a typical enterprise environment, a DAM may capture or sniff about 100 million transactions (e.g., requests and responses) per second (TPS). Sniffing and analyzing such huge amounts of data in real-time or near real-time requires appropriate computer infrastructure that may be very expensive.

Therefore, a method for reducing the amount of analyzed data is required.

SUMMARY

According to embodiments of the invention, a system and method for dynamically determining a trust level of an end-to-end link of a computer database may include: in a preparation stage: capturing a first set of messages of a first end-to-end link; compressing a skeleton of each message of the first set of messages to generate a construct of each message of the first set of messages, wherein the skeleton comprises the message without a value field; and creating a characteristic histogram of the constructs of the first set of messages; during an operation stage: capturing a second set of messages of the first end-to-end link; compressing a skeleton of each message of the second set of messages to generate a construct of each of the second set of messages; creating a work histogram of the constructs of the second set of messages; and determining a trust level of the first end-to-end link by comparing the work histogram with the characteristic histogram.

According to embodiments of the invention, an end-to-end link may be defined according to parameters of the end-to-end link, wherein the parameters are selected from: host name, service name, database name, client host name, operating system user and database user.

According to embodiments of the invention, determining the trust level of the end-to-end link may include determining the trust level of the end-to-end link to not trusted if at least one of the constructs of the second set of messages is not included in the characteristic histogram of the constructs of the first set of messages.

According to embodiments of the invention, compressing a skeleton may include hashing the skeleton to produce a hash.

Embodiments of the invention may include, in the preparation stage: adding the commands of the first set of messages divided into command groups to the characteristic histogram; during the operation stage: adding the commands of the second set of messages divided into the command groups to the work histogram.

Embodiments of the invention may include, in the preparation stage: obtaining a plurality of sets of messages, wherein each set pertains to an end-to-end link of a set of end-to-end links; generating a construct of each message of the plurality of sets of messages; creating a characteristic histogram for each end-to-end link of the set of end-to-end links, wherein each characteristic histogram is created from the constructs of a set of messages of an end-to-end link of the set of end-to-end links; and performing clustering of the characteristic histograms to determine clusters of end-to-end links; during operation: comparing the work histogram with each of the characteristic histograms of a cluster of the first end-to-end link to determine the trust level of the first end-to-end link.

According to embodiments of the invention, capturing a second set of messages may be repeated for every new session and periodically.

According to embodiments of the invention, the trust level may be selected from trusted and not trusted, and embodiments of the invention may include: not performing comprehensive security analysis for future messages of the end-to-end link if the trust level is trusted; and applying policy rules to future messages of the end-to-end link if the trust level is not trusted.

Embodiments of the invention may include, adding the construct of each of the second set of messages to the characteristic histogram if the trust level is trusted.

According to embodiments of the invention, the work histogram may be compared with the characteristic histogram by comparing the variance of the work histogram with the variance of the characteristic histogram.

DETAILED DESCRIPTION

Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. Example cloud model may include for example five characteristics, at least three service models, and at least four deployment models.

Characteristics may be for example:

Service Models may be for example:

Database-as-a-Service (DBaaS): the capability provided to the consumer is to store data on a cloud infrastructure. DBaaS paradigm is a common approach for storing data in a cloud based computerized service, where users get access to data without the need for managing hardware or software.

Deployment Models may be for example:

Embodiments of the invention may provide a system and method for dynamic whitelisting or for dynamically determining a trust level of a connection of a computer database. According to embodiments of the invention, messages (e.g., the body or payload of database requests and responses) associated with a trusted end-to-end links may not undergo comprehensive security analysis.

Current architecture of DAM and FAM products is based on a two steps process. The first step is performed in real-time by an agent software or application which is installed on the data source machine and the second step is performed offline by a security server application, also referred to as a collector. The data source machine may include a database, e.g., a database server, a file server, etc., or a combination of a database and a proxy or network gate of the database. The agent software may be installed on the database itself or on the proxy or network gate of the database. Installing the agent software on a proxy or network gate of a database may enable capturing data packets also referred to as data records in cloud-based databases (when DBaaS model is used) that are many times operated by a third party that does not allow installation of software applications on its databases.

According to prior art applications, the agent application may capture or receive all the data packets including requests and responses of the monitored database, read the header information, perform an initial rile processing on the header information, and then send the data packet to the collector application for an offline comprehensive security analysis. The comprehensive security analysis performed by the collector may include getting or receiving the data packet from the agent, parsing the data packet, structuring the data, e.g., classifying the data and mapping the metadata to the data, building the data hierarchy, applying the policy rules on the classified data to identify sensitive data and detecting a data breach or data tampering, according to the rules matching. Thereafter, the collector may send the data for further analysis and auditing by other components.

Many of the processes of the comprehensive security analysis performed by the collector rely on the mapping and classification of the data, which is typically running offline, due to the complexity and performance requirements of the mapping and classification of huge amounts of data (e.g., +100,000 TPS per data source or a total of 100 million TPS). The agent must not delay the transaction and must avoid latency. Sniffing and analyzing such huge amounts of data in real-time or near real-time requires appropriate computer infrastructure that may be very expensive.

One approach for decreasing the amount of analyzed data is whitelisting. Whitelisting may include determining a trust level or other rating for an end-to-end link not performing a comprehensive security analysis for requests and responses of trusted end-to-end links, and performing a comprehensive security analysis for requests and responses of not-trusted end-to-end links.

Existing solutions for whitelisting may include using techniques like user behavior analytics application (UBA), risk-assessment, signature-searching and trusted-connection to tag or label the end-to-end link as trusted or not. However, all these techniques are static, e.g., once a connection is classified as trusted, the entire volume of communication in this connection is considered as trusted. Thus, those techniques are exposed to attacks such as hijacking, in which an attacker takes over a portion of a connection and acts as one of the participants.

Embodiments of the invention may improve the technology of security analysis by providing a method for whitelisting, or determining a trust level or rating of an end-to-end link, that is both dynamic (e.g., performed periodically in run time) and at the same time requires low computational resources (comparing to full analysis of the communication). Embodiments of the method for determining a trust level of an end-to-end link may be performed periodically, in real-time or near real-time, and thus may enable detecting abnormal behavior of communications over end-to-end links that were previously determined as trusted. Therefore, assuming that a hijacker may behave differently than a normal user of an end-to-end link, hijacking attacks, in which an attacker takes over a portion of an end-to-end link that was previously categorized as trusted, may be detected.

A database connection may be a software structure that may enable a client application or software to communicate with a data source machine, e.g., send commands and receive replies. In many applications, connections are pooled to facilitate connection reuse. Thus, a plurality of users may use a plurality of connections in collaboration. According to embodiments of the invention, an end-to-end link may represent communication between a single client and a single database. The link may be carried over multiple connections in one or more connection pools.

According to embodiments of the invention, an end-to-end link may be defined by a tuple or a combination of parameters related to a request sent to the database or a response from the database, that specifies the end-to-end link. The tuple may include parameters relating request or response to a single user. Those parameters may be taken from the header of the request or response. Ideally, and end-to end link may be unique to a single user, or at least as close as possible to a single user based on the available parameters. the tuple of parameters may include user information, source program information and database information. The user information may include parameters identifying the user itself. For example, the user information parameters may include one or more of the database user identification (DB User), operating system user identification (OS User), application user identification (App User) and the client host address (ClientHost). The source program information may include parameters identifying the application or software used to connect the user to the database, e.g., SQLplus, Java database connectivity (JDBC) etc. The database information may include parameters identifying the database. For example, database information may include one of a name of the database (DbName) or service name (ServiceName), and one of the database server host name (ServerHostName) or the database server IP. Other user information parameters, source program information parameters, database information parameters or other parameters may be used.

Using a tuple and not just the database user identification (DB User) to specify an end-to-end link may enable defining a single end-to-end link that is used by a single user, e.g., a single user may be detected in situations where multiple users share the same connection or connection pool. A situation in which multiple users use or share the same connection or connection pool is very common in many applications, for example, multiple bank tellers may access the same database using a single connection.

According to embodiments of the invention, a signature of an end-to-end link may be generated. The signature may be generated based on messages pertaining to the end-to-end link that may represent the normal behavior of the end-to-end link. In some embodiments, the signature may include or may be generated from a compressed or reduced size version of the messages without the value field and/or from commands used in the end-to end link. According to some embodiments of the invention, the end-to-end links may be divided to groups or clusters of similar end-to-end links, for example, by a classifier or by performing clustering analysis on the signatures. Each cluster may include a group of end-to-end links with small distance between signatures of the end-to-end links in the cluster. Once a signature is created, the communication pertaining to the end-to-end link may be checked periodically by comparing the communication over the end-to-end link to the signature, or to the cluster of signatures. A trust level of the end-to-end link may be determined based on the comparison.

In some embodiments, the signature may be generated by creating a construct of each message (without the value filed) of a set of messages of an end-to-end link, and creating a characteristic histogram from the constructs, where the signature may be or may include the characteristic histogram. For example, the construct may include a hash or other compressed version of the message without the value field of the message. Removing the value field of the message before compressing the message may increase the uniformity of the compressed values. The operation used for generating the construct in some embodiments may reduce the number of bits, be very fast to compute and minimize collisions (a collision may occur when the same construct is generated for different messages). To generate the characteristic histogram from the constructs, the constructs may be divided into bins. It is assumed that during normal operation of an end-to-end link, the vast majority of the messages (without the value field) are similar, therefore hashing the messages without the value field may generate a typical characteristic histogram.

In some embodiments, the characteristic histogram may also include commands of the set of messages of the end-to-end link. For example, the characteristic histogram may be generated by grouping the commands to bins of commands groups. For example, the command groups may include data definition language commands (DDL), data manipulation language commands (DML), data query language commands (DQI), data control language commands (DCL), transaction control language commands (TCL), etc. It is assumed that during normal operation of an end-to-end link, the vast majority of the commands will repeat themselves, therefore counting the number of commands in each command group may generate a typical commands histogram the may be included in the characteristic histogram. Thus, according to some embodiments, the signature of an end-to-end link may include a characteristic histogram including construct and/or command bins.

In many database applications, the same operations are repeating, by the same user and by other users of the same role. For example, all the users who are retail bankers may repeat the same operations many times during the day. For example, they may perform business transactions like money movement, loan approvals, checking accounts to many clients, etc., where each business transaction may be performed by hundreds of SQL statements. Thus, according to embodiments of the invention, removing the values from the messages may detect the repeating and common activities. Therefore, as long as the same user is using the end-to-end link for the same purposes, the general shape of the characteristic histogram may remain the same. In contrast, large deviations in the general shape of the characteristic histogram may serve as an indication of a possible security problem.

Hardware and software layer60includes hardware and software components, such as the hardware and software components depicted inFIG. 10. Examples of hardware components include: processors (e.g., processor705depicted inFIG. 10) such as mainframes61, RISC (Reduced Instruction Set Computer) architecture based servers62, servers63and blade servers64; storage devices65(e.g., storage device730depicted inFIG. 10); and networks and networking components66. In some embodiments, software components include network application server software67and database software68.

According to some embodiments, a monitored database may be implemented on virtual storage72and physically located on storage devices65. The database may be managed by database software68that may include an agent software according to embodiments of the invention. A collector or a data security application may be implemented by software running on a virtual server71. However, other architecture and hardware may be used.

Reference is made toFIG. 3, depicting a system300, according to embodiments of the invention. According to some embodiments, security server330may be implemented on a virtual server71and data source machine320may be implemented on virtual storage72, however, other implementations may apply. It should be understood in advance that the components and functions shown inFIG. 3are intended to be illustrative only and embodiments of the invention are not limited thereto.

Networks340may include any type of network or combination of networks available for supporting communication between database client310data source machine320and security server330. Networks340may include for example, a wired, wireless, fiber optic, or any other type of connection, a local area network (LAN), a wide area network (WAN), the Internet and intranet networks, etc. Additionally or alternatively, any of database client310data source machine320and security server330may be connected to each other directly.

According to some embodiments, a database client application312running on database client310may communicate with data source machine320, for example, by generating and submitting data packets or data records350including for example database queries to data source machine320. Data packets may pertain to end-to-end link360.

In one example, a query transmitted from database client310to data source machine320, may be or may be included in one or more database protocol packets, also referred to as data packets350that includes a header352and a payload354. Header352may include metadata such as machine information, network information, user information, client information, etc. For example, the header may include the following parameters used to define end-to-end link360, ServerHostName, ServiceName, DbName, ClientHost, OsUser, DbUser, etc. Payload354may include data and/or statements for a database query, e.g., SQL requests or responses. While a single database client310and a single data source machine320are shown inFIG. 3, one or more data source machines320may provide database services to one or more database clients310or client applications312.

In one example, payload354of data packet350may include a query (e.g., a request or an SQL statement) or a response to a query, referred to uniformly herein as a message510(depicted inFIG. 5). A query may include a structured query language (SQL) statement, for accessing data in tables managed by the database management application (e.g., database management application324, and/or database software68). SQL represents a standardized language for defining and manipulating data in a relational database. For example, under a relational database model, the database may be perceived as a set of tables that include data, and data may be retrieved by using SQL statements to specify a result table that can be derived from one or more tables. The query may be defined in one or more additional or alternate languages or protocols for defining and manipulating data in a relational database or in other types of databases.

Data source machine320may include a database326, e.g., a database server, a file server, etc., or a combination of a database326and a proxy or network gate of the database326, a database management application324, and an agent application322. An example of a data source machine that includes a database server410is depicted inFIG. 4A. Database server410may include database326managed by database management application324, that may include or may communicate with an agent application322. Similar architecture may apply to a file server.FIG. 4Bdepicts a data source machine that includes a combination of a gateway, a network gate or a proxy422and a database server (or file server)420. Database server420may include a database326managed by database management application324. In this embodiment, an agent application322is located at gateway, network gate or proxy422.

In one example, in response to a query from database client310, database management application324may handle the query and generate a response for the query that is returned to database client310in a database server response.

Data packet350may be streamed between database client310and data source machine320on end-to-end link360. Agent322may be a computer-executed process that may capture, sniff or intercept one or more data packets350along the communication stream between database client310and data source machine320, without interfering with the communication of data packets350to data source machine320. Agent322may be implemented at one or more points along the communication stream between database client application312and data source machine320to monitor for and capture or intercept data packets350without requiring the participation of the database management application432(or database software68), and without relying on any form of native auditing or native logs of the database management application324. While drawn as part of data source machine320, agent322may intercept data packets350at other locations such as, but not limited to, the database memory of database server410and420, within network340, at the operating system level, or at the level of database libraries.

End-to-end link360may be defined by a tuple or a combination of parameters related to data packets350(typically found in header352). The tuple may include one or more of user information parameters such as OsUser, DbUser, App User, ClientHost (e.g., related to database client310), etc., source program information, and database information such as ServiceName or DbName and ServerHostName or database server IP (e.g., related to database326). Using a tuple and not just the username (DB User) to specify end-to-end link360may be needed to identify end-to-end links360within connections or connection pools, e.g., a situation in which multiple users use or share a single connection or a connection pool.

According to embodiments of the invention, agent322may capture or intercept a plurality of data packets350flowing between data source machine320and data client310. Agent322may capture both data packets350including query packets transmitted from data source machine320to data client310and data packets350including response packets transmitted from data client310to data source machine320. Data packets350may be captured in real-time substantially without adding latency or delay.

According to some embodiments, agent322may decrypt captured data packets350to obtain header352of each data packets350. Agent322may analyze headers352to obtain the header information and to associate the packet with an end-to-end link360based on the header information. According to some embodiments of the invention, end-to-end link360may be associated with a trust level, a security status or a security grade or rating. According to embodiments of the invention, agent322may associate each data packet350with the security status or trust level of end-to-end link360to which it pertains. The trust level may be a categorial parameter, that may for example be either “trusted” or “not trusted”. Other trust levels or ratings may be used.

In some embodiments, agent322may apply policy rules to future messages of a not trusted end-to-end link360. According to embodiments of the invention, agent322may determine whether a data packet350needs to be sent to security server330based on or according to the trust level. A trusted end-to-end link360may be placed in a whitelist, and messages510associated with a trusted end-to-end link360may be ignored, e.g., excluded from further security analysis, and specifically from the comprehensive security analysis performed by security server330. Thus, agent322may not send messages510associated with a trusted end-to-end link360to security server330. On the other hand, if end-to-end link360is categorized as not trusted, messages510associated with end-to-end link360(that is not trusted) may be sent by agent322to security server330to undergo a comprehensive security analysis. In some embodiments, agent322may issue an alert if end-to-end link360is categorized as not trusted. Other security measures may be taken in case end-to-end link360is categorized as not trusted according to embodiments of the invention. It should be readily understood that a single end-to-end link360may be categorized as trusted at sometimes, and as not trusted at other times, as disclosed herein.

Thus, when capturing a data packet350, agent322may decrypt the header352to obtain header information, also referred to as metadata, including, for example, machine information, network information, user information, client information, etc. agent322may determine the end-to-end link360associated with the data packet350based on the header information. Once the end-to-end link360is known, agent322may verify the security status associated with the end-to-end link360and act accordingly. Agent322may determine based on the security status whether message510associated with header352should be blocked, should be allowed to flow without further analysis or whether further security analysis is required. For example, a header security rule may define that messages510from a certain end-to-end link360should be blocked if a user is known as a malicious entity, should be allowed to flow without further analysis (e.g., ignored) if end-to-end link360is trusted, or should be further analyzed if the status of end-to-end link360is not trusted. Other header security rules or a combination of header security rules may be used. Since the header structure is known, decrypting and analyzing header352may be performed in real-time without introducing significant delay.

According to embodiments of the invention, agent322may generate a signature362to end-to-end link360. Agent322may generate signature362based on messages510sent over end-to-end link360and captured by agent322, e.g., in a preparation stage and during operation stage as disclosed herein. Thus, signature362may represent the normal behavior of end-to-end link360. To generate signature362, agent322may capture messages510over a time window of a preparation stage (also referred to as a training stage) or until a sufficient number of messages510are captured. For example, in a typical application, about 150,000-400,000 messages510may be required to generate signature362. In some embodiments, the preparation stage may include a plurality of sessions (a session may refer to a logical representation of a connection to a database from login to logout). In some embodiments, agent322may capture messages510in a first one or two minute of 20-40 sessions to generate signature362. Other time windows for capturing messages for generating signature362may be used. According to embodiments of the invention, agent322may generate signature362to end-to-end links360with high volume or high rate of messages510. For example, agent322may not generate signatures to sessions with messages rate below a threshold, e.g., below 100 messages per second.

In some embodiments, all the messages510used for generating signature362may undergo comprehensive security analysis as disclosed herein, or otherwise inspected to verify that they represent normal behavior of end-to-end link360.

Signature362may include a compressed or reduced size version of messages510without the value field captured during the preparation stage. Once signature362is created, agent322may periodically check the communication over end-to-end link360by comparing messages510of end-to-end link360to signature362. Agent322may calculate or determine a trust level of end-to-end link360based on the comparison. According to some embodiments, the preparation stage and the generation of signature362may be repeated periodically, e.g., once a week or once a year.

According to some embodiments, signature362may include a characteristic histogram related to messages510sent over end-to-end link360during the preparation stage. The histogram generated during the preparation stage may be referred to herein as the characteristic histogram of end-to-end link360. In this case, a comparison may be performed by sampling messages510over a time window during an operation stage, generating a work histogram and comparing the work histogram to the characteristic histogram as disclosed herein.

Reference is now made toFIG. 5, depicting messages510, skeletons520, constructs530and a histogram540of end-to-end link360, according to embodiments of the invention. Histogram540may represent either the characteristic histogram or the work histogram. For example, for generating characteristic histograms (e.g., signature362), agent322may capture messages510during a training or preparation stage and for generating work histograms agent322may capture messages510during run time or operation stage.

In some embodiments, agent322may generate histogram540by generating a skeleton520from each message510used for generating histogram540, a construct530from each skeleton520and a histogram540from constructs530. Histogram540may be (or may be a part of) signature362of end-to-end link360for messages captured during a preparation stage, and the work histogram for messages captured during run time or operation stage.

Agent322may generate skeleton520from message510by removing the values from message510. The values be or may refer to specific data elements that are stored in the database tables. Thus, skeleton520may include message510, without the value field. Examples for SQL statements are provided below:

Example 1: Select NAME from EMPLOYEE whereDEPARTMENT=“SALE” and BAND>8

Example 2: SELECT FULL_NAME, ADDRESS from CUSTOMER,PHONE where PHONE.PHONE_NO like “01230123012%”andCUSTOMER.CUSTOMER_ID=PHONE.CUSTOMER_ID

The values in example 1 are ‘SALE’ and ‘8’, and in Example 2 the value is the phone number ‘01230123012’. These examples are for the requests. Responses typically include column heading and the values. Thus, to generate the skeleton520from the responses values may be removed and the skeleton520may include the column headings.

Agent322may generate construct530from skeleton520by hashing skeleton520or using other compression method. The operation used for generating construct510may reduce the number of bits of message510, be very fast to compute and minimize collisions, e.g., in some cases should not or should rarely generate the same construct350for different skeleton520. In typical applications, a length of a skeleton520may range from tens of characters up to thousands of characters for complex statements. A hash of the skeleton520may be up to about 15-25 characters.

For generating characteristic histogram540, agent322may divide the values of constructs530to a series of intervals, typically, the intervals are consecutive, adjacent and non-overlapping, each interval may be referred to as a bin or a bucket. Agent322may then count how many constructs530fall within each bin. The bins may be of equal size or not. It is assumed that messages510of an end-to-end link360may repeat themselves or may be similar to each other, especially when the values are removed. Thus, generating a characteristic histogram540based on the skeleton520of the messages510may represent the common behavior of end-to-end link360. Therefore, it may be expected that the distribution of messages510of an end-to-end link360would remain similar to the characteristic histogram.

According to some embodiments, histogram540may include the commands used in messages510additionally or alternatively to constructs530. For example, agent322may divide commands550included in messages510into command groups. For example, the command groups may include DDL, DML, DQI, DCL, TCL, etc. Thus, each command group may be a bin in histogram540and agent322may place each command in the appropriate bin based on the type of the command.

According to embodiments of the invention, during an operation stage, and as long as end-to-end link360is labeled or categorized as trusted, agent322may not send messages350associated with end-to-end link360to security server330for a comprehensive security analysis. This may dramatically reduce the computational power required at security server330. However, in order to detect hijacking attacks, agent322may sample messages510associated with end-to-end link360from time to time, e.g., periodically, for example, every 10-30 minutes and/or for every new session, and check for variations from the signature, e.g., the characteristic histogram. According to embodiments of the invention, agent322may sample messages510of end-to-end links360and check for variations from the signature362only in sessions with high volume or high rate of messages510. For example, agent322may not check against a signature messages of sessions with messages rate below a threshold, e.g., below 100 messages per second. Instead, messages510of those sessions may undergo a comprehensive security analysis since they do not contain a large amount of data and therefore do not introduce high loads to the system.

For example, agent322may sample messages510associated with end-to-end link360during a sampling time window. The sampling time window may be of fixed length (e.g., 1-2 minutes) or may continue until a sufficient number of messages510are sampled (e.g., 100-400 messages). According to embodiments of the invention, agent322may generate a work histogram540from the sampled messages350as disclosed herein.

Reference is now made toFIG. 6A, which depicts comparison of a work histogram610of an end-to-end link360with a characteristic histogram620of end-to-end link360, according to embodiments of the invention. InFIG. 6A, characteristic histogram620is generated in a preparation stage and work histogram610is generated during an operation stage, as disclosed herein. According to embodiments of the invention, agent322may compare602work histogram610with characteristic histogram620to determine a trust level604of end-to-end link360. In some embodiments, agent322may determine trust level604directly from the comparison602, and in some embodiments, agent322may calculate a security grade based on the comparison602and may determine trust level604by comparing the security grade to a threshold. Comparison602may be performed using any applicable statistical or other method. For example, agent322may compare602the variance of the work histogram610with the variance of the characteristic histogram620. For example, agent322may determine that end-to-end link360may remain trusted if for example the following is true and that end-to-end link360may be classified as not trusted if the following is not true (other rules may be used):
work_histogram_variance<2*characteristic_histogram_variance  (Equation 1)

Reference is now made toFIG. 6B, which depicts comparison606of a work histogram610of end-to-end link360with a plurality of characteristic histograms620,622,624of a plurality of end-to-end links360,362,364each pertaining to a cluster630of end-to-end links360,362,364, according to embodiments of the invention. While cluster630is depicted as including three end-to-end links360,362,364, this is an example only and a cluster630may include any number of end-to-end links.

Agent322may find clusters630of end-to-end links360,362,364, where the features used for the clustering may include, for example, constructs of end-to-end links360,362,364and/or commands used in end-to-end links360,362,364. for example, the commands used in end-to-end links360,362,364may be grouped to command groups such as DDL, DML, DQI, DCL, TCL, etc. A clustering algorithm may be used to find clusters each including similar end-to-end links, for example those having common constructs and commands. In some embodiments, k-means clustering may be used to partition end-to-end links360into k clusters630in which each end-to-end link360belongs to the cluster630with the nearest mean. Other clustering methods may be used.

In some embodiments, agent322may compare606(e.g., using Equation 1) work histogram610with each of characteristic histograms620,622and624of cluster630of end-to-end link360to determine the trust level604of end-to-end link360. For example, Equation 1 in some embodiments should be true for all characteristic histograms620,622and624for end-to-end link360to be considered trusted, and it may be enough that Equation 1 will not be true for one of characteristic histograms620,622and624for end-to-end link360to be considered not trusted. Other rules and terms may be used to determine a trust level of end-to-end link360.

Reference is now made toFIG. 6C, which depicts comparison608of a work histogram610of end-to-end link360with a common characteristic histogram640of a cluster630of end-to-end links360,362,364, according to embodiments of the invention. In some embodiments, a common characteristic histogram640may be generated, common to all end-to-end links of cluster630and the work histogram610of end-to-end link360may be compared608(e.g., similarly to comparison602) against the common characteristic histogram540to determine a trust level of end-to-end link360.

According to embodiments of the invention, calculating a trust level of end-to-end link360should be fast to enable real-time operation. According to embodiments of the invention, this requirement is achieved by embodiments of the invention since calculating hash values and comparing histograms require low computational resources. Dynamic whitelisting according to embodiments of the invention may dynamically reduce the computational power required for performing a comprehensive security analysis.

Returning toFIG. 3, according to embodiments of the invention, agent322may send data packets350of end-to-end links360to security server330if a trust level of end-to-end link360changes from trusted to not trusted. Additionally or alternatively, agent322may issue a security alert if a trust level of end-to-end link360changes from trusted to not trusted. Additionally or alternatively, agent322may block data packets350of end-to-end links360if a trust level of end-to-end link360changes from trusted to not trusted.

Security server330may implement or execute a data security application332. Data security application332may be or may include a firewall, a DAM and/or a FAM, an external database non-intrusive security mechanism (EDSM), enterprise database auditing, and real-time protection. Data security application332may provide a database activity monitoring service of data source machine320, including performing the comprehensive security analysis. Data security application332may provide continuous monitoring of database activity of data source machine320. Examples of data security application332may include, but are not limited to, the Guardium® application available from International Business Machines Corporation.

According to embodiments of the invention, data security application332may receive or obtain data packet350. The comprehensive security analysis may include parsing the data packet, mapping metadata to data, building hierarchy of the data (e.g., building a hierarchical-tree of name-value), and processing policy rules. Processing policy rules may require associated names and values. For example, a rule may include “if name like % ID % and value match specific regular expression (Regex)” or a rule may expect a zip code plus street name.

According to some embodiments, data security application332may extract a database query or a response to a database query from the intercepted messages510, parse the extracted database query or response and create a security structure according to database protocol rules. The rules may include, but are not limited to, a type of operation or command identified in a query, a database object to be operated on by the operation, and a user identifier of the user requesting the query, identifiers for a service IP address, a client IP address, a client MAC, a network protocol used to access data, a database type, a service name for the name of a service providing data, a name of a database accessed, a source application used for the data access, an application user name, and operating system user, a database related field, an error code, an exception type, a service IP address of the location of data accessed, and additional or alternate rules.

According to some embodiments, data security application332may validate a possible database object access violation in the security structure against security policies defined by the policy rules. In one example, if the security structure does not validate against the security policies, data security application332may issue an alert to an administrator or other entity indicating that the intercepted data packet350has failed to validate against the security rules. In one example, an administrator or service may set each of the rules. According to some embodiments, the security rules may include one or more settings such as, but not limited to, an operation type setting specifying the type of operation access is or is not allowed for, an object setting specifying one or more particular database objects being acted upon by the operation, and a user setting specifying one or more user identifiers for users requesting the operation on the database object. For example, operations that may be restricted by the security rules may include operations such as, but not limited to, create, select, update and delete. The security settings may include additional or alternate types of settings.

Each of database client310, security server330and data server420may be or may include a computing device such as computing device700depicted inFIG. 6. One or more databases326may be or may include a storage device such as storage device730.

Reference is made toFIG. 7, which is a flowchart of a method for associating a message with an end-to-end link, according to embodiments of the invention. An embodiment of a method for associating a message with an end-to-end link may be performed, for example, by the systems shown inFIGS. 1, 2, 3, 4 and 9. A process of associating a message with an end-to-end link may be repeated for a plurality of data packets flowing between a data source machine and a data client.

In operation742, a data packet (such as data packet350) may be captured. The data packet may include a header352and a payload354including query sent to a database by a data client or a response of the database to a query (e.g., message510). For example, an agent software instance located at and executed at the data source machine (e.g. the data base server or a proxy or network gate of the database server) may capture the data packet. In operation744, agent322may decrypt the data packet to obtain a header of the data packet and the message. Agent322may analyze the header to obtain parameters of the end-to-end link such as ServerHostName, ServiceName, DbName, ClientHost, OsUser, DbUser, etc. In operation746, agent322may analyze the header as disclosed herein to associate the packet with an end-to-end link, for example, based on the parameters extracted from the header.

Reference is made toFIG. 8, which is a flowchart of a method for generating a characteristic histogram, according to embodiments of the invention. An embodiment of a method for generating a characteristic histogram may be performed, for example, by the systems shown inFIGS. 1, 2, 3, 4 and 9. An embodiment of a method for generating a characteristic histogram may be performed, for example, in a training stage or preparation stage. According to some embodiments, generating a characteristic histogram may be repeated periodically, e.g., once a week or once a month, to keep the characteristic histogram updated. An embodiment of a method for generating a characteristic histogram may be performed on an end-to-end link360with high volume or high rate of messages. For example, above 100 messages per second.

In operation750, agent322may obtain a set of messages of a single end-to-end link. The set of messages may include a predetermined number of messages or messages obtained over a predetermined time window. For example, in a typical application, agent322may capture about 150,000-400,000 messages of a single end-to-end link in operation750. In some embodiments, agent322may capture the messages in a plurality of sessions of a single end-to-end link. In some embodiments, agent322may capture messages in the first one or two minutes of 20-40 sessions of a single end-to-end link in operation750. Other time windows for capturing messages for generating the signature may be used. In operation752, agent322may generate a skeleton from each of the messages. The skeleton may include the message text except for the value field. In operation754, agent322may generate a construct, or a compressed form of each skeleton. For example, agent322may hash the skeleton to obtain the construct. In operation755, agent322may extract the commands from each of the messages. In operation756, agent322may generate a characteristic histogram for the end to end link. For example, agent322may generate the characteristic histogram from the constructs, as disclosed herein. Additionally or alternatively, agent322may generate the characteristic histogram from the commands of the messages by dividing the commands to commands groups as disclosed herein. Operations750-756may be repeated for a plurality of end-to-end links so that a characteristic histogram may be generated for each of the plurality end-to-end link. In operation758, agent322may cluster or classify end-to-end links to obtain end-to-end links clusters. In some embodiments, the clustering or classification may be performed based on the constructs and commands of the end-to-end links, as disclosed herein. In some embodiments, agent322may generate characteristic histograms of a cluster of end-to-end links. In some embodiments, agent322may generate characteristic histograms for each of the plurality of end-to-end links in a cluster.

Reference is made toFIG. 9, which is a flowchart of a method for dynamically, determining a trust level of an end-to-end link of a computer database, according to embodiments of the invention. An embodiment of a method for dynamically determining a trust level of an end-to-end link may be performed, for example, by the systems shown inFIGS. 1, 2, 3, 4 and 9. A process for determining a trust level of an end-to-end link of a computer database may be repeated periodically and/or for every new session. An embodiment of a method for dynamically determining a trust level of an end-to-end link may be performed, for example, for end-to-end links with high volume or high rate of messages. For example, agent322may not determine a trust level of an end-to-end link for sessions with messages rate below a threshold, e.g., below 100 messages per second. Instead, messages of sessions with messages rate below a threshold may all undergo a comprehensive security analysis.

In operation760, agent322may obtain a set of messages (also referred to as the second set of messages) of a single end-to-end link. The set of messages may include a predetermined number of messages (e.g., 100-400 messages) or messages obtained over a predetermined time window (one-two minutes). In operation762, agent322may generate a skeleton from each of the messages. The skeleton may include the message text except for the value field. In operation764, agent322may generate a construct, or a compressed form of each skeleton. For example, agent322may hash the skeleton to obtain the construct. In operation765, agent322may extract the commands from each of the messages in the second set of messages. In operation766, agent322may generate a work histogram for the end-to-end link. For example, agent322may generate a work histogram from the constructs, as disclosed herein. Additionally or alternatively, agent322may generate the work histogram from the commands of the messages by dividing the commands into command groups as disclosed herein.

In operation768, agent322may compare the work histograms with the characteristic histograms. In some embodiments, agent322may compare the work histograms with the characteristic histograms of the same end-to-end link. In some embodiments, agent322may compare the work histograms with the characteristic histograms of a cluster of end-to-end links to which the end-to-end link pertains. In some embodiments, agent322may compare the work histograms with each of a plurality of characteristic histograms of a cluster of the end-to-end links to which the end-to-end link pertains.

In operation770, agent322may determine a trust level of the end-to-end link. For example, agent322may determine a trust level of the end-to-end link based on the comparison. In some embodiments, agent322may compare the variance of the work histograms with the variance of the characteristic histogram and determine that the trust level of the end-to-end link is not trusted if the variance of the work histogram is larger than the characteristic histogram by a predetermined factor, e.g., two. Other rules may apply. Additionally or alternatively, agent322may determine the trust level of the end-to-end link as being not trusted if at least one of the constructs of the second set of messages is new. Agent322may determine that a construct of the second set of messages is new if the construct is not included in the characteristic histogram. Thus, according to some embodiments, it is enough that the work histogram of an end-to-end link is different from the characteristic histogram of the end to end link, or that at least one of the constructs of the second set of messages is new, for an end-to end link to be considered or categorized as not trusted. The end-to-end link may be considered or categorized as trusted otherwise.

According to some embodiments, if it is determined that the end-to-end link is trusted, agent322may not send messages of the end-to-end link to security server330for comprehensive security analysis. According to some embodiments, agent322may add the constructs used to generate the work histogram to the characteristic histogram to keep the characteristic histogram updated. According to some embodiments, if it is determined in operation770that the end-to-end link is not trusted, agent322may apply policy rules to future messages of the end-to-end link, and if the end-to-end link is trusted, agent322may omit applying policy rules to future messages of the end-to-end link. For example, agent322may send messages of the end-to-end link to security server330for comprehensive security analysis if the end-to-end link is not trusted. In operation778, agent322may issue a security alert if the end-to-end link is not trusted.

FIG. 10illustrates an example computing device according to an embodiment of the invention. Various components such as database client310, security server330, data source machine320, database servers410and420, gateway or proxy422, agent application322and other modules, may be or include, or be executed by, computing device700, or may include components such as shown inFIG. 10. For example, a first computing device700with a first processor705may be used to classify data in real-time for data streaming, according to embodiments of the invention.

Computing device700may include a processor705that may be, for example, a central processing unit processor (CPU), a chip or any suitable computing or computational device, an operating system715, a memory720, a storage730, input devices735and output devices740. Processor705may be or include one or more processors, etc., co-located or distributed. Computing device700may be for example a workstation or personal computer, or may be at least partially implemented by one or more remote servers (e.g., in the “cloud”). For example, computing device700may be included in cloud computing environment50depicted inFIGS. 1 and 2.

Operating system715may be or may include any code segment designed and/or configured to perform tasks involving coordination, scheduling, arbitration, supervising, controlling or otherwise managing operation of computing device700, for example. Operating system715may be a commercial operating system. Operating system715may be or may include any code segment designed and/or configured to provide a virtual machine, e.g., an emulation of a computer system. Memory720may be or may include, for example, a Random Access Memory (RAM), a read only memory (ROM), a Dynamic RAM (DRAM), a Synchronous DRAM (SD-RAM), a double data rate (DDR) memory chip, a Flash memory, a volatile memory, a non-volatile memory, a cache memory, a buffer, a short term memory unit, a long term memory unit, or other suitable memory units or storage units. Memory720may be or may include a plurality of, possibly different memory units.

Executable code725may be any executable code, e.g., an application, a program, a process, task or script. Executable code725may be executed by processor705possibly under control of operating system715. For example, executable code725may be or include software for dynamically determining a trust level of an end-to-end link of a computer database, according to embodiments of the invention. In some embodiments, more than one computing device700may be used. For example, a plurality of computing devices that include components similar to those included in computing device700may be connected to a network and used as a system.

Storage730may be or may include, for example, a hard disk drive, a floppy disk drive, a Compact Disk (CD) drive, a CD-Recordable (CD-R) drive, a universal serial bus (USB) device or other suitable removable and/or fixed storage unit. Storage730may include or may store one or more databases including database326, In some embodiments, some of the components shown inFIG. 10may be omitted. For example, memory720may be a non-volatile memory having the storage capacity of storage730. Accordingly, although shown as a separate component, storage730may be embedded or included in memory720.

Input devices735may be or may include a mouse, a keyboard, a touch screen or pad or any suitable input device. It will be recognized that any suitable number of input devices may be operatively connected to computing device700as shown by block735. Output devices740may include one or more displays, speakers and/or any other suitable output devices. It will be recognized that any suitable number of output devices may be operatively connected to computing device700as shown by block740. Any applicable input/output (I/O) devices may be connected to computing device700as shown by blocks735and740. For example, a wired or wireless network interface card (NIC), a modem, printer or facsimile machine, a universal serial bus (USB) device or external hard drive may be included in input devices735and/or output devices740. Network interface750may enable device700to communicate with one or more other computers or networks. For example, network interface750may include a or Bluetooth device or connection, a connection to an intranet or the internet, an antenna etc.

Embodiments described in this disclosure may include the use of a special purpose or general-purpose computer including various computer hardware or software modules, as discussed in greater detail below.

For the processes and/or methods disclosed, the functions performed in the processes and methods may be implemented in differing order as may be indicated by context. Furthermore, the outlined steps and operations are only provided as examples, and some of the steps and operations may be optional, combined into fewer steps and operations, or expanded into additional steps and operations.

This disclosure may sometimes illustrate different components contained within, or connected with, different other components. Such depicted architectures are merely exemplary, and many other architectures can be implemented which achieve the same or similar functionality.

Aspects of the present disclosure may be embodied in other forms without departing from its spirit or essential characteristics. The described aspects are to be considered in all respects illustrative and not restrictive. The claimed subject matter is indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.