Source: https://patents.google.com/patent/US8286239B1/en
Timestamp: 2019-09-22 12:33:16
Document Index: 218751092

Matched Legal Cases: ['§3', '§3', '§3', '§3', '§3', '§3', '§3', '§3']

US8286239B1 - Identifying and managing web risks - Google Patents
Identifying and managing web risks Download PDF
US8286239B1
US8286239B1 US12/178,772 US17877208A US8286239B1 US 8286239 B1 US8286239 B1 US 8286239B1 US 17877208 A US17877208 A US 17877208A US 8286239 B1 US8286239 B1 US 8286239B1
US12/178,772
2011-03-18 Priority claimed from US13/051,519 external-priority patent/US8763071B2/en
2012-10-09 Publication of US8286239B1 publication Critical patent/US8286239B1/en
2017-07-11 Assigned to SAFECHANNEL INC reassignment SAFECHANNEL INC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SUTTON, MICHAEL
Systems, methods and apparatus for identifying web risks use a web risk service external to network edges of at least one system. The web risk service receives a web request from a computer within the at least one system, the web request identifying at least one network address. The web risk service determines a web risk index score for the at least one network address, and compares the determined web risk index score to at least one threshold value. Based on the comparison, the service determines how to handle the web request, e.g., by forwarding, blocking, and/or logging the web request.
This disclosure relates to identifying risks associated with a web request and handling the web request according to the identified risks.
Layered security systems are prone to processing inefficiencies and can require many resources within the enterprise to maintain the systems. For example, a company may have a multi-layered security system deployed within its network. A file received on company computers may be processed by a content filtering system, an intrusion detection system and pass through the company's firewall to each computer that receives the file. Furthermore, each computer may include malware scanning software that may scan the file when it is received.
Protecting against threats on the Internet is particularly difficult. Malware detection software must be updated periodically on user's computing devices and gateway security systems. As new malware is found, new signatures and/or logic must be added. Distribution of changes in such software programs is expensive and time consuming. Another issue is that for small/mobile computing devices such as phones, PDAs, and Laptops, malware detection programs consume considerable space and require considerable processing time. Yet another problem is the number of devices on which malware detection programs must typically be updated. Typically, organizations have several thousand computing systems, several operating systems, and hardware platforms and malware products from different vendors.
The subject matter described in this specification is directed to identifying web risks, such as malware, resulting from web requests, and handling the web requests accordingly.
In general, one aspect of the subject matter described in this specification can be embodied in systems that include a web risk service external to network edges of at least one protected system. The web risk service is configured to receive a web request from a computer within the protected system. The web risk service will calculate a web risk index score for the destination network address of a given web request; compare the determined web risk index (WRI) score to at least one threshold value; and handle the web request based on the comparison. Other implementations of this aspect include corresponding systems, apparatus, and computer program products.
According to one feature, the web risk service operates at a processing node. According to another feature, the web risk service is further configured to transmit a status of the web request to the computer. The web risk service can be configured to perform a lookup in at least one web risk table to determine the web risk index score. The at least one web risk table can reside within the web risk service. The at least one web risk table can include the values of one or more web risk indicators for a particular domain name, URL, or server identified by the web request.
According to yet another feature, the web risk service can be further configured to identify a plurality of values, each of the plurality of values associated with a respective risk indicator, where at least some of the identified plurality of values contribute to the web risk index score. Each of the respective risk indicators can be associated with at least one of a third party analysis, a passive analysis, or an active analysis. According to another feature, the web risk service is configured to determine the web risk index score by calculating a weighted average of at least some of the identified plurality of values. The web risk service is configured to handle the web request by permitting, blocking, or permitting but logging the web request and/or permitting but providing a warning message to the end user.
Another aspect of the subject matter described in this specification can be embodied in systems that include a web risk service external to the network edges of a protected system, where the web risk service is configured to: receive a web request from a computer within the protected system, the web request identifying at least one network address; determine at least one value identifying a risk indicator for the at least one network address; compare the at least one value to a definitive rule list; and handle the web request based on the comparison. Other implementations of this aspect include corresponding systems, apparatus, and computer program products.
The subject matter described in this specification also includes other aspects. One or more of the following advantages can be realized by implementations of the subject matter described in this specification. Identifying web requests with elevated risk potential is enhanced, resulting in reduced likelihood of system or network compromise, improved network performance and the prevention of information leakage.
FIG. 3 illustrates analysis components that contribute to the calculation of a WRI score.
FIG. 4 shows an example table representing a WRI score calculation.
FIG. 5 is a flow diagram of an example process of identifying the risk of a web request and handling the web request accordingly.
FIG. 6 is a flow diagram of another example process of identifying the risk of a web request and handling the web request accordingly.
FIG. 1 is a block diagram of a distributed security system 100. The system 100 can, for example, be implemented as an overlay network in a wide area network (WAN), such as the Internet. The system 100 includes content processing nodes 110 that detect and preclude the distribution of security and performance threats, e.g., malware, sent from or requested by an external system. Example external systems can include an enterprise 200, a computer device 220, and a mobile device 230, or other network and computing systems.
Data communications can be monitored by the system 100 in a variety of ways, depending on the size and data requirements of the external system. For example, an enterprise 200 may have multiple routers that are used to communicate over the Internet, and the routers may be configured to establish communications through the nearest (in traffic communication time) processing node 110. A mobile device 230 may be configured to communication to a nearest processing node 110 through any available wireless access device, such as an access point, or a cellular gateway. A single computer device 220, such as a consumer's personal computer, may have its browser, e-mail program, or other network capable application configured to access the nearest processing node 110, which, in turn, serves as a proxy for the computer device 220. Alternatively, an Internet provider may have all of its customer traffic processed through processing nodes 110.
In some implementations, each authority node 120 can be implemented by a plurality of computer and communication devices, e.g., server computers, gateways, switches, etc. In some implementations, the authority nodes 120 can serve as an application layer 160. The application layer 160 can, for example, manage and provide policy data, threat data, and data inspection engines and dictionaries for the processing nodes.
In some implementations, the processing node 110 may act as a forward proxy that handles and responds to user requests addressed to remote sites and/or directly to the processing node 110. In other implementations, the processing node 110 may access user requests that are passed through processing node 110 in the transparent mode. A protected system, e.g., enterprise 200, can, for example, choose one or both of these modes.
In another example, the processing nodes 110 can be deployed at Internet service provider (ISP) nodes. The ISP nodes can redirect subject traffic to the processing nodes 110 in a transparent proxy mode. Protected systems, such as the enterprise 200, can use a multiprotocol label switching (MPLS) class of service for indicating the subject traffic that is to be redirected. For example, within the enterprise, an access agent 180 can be configured to perform MPLS labeling.
A wide area network (WAN) 101, such as the Internet, or some other combination of wired and/or wireless networks, connects the data communication of the processing node 110, authority node 120 and logging node 140. The external systems 200, 220 and 230 likewise communicate over the WAN 101 with each other or other data providers and publishers. Some or all of the data communication of each of the external systems 200, 220 and 230 can be processed through the processing node 110.
In some implementations, a client access agent 180 a can be included on a client computer 208. The client access agent 180 a can, for example, facilitate security processing by providing a hash index of files on the user computer 208 to a processing node 110 for malware detection. Other security operations can also be facilitated by the access agent 180 a.
The computer device 220 and the mobile device 230 can also store information warranting security measures, such as personal bank records, medical information, and login information, e.g., login information to the server of the enterprise 200, or to some other secured data provider server.
In some implementations, the processing nodes 110 are external to network edges of the protected systems 200, 220 and 230. Each processing node 110 stores security policies 113 received from the authority node 120 and monitors content items requested by or sent from the external systems 200, 220 and 230. In some implementations, each processing node 110 can also store a detection process filter 112 and/or threat data 114 to facilitate the decision of whether a content item should be processed for threat detection.
The processing node 110 can include data inspection engines 116. Each data inspection engine 116 can be configured to perform a threat detection process to classify content items according to a threat classification for a corresponding threat. For example, the data inspection engines can include a virus scanner engine 116A that can classify a content item as infected or clean, a network URL filter 116B that can classify a URL address as allowed or restricted, a data leakage protection (DLP) engine 116C that can identify a content item as secure or leaking, a dynamic content categorization (DCC) engine 116D that can classify a content item as passed or failed, and a web risk service (WRS) 116E that evaluates and determines how to handle web requests for which a definitive allow/block decision may not have been provided by other inspection engines. The WRS 116E is described in greater detail with respect to FIGS. 3-6.
According to some implementations, the data inspection engines 116 can include a malware detection engine (not illustrated). The list of the data inspection engines 116A-116E is illustrative only; many other data inspection engines 116 can also be used, as can multiple instances of data inspection engines, e.g., different type data leakage engines implementing different data leakage algorithms. The calling of any particular data inspection engine 116 can be predicated on the type of content item to be threat detected. For example, a URL request from the enterprise 200 may cause the processing node manager 118 to call only the URL filter engine 116B.
Because the amount of data being processed by the processing nodes 110 can be substantial, the detection processing filter 112 can be used as the first stage of an information lookup procedure. For example, the detection processing filter 112 can be used as a front end to the threat data 114. Content items can be mapped to index values of the detection processing filter 112 by a hash function that operates on an information key derived from the information item. The information key is hashed to generate an index value (i.e., a bit position). A value of zero in a bit position in the guard table can indicate, for example, absence of information, while a one in that bit position can indicate presence of information. Alternatively, a one could be used to represent absence, and a zero to represent presence.
In some implementations, the processing node 110 can utilize a layered threat detection architecture 111. The layered threat detection architecture 111 can utilize a multi-layered feedback scheme that abstracts threat data between each layer. An example layered threat detection architecture 111 is shown and described in FIG. 3. Other threat detection architectures can also be used, however, and such architectures need not be layered.
The authority node 120 can also store master threat data 124. The master threat data 124 can classify content items by threat classifications, e.g., a list of known viruses, a list of known malware sites, spam e-mail domains, web risk tables, etc. The authority node manager 128 can be used to manage the master threat data 124, e.g., receive updates from a processing nodes 110 when a processing node 110 has processed a content item and update the master threat data 124 with any pertinent results. In some implementations, the master threat data 124 can be distributed to the processing nodes 110, which then store a local copy of the threat data 114.
In some implementations, the authority node 120 can also monitor the health of each processing node 110, e.g., the resource availability in each processing node 110, detection of link failures, etc. Based on the observed health of each process node 110, the authority node 120 can redirect traffic among processing nodes 110 and/or balance traffic among the processing nodes 110. Other remedial actions and processes can also be facilitated by the authority node 120.
The authority node manager 128 can receive the responsive threat data request from the processing node 110 and can determine if the responsive threat data is stored in the authority node data store. If responsive threat data is stored in the master threat data 124, then the authority node manager 128 provides a reply that includes the responsive threat data to the processing node 110 so that the processing node manager 118 can manage the content item in accordance with the security policy data 113 and the classification of the content item.
The authority node manager 128 can receive the responsive threat data request from the processing node 110 and can determine if the responsive threat data is stored in the authority node data store. If responsive threat data is stored in the master threat data 124, then the authority node manager 128 provides a reply that includes the responsive threat data to the processing node 110 so that the processing node manager 118 can manage the content item in accordance with the security policy data 113 and the classification of the content item, and further update the local detection processing filter 112.
§3.0 Identifying Web Risks
The systems described with respect to FIGS. 1 and 2 are operable to calculate a web risk index score associated with all web requests and responses that pass through a processing node. The web risk index score for a given web request is compared against one or more previously established threshold values in order to determine how the request should be handled. For example, the request could be permitted, blocked, permitted but provide a user warning message, or permitted but logged for later analysis.
According to one implementation these processes are performed by a web risk service that can be implemented, for instance, at processing nodes. A processing node responsible for calculating risk scores can also be leveraged to impose restrictions on web requests based on calculated web risk index scores. This permits complete inspection of all data to and from a given resource.
As shown in FIG. 3, according to some implementations a web risk index score is based on a comprehensive weighting of a variety of risk indicators identified by three separate analysis categories: third party analysis 305, passive analysis 310, and active analysis 315. WRI scores can be calculated in real-time by the web risk service for all requests and responses. The WRI scores can then be leveraged to implement security policies by blocking or logging those requests which exceed a given and customizable risk threshold. Risk indicators calculated for past WRI score calculations can be leveraged for subsequent requests, thus limiting the processing required to calculate individual WRI scores for each request. For instance, past WRI scores and previously retrieved risk indicators could also be shared among entities, further increasing performance and accuracy.
§3.1 Control Categories
The various risk indicators for a web request can be drawn from up to three separate control categories, where each category is defined based upon the source of the risk information obtained. The first category, third party analysis, identifies web risk information obtained from a source other than the destination of the web request that will be scored by a processing node. This could include, for instance, third party blacklists or other data sources containing information about the destination of a web request. The second category, passive analysis, identifies risk information contained in the original web request query or web response(s). The third category, active analysis, identifies web risk information based on one or more separate queries to the destination of the web request that is to be scored. The additional query/queries are unique and are designed to solicit a different response in order to identify separate risk information.
§3.1.1 Third Party Analysis
A variety of sources can be leveraged for third party analysis including datasets of known malicious sites (i.e. black lists) or datasets containing factual data unrelated to the malicious nature of a site can provide insight into the potential of increased risk at a given site. The existence of sites in such lists can therefore be web risk indicators that can impact the WRI score of a web request or response.
Black lists, which are lists of web based resources that are known to be malicious, can be used for third party analysis. Black lists may take the form of domain names, fully qualified domain names, IP addresses, Uniform Resource Locators (URLs), or any other format, which uniquely identify a web-based resource. Black lists may also be comprised of string based search patterns, such as regular expressions. The existence or absence of the request in a given black list can contribute to the WRI score. Black lists may be available as free or commercial services. A first example of black lists is phishing archives, which are lists of web sites hosting pages set up for the purpose of phishing activities. Examples of such archives include the Google Safe Browsing API, PhishTank API or Netcraft Phishing Site Feed. A second example of black lists are Autonomous System Number Archives (ASN) identifying malicious sources. An ASN is a number used to identify a collection of IP networks and routers under the control of a single entity, and black lists can identify ASNs which are known to be the source of malicious activity.
Another resource that can be used in third party analysis are Whois queries, which can be leveraged to identify registration information for a given domain name. If the whois records for the domain name of a query correspond to the whois records of a previous query for which the WRI score, or a portion of the WRI score attributable to the domain name, was above an acceptable threshold, this can serve to increase the WRI score of a current web request. This is based on the proposition that the whois records suggest that the same individual or group is responsible for managing both domains. Because at least one is known or suspected to be malicious, the second domain is also suspected of being malicious because it is likely to be managed by the same individual or group.
Recent domain name registrations can be another factor used in third party analysis. Because attackers often register new domain names for the purposes of conducting a specific attack and discontinue use of the site in a short time period, a request sent to sites with recently registered domain names can results in a higher WRI score.
Yet another resource that can be used in third party analysis is past results. It is common for attackers to house multiple attacks on the same server. Therefore, if an otherwise innocuous resource is requested from a server for which the WRI score (or a portion of the WRI score attributable to the server) has been above an acceptable threshold, this can result in an increase in the WRI score of subsequent requests. Referencing Autonomous System Numbers or Classless Inter Domain (CIDR) blocks could also extend this approach to blocks of IP addresses owned or managed by the same entity.
On-Line Discussion Forums can also be leveraged for third party analysis. For instance, on-line discussion forums may contain information regarding web sites/pages known to be vulnerable to attack. The content of such forums could regularly be parsed to identify vulnerable sites, and the existence of a web site in such a forum could serve to increase the WRI score for a request to such a site.
Third party analysis can also leverage the use of binary search engines that permit searches to identify the existence of binary files hosted on web sites. An example of such a search engine is the beta Google Binary Search engine, which allows for example, searching for values within the Portable Executable (PE) header of Windows based binary files. Once known malicious code samples are obtained, characteristics of the files could be identified and binary search engines could then be leveraged to identify web sites hosting such files. The existence of known malicious code being hosted on a web site could increase the WRI score for a web request.
The various third party analysis methods and datasets provided above are example variables that can be components of the overall computation of a WRI score, but still other methods and datasets can also be used.
§3.1.2 Passive Analysis
Passive analysis refers to an in-line analysis of the original request/response made by an end user computer in real time. Such requests can be reviewed for web risk indicators that can affect the WRI score for a web request or response.
Examining for obfuscated code in the response body is a first form of passive analysis. In particular, client side code (e.g. HTML, JavaScript, etc.) can be obfuscated using a variety of methods so that it is not human readable when viewing the page source code without first decoding the data. While there are legitimate reasons for obfuscating client side code, it does not provide a high level security and is commonly employed by malicious sites (i.e. phishing sites, botnets, etc.) as a simple means of avoiding detection. Therefore, the existence of obfuscated code, when not deemed to be employed for a legitimate reason, can serve to raise the WRI score.
Identifying web requests seeking to leverage client side vulnerabilities is another method of passive analysis. A variety of web-based attacks are designed to target end users when making requests to malicious web pages. Such attacks may target vulnerabilities in the web browsers used, employ social engineering to convince the end user to perform an action that will lead to a compromise or combine the two approaches. Examples of attacks involving social engineering would include cross-site scripting (XSS) and cross-site request forgery (CSRF) for which the victim must be convinced to send a specific web request. Inspection of the web request/response traffic in real-time could allow for such attacks to be identified, resulting in an increased WRI score.
Another example of passive analysis is identification of zero pixel iFRAMEs. Zero pixel iFRAMES are an example of an HTML element that is a common indication of malicious activity on a web page. This element is often used to hide any visual indication of a link, which is used to redirect the victim to an alternate page. While the existence a zero-pixel iFRAME would not by itself definitively identify a malicious web page, it can serve to increase the WRI score for a web request or response.
Yet another example of passive analysis is the identification of valid or invalid SSL certificates. There is a financial cost associated with obtaining an SSL certificate, especially from a Trusted Root Certificate Authority which attackers are often unwilling to bear. Therefore, the existence of a valid SSL certificate, especially from a Trusted Root Certificate Authority could serve to lower the overall WRI score. Enhanced SSL certificates have both a higher cost and a greater degree of control associated with validating the certificate owner. Therefore, the existence of a valid Enhanced SSL certificate could serve to lower the WRI score to a greater degree than the existence of a standard SSL certificate could lower the WRI score. Conversely, attackers may host malicious content on an SSL enabled site to enhance the appearance of legitimacy as end users tend to mistake secure browser communications with overall site security. The certificates on such sites may be invalid or self-signed. The existence of invalid or self-signed certificates could serve to increase the WRI score.
The various passive analysis methods and datasets provided above are example variables that can be components of the overall computation of a WRI score, but still other methods and datasets can also be used.
§3.1.3 Active Analysis
A third form of analysis that can impact a WRI score is active analysis, which can be performed through additional web requests sent to the server, which houses the destination of the original web request. Active analysis can be performed to identify web risk indicators related to the same web page or other resources housed on that same server. In some implementations active analysis may be limited to requests deemed appropriate, e.g., by the owners of the servers being analyzed.
One active analysis method seeks to identify server side vulnerabilities, which can impact the WRI score. Web sites can be susceptible to a variety of vulnerabilities. While the direct impact of such vulnerabilities may be on the server itself, due to the fact that the vulnerable server may store confidential data related to end users of the system, these end users can indirectly become victims during an attack. Active analysis of sites visited by end users of the system could be used to identify potential vulnerabilities on a site to be visited, which could then influence the overall WRI score for a given request. Such active analysis could occur either in real-time or subsequent to the initial request and the findings could then be leveraged for subsequent requests to the same site. Active analysis would involve queries designed to identify the existence of potentially vulnerable components such as resources with improper access controls, data injection flaws (i.e. SQL injection), information leakage (i.e. authentication credentials included in source code comments), malicious file execution, etc. Resources such as the OWASP Top 10 or Web Application Security Consortium, Threat Classification1, detail a variety of web server vulnerabilities which can lead to server compromise and increase the risk level for users accessing resources on that server. Such analysis is commonly referred to as black box testing.
Another active analysis method includes fingerprinting web sites to identify possible malicious web sites or code. For instance, additional requests could be made to the server housing the requested resource in order to determine the technologies being deployed on that server. For example, ‘banner grabbing’ requests could be made to receive responses which reveal the name and version number of software deployed on the server. Additionally, technologies implementing identical protocols may respond to requests with subtle differences. Such requests could be compared to fingerprinting databases to obtain confidence scores identifying the likely technologies deployed on the server. Once potential technologies are identified, they can be compared against databases of known vulnerabilities to determine if the server in question may be vulnerable to attack. If so, the web risk index score could be increased for web requests or responses transmitted to or from the server in question.
§3.2 Calculating the WRI Score
where A is the weighting for third party analysis risk indicators, B is the weighting for passive analysis risk indicators, and C is the weighting for active analysis risk indicators. Xn represents the value for the nth risk indicator of third party analysis, Yn represents the value for the nth risk indicator of passive analysis, and Zn represents the value for the nth risk indicator of the active analysis.
Although the above equation equally weights each of the values (representing a respective risk indicator) within each analysis, it will be appreciated that one or more of the values can be weighted individually. For instance, the value X1 in the third party analysis may be weighted more or less than the remaining values of third party analysis. Additionally, one or more values may be weighted to zero, such that they are not included in the overall WRI score. Thus, it will be appreciated that the WRI score calculation shown above is illustrative and that other equations may be used.
§3.2.1 Exceptions to WRI Score Calculation and Illustrative WRI Score
In some implementations WRI scores would not need to be calculated during a request for which a definitive rule was in place, which allowed or disallowed the request outright. As an illustrative example, a rule could be implemented that prohibits any traffic to Site X through, for example, URL filtering controls, such that a WRI score calculation is not required. Any request to Site X would be denied and there would be no reason to perform a WRI score calculation. Thus, the WRI score is calculated where a rule is not in place to definitively determine if a given request should be allowed or denied.
An illustrative example of a WRI score calculation is represented by the table 400 of FIG. 4. The table 400 includes some (but not all) variables described above for the third party analysis, passive analysis, and active analysis. For instance, the WRI score is based on risk identifiers that include the identification of: a domain name in a black list (part of a third party analysis), a recent domain name registration (part of a third party analysis), a zero pixel iFRAME (part of a passive analysis), and a valid SSL Certificate (part of a passive analysis). It will be appreciated that the table 400 is illustrative only, and that one or more other risk indicators described above could have been included in the table 400 as contributing to a WRI score.
As shown in the table 400, values for each risk identifier FIG. 4 are identified. Although these are represented as positive and negative integer values ranging from −5 to +5, almost any values may be attributed to each risk identifier. Thus, a value of +1, 0, or −1 may alternatively be used to indicate whether a particular risk identifier exists and/or is satisfied. Additionally, in the illustrative example of FIG. 4 a scaling factor may be applied to each value to scale, or weight, the risk indicator. This may be done, for instance, to indicate that the factor is more important than other risk indicators in calculating the WRI score. Here, the WRI score is 9 based on the total WRI score contribution from each weighted value of the risk identifiers.
The WRI score can be compared to one or more predefined and customizable risk thresholds that would be used to determine if the request is permitted. As described below, however, the values for each of the risk indicators (or scaled values for each risk indicator) may be compared against a set of definitive rules that mandate how a web request is handled (e.g., forwarded or blocked) regardless of the WRI score and/or existence and/or values for other risk indicators. For instance, referring to the example table 400 of FIG. 4, a definitive rule may indicate that the existence of a domain name in a black list will automatically result in the blocking of the web request.
According to some implementations, WRI scores and/or values for risk indicators (and/or scaled values for risk indicators) may be stored in one or more tables accessible by the web risk service. Upon receipt of a web request the service can perform a lookup in the one or more tables to identify such values, which will prevent the service from having to recalculate or recollect information for a previous request. This process is described in greater detail with respect to FIG. 6, below.
§3.3 Illustrative WRI Calculation Process
FIG. 5 is a flow diagram of an example process 500 for identifying and handling web risks. The process 500 can, for example, be implemented in a web risk service in one or more processing nodes in the system 100 of FIGS. 1 and 2. The stages shown in FIG. 5 are each processes that can be executed independently and concurrently as appropriate.
At stage 502 a web request is received from a computer, where the web request identifies a network address. At stage 504, a WRI score is determined for the network address. For instance, the web risk service can calculate the WRI using appropriate risk indicators, such as provided by third party analysis, passive analysis, and active analysis. At stage 506, the WRI score is compared to at least one threshold value. Next, at stage 508, a determination is made based on the comparison whether the web request is forwarded, blocked, and/or logged by the web service. For instance, if the web request's WRI score fails to exceed a first threshold value it may be forward to its destination. If it exceeds the threshold value it may be forwarded but logged. If the WRI score exceeds a second threshold value it may be blocked. This is an illustrative example only, thus it will be appreciated that only a single threshold value may be used to forward or block web requests.
FIG. 6 is a flow diagram of an example WRI score calculation and threshold comparison process 600. The process 600 can, for example, be implemented in a web risk service in one or more processing nodes in the system 100 of FIGS. 1 and 2. The stages shown in FIG. 5 are each processes that can be executed independently and concurrently as appropriate.
At stage 602 a web request is received from a computer, where the web request identifies a network address. At stage 604 a lookup is performed in one or more WRI table(s) to identify values for risk identifiers. For instance, past WRI risk identifiers could be stored and shared among entities that are part of the same system 100, resulting in increased performance and accuracy. Thus, if the WRI table(s) include values for each risk identifier (stage 606), the calculation and/or identification of such values is avoided (stages 608, 610, 612). Additionally, although not illustrated in this example, in some implementations, the WRI score for a past web request may be stored and retrieved in a table such that a new WRI score calculation need not occur.
If the WRI table(s) do not include values for each risk identifier (stage 606), third party, passive, and active analysis occurs (stages 608, 610, 612) to identify and/or calculate values for risk indicators. Although each of third party, passive, and active analysis are illustrated, one or more of these stages may be eliminated and thus do not have to contribute to the WRI score.
Next, at stage 614, it is determined whether any of the identified values satisfy a definitive rule for allowing or blocking the request. For instance, if one of the values satisfies or exceeds a threshold value, then the web request could be allowed or blocked based on that determination alone. A definitive block/allow rule could also be encountered prior to the process defined in FIG. 6, for example when implementing alternate data inspection engines 116, in which case the entire process could be bypassed altogether. For instance, referring to the example table 400 of FIG. 4, a definitive rule may indicate that the existence of a domain name in a black list will automatically result in the blocking of the web request, regardless of the final WRI score of that web request, and regardless of whether any other risk indicators exist. The definitive rules can be configurable by a system administrator and retrieved for comparison to the identified values. If a definitive rule is applicable, then the web request is handled according to the rule (e.g., allowed or blocked) at stage 616.
At stage 618, a WRI score is calculated based on a combination of the one or more identified values. In some implementations, a weighted average of all risk indicators is used to calculate the WRI score. According to other implementations, each risk indicator may include a scaling factor to increase or decrease it's impact in calculating the WRI score, as described above with respect to the example of FIG. 4.
At stage 620, the WRI score is compared to at least one threshold value. According to some implementations, the at least one threshold value is previously established and stored by the service. Next, at stage 622, the web request is handled based on the comparison. According to some implementations, the web risk service is configured to transmit a status of the web request (e.g., blocked, allowed, warning, logged, etc.) to the computer that transmitted the request. For example, the request could be permitted, blocked, or permitted but logged for later analysis. As an example, if the web request's WRI score fails to exceed a first threshold value it may be forward to its destination. If it exceeds the threshold value it may be forwarded by logged. If the WRI score exceeds a second threshold value it may be blocked. This is an illustrative example only, thus it will be appreciated that only a single threshold value may be used to forward or block web requests
a web risk service operating on a server comprising a processing node and external to network edges of at least one system, the web risk service configured to:
receive a web request from a computer within the at least one system, the web request identifying at least one network address;
determine a web risk index score for the at least one network address, the web risk index score based on a third party analysis, a passive analysis comprising an in-line analysis of the web request in real time, and an active analysis comprising a plurality of queries to the at least one network address with each query configured to solicit a different response in order to identify separate risk information;
compare the determined web risk index score to at least one threshold value;
handle the web request based on the comparison; and
provide the determined web risk index score to an authority node such that the authority node shares the determined web risk index score with a plurality of additional processing nodes;
wherein the web risk service operates at the processing node with all web requests from the computer sent over the Internet via a tunnel, a transparent proxy, a forward proxy, or redirection to the processing node.
2. The system of claim 1, wherein the web risk service is further configured to transmit a status of the web request to the computer.
3. The system of claim 1, wherein the web risk service is further configured to perform a lookup in at least one web risk table to determine the web risk index score.
4. The system of claim 3, wherein the at least one web risk table resides within the web risk service.
5. The system of claim 4, wherein the at least one web risk table comprises the values of one or more web risk indicators for a particular domain name, URL, or server identified by the web request.
6. The system of claim 1, wherein the web risk service is further configured to:
identify a plurality of values, each of the plurality of values associated with a respective risk indicator based on the third party analysis, the passive analysis and the active analysis, wherein at least some of the identified plurality of values contribute to the web risk index score.
7. The system of claim 6, wherein the web risk service is configured to determine the web risk index score by calculating a weighted average of at least some of the identified plurality of values.
the third party analysis comprises utilizing datasets of known malicious sites and analyzing datasets comprising factual data to provide insight into potential risk;
the passive analysis comprises an in-line analysis of the web request in real time to obfuscated code, client side vulnerabilities, common indications of malicious activity, and invalid certificates; and
the active analysis comprises queries sent by the server to identify vulnerable components and fingerprint web sites.
9. The system of claim 1, wherein the web risk service is configured to handle the web request by permitting, blocking, or permitting but logging the web request.
10. The system of claim 1, wherein the web risk service is configured to handle the web request by permitting the web request and providing a warning message for display on the computer.
a web risk service operating between a plurality of servers and external to network edges of at least one system, the web risk service configured to:
perform a third party analysis, a passive analysis, and an active analysis to determine a plurality of values identifying risk indicators;
determine at least one value of the plurality of values identifying a risk indicator for the at least one network address;
compare the at least one value to a definitive rule list;
provide the risk indicator for the at least one network address to an authority node such that the authority node shares the risk indicator with a plurality of processing nodes;
wherein the passive analysis comprises an in-line analysis of the web request in real time to obfuscated code, client side vulnerabilities, common indications of malicious activity, and invalid certificates;
wherein the active analysis comprises a plurality of queries sent by the server to identify vulnerable components and fingerprint web sites, wherein each query is configured to solicit a different response in order to identify separate risk information;
wherein the plurality of servers comprise at least one processing node with all web requests from the computer sent over the Internet via a tunnel, a transparent proxy, a forward proxy, or redirection to the at least processing node.
12. A method of malware detection, comprising:
a server comprising a processing node receiving a web request from a computer within the at least one system over the Internet via a tunnel, a transparent proxy, a forward proxy, or redirection, the web request identifying at least one network address, wherein the computer is connected to an external network directly through the processing node;
the server determining a web risk index score for the at least one network address, the web risk score based on a third party analysis, a passive analysis, and an active analysis;
the server comparing the determined web risk index score to at least one threshold value;
the server handling the web request based on the comparison; and
providing the determined web risk index score to an authority node such that the authority node shares the determined web risk index score with a plurality of additional processing nodes;
wherein the passive analysis comprises an in-line analysis of the web request in real time to obfuscated code, client side vulnerabilities, common indications of malicious activity, and invalid certificates; and
wherein the active analysis comprises a plurality of queries sent by the server to identify vulnerable components and fingerprint web sites, wherein each query is configured to solicit a different response in order to identify separate risk information.
13. The method of claim 12, further comprising transmitting a status of the web request to the computer.
14. The method of claim 12, further comprising performing a lookup in at least one web risk table to determine the web risk index score.
15. The method of claim 14, wherein the at least one web risk table comprises the values of one or more web risk indicators for a particular domain name, URL, or server identified by the web request.
16. The method of claim 12, further comprising identifying a plurality of values, each of the plurality of values associated with a respective risk indicator, wherein at least some of the identified plurality of values contribute to the web risk index score.
17. The method of claim 16, wherein determining the web risk index score comprises calculating a weighted average of at least some of the identified plurality of values.
18. The method of claim 12, wherein handling the web request comprises handling the web request by permitting, blocking, or permitting but logging the web request.
19. The method of claim 12, wherein handling the web request comprises handling the web request by permitting the web request and providing a warning message for display on the computer.
US12/178,772 2008-07-24 2008-07-24 Identifying and managing web risks Active 2030-06-25 US8286239B1 (en)
US8286239B1 true US8286239B1 (en) 2012-10-09
ID=46964366
US12/178,772 Active 2030-06-25 US8286239B1 (en) 2008-07-24 2008-07-24 Identifying and managing web risks
US (1) US8286239B1 (en)
US9800476B2 (en) 2015-02-09 2017-10-24 International Business Machines Corporation Dynamic system segmentation for service level agreements enforcement
EP3251010A4 (en) * 2015-01-30 2018-09-05 Anomali Incorporated Space and time efficient threat detection
WO2019052469A1 (en) * 2017-09-14 2019-03-21 北京金山云网络技术有限公司 Network request processing method and apparatus, electronic device, and storage medium
US10291633B1 (en) * 2016-10-18 2019-05-14 The United States Of America As Represented By The Secretary Of The Army Bandwidth conserving signature deployment with signature set and network security
2008-07-24 US US12/178,772 patent/US8286239B1/en active Active
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SUTTON, MICHAEL;REEL/FRAME:043146/0453