Patent Description:
By way of example, one such computing system is a data center. Data centers are often deployed in a facility that houses computer systems and various associated components, such as telecommunication systems and storage systems. The computing systems are often deployed on networked computer servers (or machines) and include storage that organizations and individuals use to organize and process large amounts of data, and to store that data. The data centers can host applications that are served by the networked computer servers.

Because data centers often host crucial information for various organizations, they are also often a target of surreptitious intrusion. The surreptitious intrusions can be from external sources or from internal sources, such as disgruntled employees.

In order to address such malicious activity, various systems have been developed to detect malicious events in a data center. Such events may include malicious actions that tend to indicate that the data center has been surreptitiously accessed by an unauthorized user. These types of accesses are often called surreptitious or malicious penetration events. Similarly, in order to protect the information stored in data centers, remediation tooling has also been developed that performs a remediation action when a surreptitious penetration event has been detected.

By way of example, some malicious actions include establishing a link to an external IP address. This is often done in an attempt to download information from the data center to the external IP address (or to steal that information). Thus, systems have been developed that detect this type of action as a malicious action. It is indicative of a surreptitious penetration event. In addition, remediation systems have been developed that attempt to block that connection, or to disconnect the data center from the external IP address. The tooling that has been developed to remedy the malicious actions can be automated tooling or manual tooling.

It is also common for one or more data centers to be configured to be setup in different environment configurations. For instance, some tenants may want their data center assets configured according to one configuration, while other tenants may want their data center assets to be configured according to a different configuration. Thus, data centers can be setup in different "environments" that reflect these different types of configurations.

Patent document <CIT> teaches an orchestration system that receives a requested update, which may include a configuration change, a code change, a change to binary, or other change to the installation. A mirror instance of the installation is instantiated on a cloud infrastructure where the requested update is applied and scanned for cybersecurity threats.

The invention is defined in claims <NUM>, <NUM> and <NUM>. In embodiments, a set of high level test logic is configured to include a set of insertion points. The high-level test logic can be controlled to insert test conditions into a data center configuration. It can also be configured to execute remediation actions that are to be taken, and validation actions to be performed in order to determine whether the remediation action works against the test conditions. Different instances of the high-level test logic can be configured for different environments and different test conditions.

The claimed subject matter is not limited to implementations that solve any or all disadvantages noted in the background.

<FIG> shows one example of a computing system architecture <NUM>. Architecture <NUM> illustratively includes one or more data center computing systems <NUM>-<NUM> that have different configurations such as environment configuration <NUM> (in system <NUM>)-environment configuration n (in system <NUM>), respectively.

<FIG> also shows that one or more user computing systems <NUM>-<NUM> are connected to the data center computing systems <NUM>-<NUM> over network <NUM>. Therefore, network <NUM> can be a wide area network, a local area network, a near field communication network, a cellular communication network, or a wide variety of other networks or combinations of networks.

In the example shown in <FIG>, data center computing systems <NUM>-<NUM> can be similar or different. For purposes of the present description, it will be assumed that they are similar so that only data center computing system <NUM> is described in more detail. Data center computing system <NUM> illustratively includes a plurality of different processor/server machines <NUM>, a back end system <NUM>, a front end system <NUM>, data store <NUM>, penetration detection system <NUM>, intrusion remediation system <NUM>, and it can include a wide variety of other data center functionality <NUM>. Front end system <NUM> illustratively exposes an interface that can be accessed by the various user computing systems <NUM>-<NUM>, in order to access the functions, applications and data stored on, and/or hosted by, data center computing system <NUM>. Back end system <NUM> illustratively interacts with data store <NUM> based upon the inputs received from the users through front end system <NUM>. Back end computing system <NUM> can also perform a wide variety of automated functions as well. In one example, the front end system <NUM> and back end system <NUM> are implemented by the processor/server machines <NUM>. The machines <NUM> can be separate computers, but networked, with their own memory, processors, etc..

Penetration detection system <NUM> is illustratively configured to detect malicious activity, or malicious events, that may be indicative of a surreptitious access (or surreptitious penetration event) to data center computing system <NUM> (such as a surreptitious penetration in order to perform some types of malicious activity, by an unauthorized person). There are a variety of different types of penetration detection systems that can be used, and that detect different types of malicious activities that may be indicative of a surreptitious penetration events. They illustratively generate alerts and output signals indicative of the detected activity. In one example, the outputs can be provided to an administrative user computing system so that remediation actions can be taken. In another example, the remediation actions can be taken automatically.

Intrusion remediation system <NUM> illustratively includes live execution logic <NUM>, remediation validation system <NUM>, and it can include a wide variety of other remediation functionality <NUM>. The live execution logic <NUM> illustratively runs during the normal operation of data center computing system <NUM>. When it receives a signal from penetration detection system <NUM> that a surreptitious penetration event (or activity indicative of such an event) has been detected, it illustratively performs remediation actions automatically, or it can be controlled by an administrative user to perform remediation actions.

In either of these examples, intrusion remediation system <NUM> illustratively includes functionality and instrumentation that can be controlled in order to respond to, and remedy, the malicious activity. By way of example, intrusion remediation system <NUM> may receive a signal from penetration detection system <NUM> indicating that a link has been created from data center computing system <NUM> to an external IP address. In that case, intrusion remediation system <NUM> may include instrumentation that sends an alert to an administrative computing system so that an administrative user can disconnect that link. In another example, intrusion remediation system <NUM> includes instrumentation that, when triggered by penetration detection system <NUM>, automatically blocks the link to the external IP address, or disconnects data center <NUM> from that external IP address. It also illustratively notifies an administrative user of the actions that have taken place (both the malicious activity and the remediation actions that have been performed).

In current systems, it can be difficult to tell whether the remediation actions that were performed actually remedied the surreptitious activity that was detected. This can be especially true where the remediation actions are taken automatically. Thus, remediation validation system <NUM> can be configured to validate that the remediation actions that are to be performed in response to detected surreptitious penetration events (or activity that is indicative of such events) actually works.

Remediation validation system <NUM> is described in greater detail below with respect to <FIG>. Briefly, however, it includes logic that is scheduled to run on data center computing system <NUM> continuously, periodically, intermittently, based on other scheduling criteria, or in other ways. It selects a set of machines <NUM> and configures them in an isolated environment so that tests can be conducted on those machines without affecting the actual runtime performance of, and data and applications on, data center computing system <NUM>. It then sets up test pre-conditions to reflect a scenario that the remediation actions are remediating against, and it runs the remediation actions in that isolated environment. It then validates whether the remediation actions resulted in the desired outcome. For instance, where the pre-condition is that the isolated environment has been connected to an external IP address, then the remediation action will be to block that connection, and the validation logic determines whether the connection has actually been blocked (e.g., the remediation validation logic may attempt to connect to the external IP address over the connection and, if unsuccessful, indicates that the remediation action performed as desired).

Remediation validation system <NUM> then generates an output to an administrative computing system indicative of the results of the validation or test of the remediation action. Again, one example of remediation validation system <NUM> is described in greater detail below.

<FIG> also shows that, in one example, user computing system <NUM> generates one or more user interfaces <NUM> for interaction by user <NUM>. User <NUM> illustratively interacts with user interfaces <NUM> in order to control and manipulate user computing system <NUM> and some portions of data center computing system <NUM> or data center computing system <NUM>, or both. Similarly, <FIG> shows that user computing system <NUM> generates user interfaces <NUM> for interaction by user <NUM>. User <NUM> illustratively interacts with user interfaces <NUM> in order to control and manipulate computing system <NUM> and some portions of data center computing system <NUM> or data center computing system <NUM>, or both data center computing systems <NUM> and <NUM>.

It will be noted that one or more of user computing systems <NUM>-<NUM> can be administrative user computing systems or other computing systems that can be used by administrative users or other users to perform functions (such as security functions, maintenance functions, or other functions) on the data center computing systems <NUM>-<NUM>. In addition, one or more of user computing systems <NUM>-<NUM> can be computing systems that are deployed in an organization or tenant where users interact with those computing systems in order to manipulate applications and data on the data center computing systems <NUM>-<NUM>. Both of these architectures are contemplated herein.

User computing systems <NUM>-<NUM> can be similar or different. For purposes of the present discussion it will be assumed that they are similar so that only user computing system <NUM> is described in more detail. User computing system <NUM> illustratively includes one or more processors or servers <NUM>, data store <NUM>, application logic <NUM>, user interface logic <NUM>, communication system <NUM>, and it can include other functionality <NUM>. Application logic <NUM> can be a client component of applications that are hosted in the data center computing systems <NUM>-<NUM>, or it can be logic that runs separate applications. User interface logic <NUM> illustratively generates user interfaces <NUM> and detects user interaction with user interfaces <NUM>. It can provide an indication of those interactions to other components of user computing system <NUM> or to other items in architecture <NUM> over network <NUM>. Communication system <NUM> is illustratively configured to communicate with other items in architecture <NUM> over network <NUM>. Therefore, the type of network <NUM> that user computing system <NUM> is connected to will determine the type of communication system <NUM> that is used to communicate over that network.

<FIG> is a block diagram showing one example of remediation validation system <NUM> in more detail. Remediation validation system <NUM> illustratively includes remediation test instrumentation configuration system <NUM>, environment handling system <NUM>, remediation test scheduling system <NUM>, remediation test execution system <NUM>, and it can include a wide variety of other remediation validation functionality <NUM>. Remediation test instrumentation configuration system <NUM> (or instrumentation configuration system <NUM>) illustratively includes framework instantiation logic <NUM>, pre-condition insertion logic <NUM>, remediation action insertion logic <NUM>, validation logic <NUM>, reset logic <NUM> and it can include other items <NUM>. Environment handling system <NUM> illustratively includes machine selection logic <NUM>, machine isolation logic <NUM>, machine return logic <NUM>, and it can include other items <NUM>. Before describing the operation of remediation validation system <NUM> in more detail, a brief description of some of the items in system <NUM> and their operation, will first be provided.

Remediation test instrumentation configuration system <NUM> illustratively provides a framework by which particular remediation actions, that are designed to remedy various surreptitious activities or conditions, can be validated. Therefore, framework instantiation logic <NUM> first detects a user input indicating that a user wishes to configure a set of test logic or validation logic to validate one or more remediation actions. It then instantiates the remediation test instrumentation (or framework). In one example, the remediation test instrumentation (or framework) is a set of high level test logic that has various insertion points where scenario-specific (e.g., malicious activity-specific or remediation action-specific) items can be inserted in order to configure the high-level code to test logic to validate various remediation actions to determine whether they actually remedied test conditions that are set up. The high-level code can be, for example, a base class that can be configured with the remediation action-specific logic.

Pre-condition insertion logic <NUM> illustratively generates a user interface that allows a user to insert pre-conditions (or test conditions) that will be setup in an environment where the remediation action is to be tested. There may not be pre-conditions that need to be setup, but if there are, they can be inserted using pre-condition insertion logic <NUM>.

Remediation action insertion logic <NUM> illustratively generates a user interface that allows a user to insert or identify the particular remediation actions that are to be triggered or that will be taken when the validation is performed. For example, they can identify the remediation actions that are taken by the intrusion remediation system <NUM> in response to the test conditions, and that are to be validated or tested.

Validation logic <NUM> includes logic that determines whether the remediation actions actually remedied or addressed the conditions that they were intended to address. For example, if the pre-condition that is to be remedied is that a link has been setup between the environment to be tested and an external IP address, and the remediation action is to block that link or connection, then validation logic <NUM> will perform one or more actions to determine whether the remediation action was successful in blocking the connection. This is just one example.

Validation logic <NUM> also illustratively generates an output indicative of the validation results. The output can take a wide variety of forms, some of which are described in greater detail below.

Reset logic <NUM> resets the environment being tested to its original state so that it can be returned to the production environment, after it is tested.

Environment handling system <NUM> illustratively sets up the environment to be tested in a particular data center where system <NUM> is deployed. Machine selection logic <NUM> illustratively selects the set of machines <NUM> that are to be tested. They are illustratively a subset of the machines in the data center computing system <NUM> so system <NUM> continues to function normally, even during the test. The machines can be selected randomly, on a rotating basis, or in other ways. Machine isolation logic <NUM> sets up an isolated environment. It configures the selected machines so that they are isolated from the rest of data center computing system <NUM>. This is because some of the testing that will be performed or some of the test pre-conditions, or the remediation actions, may be destructive in nature. Therefore, the machines are isolated from the remaining environment in computing system <NUM> so that those destructive actions do not destroy or damage any of the production data, applications, etc..

Machine return logic <NUM> illustratively configures the selected machines so that they can be returned and used in the production environment in the data center where they are deployed, after the validation is performed.

Remediation test scheduling system <NUM> can be used to schedule remediation validation system <NUM> to run the configured test instrumentation on a regular basis. For instance, it can be configured to run it continuously, periodically, intermittently, based upon one or more different triggers, or based upon other criteria.

Remediation test execution system <NUM> actually runs the configured test logic that is configured using remediation test configuration system <NUM>. It is triggered by remediation test scheduling system <NUM> to run it at desired times (such as continuously, periodically, intermittently, etc.).

<FIG> is a flow diagram showing one example of the operation of remediation test instrumentation configuration system <NUM> in configuring a set of test logic or instrumentation to perform a test or validation of one or more various remediation actions. Framework instantiation logic <NUM> first detects an input to configure a set of remediation test instrumentation. This is indicated by block <NUM> in the flow diagram of <FIG>. For instance, an administrative user, or a user with security permissions that enable him or her to setup test instrumentation, illustratively authenticates himself or herself to remediation validation system <NUM> and provides an input indicating that he or she wishes to configure the remediation test instrumentation.

Framework instantiation logic <NUM> then instantiates a framework for configuring the remediation test instrumentation. This is indicated by block <NUM>. In one example, logic <NUM> instantiates a set of high-level test logic with mechanisms for inserting remediation action-specific insertions so that various remediation actions, that are triggered in response to malicious activity, can be tested and validated. This is indicated by block <NUM>. The high-level test logic may be a validation base class that has methods that can be called to insert the remediation action-specific information (such as the pre-conditions, remediation actions, validation actions, etc.). Instantiating a validation base class is indicated by block <NUM>. The framework can be instantiated in other ways as well, and this is indicated by block <NUM>.

Framework instantiation logic <NUM> then generates a display user interface with configuration input mechanisms that allow the user to configure the framework to validate various, specific, remediation actions. This is indicated by block <NUM> in the flow diagram of <FIG>. In one example, the configuration input mechanisms can simply allow the user to select from a menu of configuration inputs. In another example, the user can type or otherwise insert the configuration inputs into the instantiated framework. These and other architectures are contemplated herein.

Framework instantiation logic <NUM> then detects user configuration inputs to configure the test/validation instrumentation for one or more remedial actions. This is indicated by block <NUM>.

By way of example, pre-condition insertion logic <NUM> illustratively detects the insertion of pre-conditions that will be established in the environment to be tested. This is indicated by block <NUM>. Remedial action insertion logic <NUM> illustratively detects user inputs inserting, or identifying, the remedial actions that are to be validated, in the framework. This is indicated by block <NUM>. Validation logic <NUM> illustratively detects user inputs inserting validation actions, that are to be executed or performed, in order to validate the remedial actions. This is indicated by block <NUM>.

Reset logic <NUM> illustratively detects user inputs inserting reset actions in the framework, that are to be performed in order to place the environment that was tested back into its original condition so that it can be returned to a production environment. This is indicated by block <NUM>. Detecting user configuration inputs to configure the test/validation instrumentation can be done in a wide variety of other ways as well, and this is indicated by block <NUM>.

Once the remediation test instrumentation has been configured, it can be stored for use in a variety of different ways. In one example, remediation test scheduling system <NUM> detects a user input scheduling the test/validation instrumentation for being run or executed in a particular data center. The test/validation instrumentation can be reused in various environments, or various different data centers. Instances of the configured instrumentation can be scheduled to run differently (e.g., at different times, under different conditions or in response to different triggers) in the different environments or data centers. Detecting a user input scheduling the test/validation instrumentation for execution in a data center is indicated by block <NUM> in the flow diagram of <FIG>.

<FIG> shows one example of a set of high-level test logic or instrumentation <NUM> that has been configured to test or validate a specific set of remedial actions. In one example, the high-level test instrumentation <NUM> is a base class that is configured for a certain scenario (e.g., to test a certain set of remedial actions) that are to be tested. Thus, in one example, instrumentation <NUM> includes test pre-conditions, and logic for establishing them, in an isolated environment that is to be tested. This is indicated by block <NUM>. It also illustratively includes remediation actions (or it identifies the remediation actions). It can also include logic for performing the remediation actions in the isolated environment, or for calling those remediation actions so that they can be tested. It can also include other information. This is indicated by block <NUM>.

Instrumentation <NUM> also illustratively includes the validation actions (or it identifies those actions) and logic to perform them, or call them, in the isolated environment. This is indicated by block <NUM>. Instrumentation <NUM> also illustratively includes reset operations and logic to perform those operations after the validations are performed, in the isolated environment. This is indicated by block <NUM>. The high-level test instrumentation <NUM> can be configured to include other items as well.

<FIG> and <FIG> (collectively referred to herein as <FIG>) illustrate a flow diagram showing one example of the operation of remediation validation system <NUM> in testing or validating a set of remediation actions in data center computing system <NUM>. It is first assumed that the test/validation instrumentation <NUM> is scheduled to be executed in the data center computing system <NUM>. Thus, the instrumentation <NUM> has been configured to test for the efficacy of a certain set of remediation actions, and it has been scheduled so that it is to run at a particular time, on a periodic basis, continually, or in response to other triggers. This is indicated by block <NUM> in the flow diagram of <FIG>.

Remediation test instrumentation execution system <NUM> then determines whether it is time to run the remediation test instrumentation <NUM>. This is indicated by block <NUM>. Again, as discussed above, it may be determined that it is time to run the remediation test instrumentation <NUM> based on elapsed time since it was last executed, based on a number of operations that have been performed in the environment since it was last executed, it can be run continuously, or the determination as to whether the instrumentation should be run can be done in other ways as well. If it is not time to run the remediation test instrumentation <NUM>, then system <NUM> waits, as indicated by block <NUM>, until it is time. However, if, at block <NUM>, it is determined that it is time to run the test instrumentation <NUM>, then instrumentation execution system <NUM> accesses the remediation test instrumentation <NUM> that has been configured, and that is to be executed at this time. This is indicated by block <NUM>. It generates a signal to environment handling system <NUM> and controls machine selection logic <NUM> to select set of data center machines <NUM>, on which the test instrumentation is to operate. This is indicated by block <NUM>. The machines that are selected are illustratively a subset of the machines <NUM> in computing system <NUM> and may be selected at random, as indicated by block <NUM>. They may be selected according to a schedule (such as a rotating schedule where all machines are selected once before they are selected a second time). This is indicated by block <NUM>. The machines may be selected in a wide variety of other ways as well (such as machines that may have a higher propensity to fail, etc.). Selecting the machines in other ways or based on other triggering criteria is indicated by block <NUM>.

Machine isolation logic <NUM> then controls the selected machines so that they are isolated from the remainder of data center computing system <NUM>. Configuring the selected machines as an isolated environment is indicated by block <NUM>.

Execution system <NUM> then begins operating the configured instrumentation <NUM> to validate the remedial actions for which it was configured. Thus, if there are any test pre-conditions <NUM>, then execution system <NUM> executes the logic for establishing those pre-conditions in the isolated environment setup in block <NUM>. Determining whether there are any pre-conditions to establish in the isolated environment and controlling the selected machines to configure the isolated environment so the pre-conditions are present are indicated by blocks <NUM> and <NUM>, respectively, in the flow diagram of <FIG>.

Execution system <NUM> then accesses the remediation actions <NUM> that are set out, or identified, in the test instrumentation <NUM>. It controls the selected machines in the isolated environment to load and execute the logic that performs the remediation actions. This is indicated by block <NUM>. By way of example, if one of the pre-conditions is that the isolated environment is connected to an external IP address, then the remediation action may be to break that connection, or to block it in another way. This is just one example.

Execution system <NUM> then controls the machines in the isolated environment to load and execute the logic that performs the validation actions <NUM> set out in the test instrumentation <NUM>. This is indicated by block <NUM> in the flow diagram of <FIG>. For instance, the validation actions may determine whether the pre-conditions are remedied. Continuing with the example discussed above, if one of the pre-conditions is to have the isolated environment connected to an external IP address, then the validation actions <NUM>, when executed, will determine whether that connection has been broken or blocked. Determining whether the pre-conditions are remedied is indicated by block <NUM> in the flow diagram of <FIG>. The validation actions <NUM> can validate the efficacy of the remedial actions in a wide variety of other ways as well, and this is indicated by block <NUM>.

Execution system <NUM> then generates an output indication that is indicative of the validation results. This is indicated by block <NUM>. In one example, the output can include a scenario description <NUM> which describes the scenario that is being tested. For instance, the scenario may indicate that the remediation actions are taken to disconnect a surreptitious connection to an external IP address. It may describe the scenario in a wide variety of other ways as well.

The output may identify the particular machines tested in the data center computing system <NUM>. This is indicated by block <NUM>. It may identify information contained in the test instrumentation <NUM>. For instance, it may identify the pre-conditions <NUM> that were established. It may identify the remediation actions <NUM> that were taken. It may provide an efficacy indicator <NUM> that indicates the efficacy of those remedial actions (such as whether they remedied what they were supposed to remedy). The results can include a wide variety of other information <NUM> as well.

Once the validation instrumentation has completed its operations, then reset logic <NUM> resets the selected machines so that they can be used in the production environment of data center computing system <NUM>. Resetting the selected machines is indicated by block <NUM> in the flow diagram of <FIG>.

By way of example, it may be that the pre-conditions that were set in the machines must be reset. It may be that the machines were configured in an isolated environment in other ways that need to be reset as well. All of these and other reset operations are contemplated herein.

Machine return logic <NUM> in environment handling system <NUM> then configures the selected machines to return them to the production environment. This is indicated by block <NUM>.

It can thus be seen that the instrumentation discussed herein provides significant advantages and enhances the security of the computing system on which it is deployed. The instrumentation functionally validates remediation actions against test conditions to enhance the likelihood that a remediation action works as designed within a specific environment. The validation instrumentation is automated to continuously test and target multiple environments and roles. The instrumentation can be used to continuously test across environments to determine whether the incident response tooling (or intrusion remediation system <NUM>) is operating properly, even as the data center service changes and expands over time. It also illustratively generates a visualization of remediation action performance that can easily alert an administrative user or engineer when remediation tooling fails. This enhances the ability to swiftly fix the remediation instrumentation.

It will be noted that the above discussion has described a variety of different instrumentation, systems, components and/or logic. It will be appreciated that such instrumentation, systems, components and/or logic can be comprised of hardware items (such as processors and associated memory, or other processing components, some of which are described below) that perform the functions associated with the instrumentation, systems, components and/or logic. In addition, the instrumentation, systems, components and/or logic can be comprised of software that is loaded into a memory and is subsequently executed by a processor or server, or other computing component, as described below. The instrumentation, systems, components and/or logic can also be comprised of different combinations of hardware, software, firmware, etc., some examples of which are described below. These are only some examples of different structures that can be used to form the instrumentation, systems, components and/or logic described above.

The present discussion has mentioned processors and servers. In one embodiment, the processors and servers include computer processors with associated memory and timing circuitry, not separately shown. They are functional parts of the systems or devices to which they belong and are activated by, and facilitate the functionality of the other components or items in those systems.

<FIG> is a block diagram of architecture <NUM>, shown in <FIG>, except that its elements are disposed in a cloud computing architecture <NUM>. Cloud computing provides computation, software, data access, and storage services that do not require end-user knowledge of the physical location or configuration of the system that delivers the services. In various examples, cloud computing delivers the services over a wide area network, such as the internet, using appropriate protocols. For instance, cloud computing providers deliver applications over a wide area network and they can be accessed through a web browser or any other computing component. Software or components of architecture <NUM> as well as the corresponding data, can be stored on servers at a remote location. The computing resources in a cloud computing environment can be consolidated at a remote data center location or they can be dispersed. Cloud computing infrastructures can deliver services through shared data centers, even though they appear as a single point of access for the user. Thus, the components and functions described herein can be provided from a service provider at a remote location using a cloud computing architecture. Alternatively, they can be provided from a conventional server, or they can be installed on client devices directly, or in other ways.

The description is intended to include both public cloud computing and private cloud computing. Cloud computing (both public and private) provides substantially seamless pooling of resources, as well as a reduced need to manage and configure underlying hardware infrastructure.

A public cloud is managed by a vendor and typically supports multiple consumers using the same infrastructure. Also, a public cloud, as opposed to a private cloud, can free up the end users from managing the hardware. A private cloud may be managed by the organization itself and the infrastructure is typically not shared with other organizations. The organization still maintains the hardware to some extent, such as installations and repairs, etc..

In the example shown in <FIG>, some items are similar to those shown in <FIG> and they are similarly numbered. <FIG> specifically shows that data center computing systems <NUM> and <NUM> can be located in cloud <NUM> (which can be public, private, or a combination where portions are public while others are private). Therefore, users <NUM> and <NUM> use user devices <NUM> and <NUM> to access those systems through cloud <NUM>.

<FIG> also depicts another example of a cloud architecture. <FIG> shows that it is also contemplated that some elements of computing systems <NUM> and <NUM> can be disposed in cloud <NUM> while others are not. By way of example, data store <NUM> can be disposed outside of cloud <NUM>, and accessed through cloud <NUM>. In another example, intrusion remediation system <NUM> (or other items) can be outside of cloud <NUM>. Regardless of where they are located, they can be accessed directly by devices <NUM> and <NUM>, through a network (either a wide area network or a local area network), they can be hosted at a remote site by a service, or they can be provided as a service through a cloud or accessed by a connection service that resides in the cloud. All of these architectures are contemplated herein.

It will also be noted that architecture <NUM>, or portions of it, can be disposed on a wide variety of different devices. Some of those devices include servers, desktop computers, laptop computers, tablet computers, or other mobile devices, such as palm top computers, cell phones, smart phones, multimedia players, personal digital assistants, etc..

<FIG> is one example of a computing environment in which architecture <NUM>, or parts of it, (for example) can be deployed. With reference to <FIG>, an example system for implementing some embodiments includes a general-purpose computing device in the form of a computer <NUM>. Components of computer <NUM> may include, but are not limited to, a processing unit <NUM> (which can comprise processors or servers from previous FIGS. ), a system memory <NUM>, and a system bus <NUM> that couples various system components including the system memory to the processing unit <NUM>. The system bus <NUM> may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus. Memory and programs described with respect to <FIG> can be deployed in corresponding portions of <FIG>.

Communication media typically embodies computer readable instructions, data structures, program modules or other data in a transport mechanism and includes any information delivery media. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media.

By way of example only, <FIG> illustrates a hard disk drive <NUM> that reads from or writes to non-removable, nonvolatile magnetic media, and an optical disk drive <NUM> that reads from or writes to a removable, nonvolatile optical disk <NUM> such as a CD ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like.

Operating system <NUM>, application programs <NUM>, other program modules <NUM>, and program data <NUM> are given different numbers here to illustrate that, at a minimum, they are different copies.

These and other input devices are often connected to the processing unit <NUM> through a user input interface <NUM> that is coupled to the system bus, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB).

The computer <NUM> is operated in a networked environment using logical connections to one or more remote computers, such as a remote computer <NUM>. The remote computer <NUM> may be a personal computer, a hand-held device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer <NUM>. The logical connections depicted in <FIG> include a local area network (LAN) <NUM> and a wide area network (WAN) <NUM>, but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.

The modem <NUM>, which may be internal or external, may be connected to the system bus <NUM> via the user input interface <NUM>, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer <NUM>, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation, <FIG> illustrates remote application programs <NUM> as residing on remote computer <NUM>. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

It should also be noted that the different examples described herein can be combined in different ways. That is, parts of one or more examples can be combined with parts of one or more other examples. All of this is contemplated herein.

Claim 1:
A computing system (<NUM>), comprising:
at least one processor; and
memory holding computer-readable instructions, which, when executed by the at least one processor, cause the computer system to:
configure a validation instrumentation (<NUM>) based on a user configuration input, the configured validation instrumentation configured to validate a remedial action performed in the computing system (<NUM>) in response to a detected malicious activity in the computing system:
select a subset of machines from a plurality of machines (<NUM>) in a production environment of the computing system (<NUM>);
configure the subset of selected machines in an isolated environment, the subset of selected machines functionally isolated from other machines in the production environment;
receive a user scheduling input identifying scheduling criteria, under which the configured validation instrumentation is to be run on the computing system (<NUM>); and
based on detecting presence of the scheduling criteria, automatically run the configured validation instrumentation to validate the remedial action on the selected machines in the isolated environment; and
return the subset of machines to the production environment after the validation is performed.