Patent Description:
Computer systems are frequently attacked by malicious actors seeking to compromise the computer systems to steal information, monitor activities on the computer systems without permission, or gain control of the computer system. Security scans are performed to detect evidence of attacks on computer systems and identify computer systems that have been compromised. Any computer system that is connected to the Internet or any wide area network, either directly or indirectly through another computer system, is potentially at risk.

<CIT> discloses an orchestration system that is configured to receive a request to monitor compliance of an enterprise infrastructure and generate an infrastructure change that is associated with the compliance of the enterprise infrastructure, based at least in part on a set of predetermined criteria. In doing so, the orchestration system may further generate one or more infrastructure change events based at least in part on instances of the infrastructure change within the enterprise infrastructure. The orchestration system may further generate a verification report for the enterprise infrastructure, based at least in part on the one or more infrastructure change events, and transmit the verification report to a registered user associated with the request. To perform security testing the orchestration system accesses a mirror of the entire enterprise. <CIT> discloses a computing apparatus, including: a network interface; one or more logic elements providing a security orchestration server engine operable for: receiving contextual data from a client via a network interface; providing the contextual data to a security orchestration state machine, the security orchestration state machine operable for deriving a policy decision from the contextual data; and receiving the policy decision from the policy orchestration state machine.

<CIT> discloses a method of environment security validation through controlled computer network exploitation. A set of parameters is received from an operator over the network. Based on these parameters an attack campaign is performed on the environment. In the course of the campaign, vulnerable hardware and software in the attacked network are identified. In another scenario, vulnerable hardware and software are mitigated until vendors provide official patches.

In accordance with a first embodiment of the present disclosure, disclosed is a method comprising: sending, from a testing computer system to an agent program running on a client computer system, test payloads for the agent program to forward to a set of target systems; receiving, at the testing computer system, test results generated by the set of target systems; generating, with the testing computer system, an analysis of the test results, wherein the analysis indicates that a particular target system of the set of target systems is implicated in a security breach; receiving, at the testing computer system, a user selection of a response to the security breach; compiling, with the testing computer system, a runtime payload that is executable by the agent program to perform the response with the particular target system; sending, from the testing computer system to the agent program, the runtime payload; and receiving, at the testing computer system from the agent program, an indication of execution of the runtime payload.

In accordance with a second embodiment of the present disclosure, disclosed is a non-transitory, computer-readable medium storing instructions that when executed by a computer system cause the computer system to perform operations comprising: sending, from a testing computer system to a client computer system, test payloads for the client computer system to forward to a set of target systems; receiving, at the testing computer system, test results generated by the set of target systems; generating, with the testing computer system, an analysis of the test results, wherein the analysis indicates that a particular target system of the set of target systems is implicated in a security breach; presenting, to a user, one or more selectable actions for the client computer system to perform with the particular target system in response to the security breach; in response to a user selection of one or the selectable actions, sending, from the testing computer system to the client computer system, a runtime payload executable by the client computer system to perform the response; and receiving, at the testing computer system, an indication of execution of the runtime payload.

In accordance with a third embodiment of the present disclosure, disclosed is a method sending, from a testing computer system to an agent program running on a client computer system, test payloads for the agent program to forward to a set of target systems; receiving, at the testing computer system, test results generated by the set of target systems; generating, with the testing computer system, one or more security breach alerts based on the test results; presenting, with the testing computer system in a user interface, the one or more alerts and a plurality of selectable responses, wherein the plurality of selectable responses include terminating one or more processes running on one or more target systems, isolating one or more of the target systems from other target systems, and running an additional scan; and in response to a user selection received via the user interface of a selectable response, sending a runtime payload executable by the agent program to perform the selected response.

This disclosure includes references to "one embodiment" or "an embodiment. " The appearances of the phrases "in one embodiment" or "in an embodiment" do not necessarily refer to the same embodiment. Particular features, structures, or characteristics may be combined in any suitable manner consistent with this disclosure.

Within this disclosure, different entities (which may variously be referred to as "units," "circuits," other components, etc.) may be described or claimed as "configured" to perform one or more tasks or operations. This formulation-[entity] configured to [perform one or more tasks]-is used herein to refer to structure (i.e., something physical, such as an electronic circuit). More specifically, this formulation is used to indicate that this structure is arranged to perform the one or more tasks during operation. A structure can be said to be "configured to" perform some task even if the structure is not currently being operated. A "computer system configured to send test payloads" is intended to cover, for example, a computer system has circuitry that performs this function during operation, even if the computer system in question is not currently being used (e.g., a power supply is not connected to it). Thus, an entity described or recited as "configured to" perform some task refers to something physical, such as a device, circuit, memory storing program instructions executable to implement the task, etc. This phrase is not used herein to refer to something intangible. Thus, the "configured to" construct is not used herein to refer to a software entity such as an application programming interface (API).

The term "configured to" is not intended to mean "configurable to. " An unprogrammed FPGA, for example, would not be considered to be "configured to" perform some specific function, although it may be "configurable to" perform that function and may be "configured to" perform the function after programming.

Reciting in the appended claims that a structure is "configured to" perform one or more tasks is expressly intended not to invoke <NUM> U. § <NUM>(f) for that claim element. Accordingly, none of the claims in this application as filed are intended to be interpreted as having means-plus-function elements. Should Applicant wish to invoke Section <NUM>(f) during prosecution, it will recite claim elements using the "means for" [performing a function] construct.

As used herein, the terms "first," "second," etc. are used as labels for nouns that they precede, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless specifically stated. For example, references to "first" and "second" target systems would not imply an ordering between the two unless otherwise stated.

As used herein, the term "based on" is used to describe one or more factors that affect a determination. This term does not foreclose the possibility that additional factors may affect a determination. That is, a determination may be solely based on specified factors or based on the specified factors as well as other, unspecified factors. Consider the phrase "determine A based on B. " This phrase specifies that B is a factor is used to determine A or that affects the determination of A. This phrase does not foreclose that the determination of A may also be based on some other factor, such as C. This phrase is also intended to cover an embodiment in which A is determined based solely on B. As used herein, the phrase "based on" is thus synonymous with the phrase "based at least in part on.

As used herein, the term "platform" refers to an environment that includes a set of resources that enables some functionality (for example, in the context of the present disclosure, performing computations in a cloud architecture). In some cases, this set of resources may be software resources, such that a platform may be said to be constituted solely of software. In other instances, the set of resources may include software and the hardware on which the software executes. Still further, the resources may constitute specialized hardware that performs the functionality; such specialized hardware may, in some cases, utilize firmware and/or microcode in order to execute. ("Modules" are one type of resource; a given module is operable to perform some portion of the overall functionality of a platform. ) The term "platform" is thus a broad term that can be used to refer to a variety of implementations. Unless otherwise stated, use of the term "platform" in this disclosure will be understood to constitute all possible types of implementations unless otherwise stated. Note that a platform need not be capable by itself of performing the specified functionality. Rather, it need only provide the capability of performing the functionality. For example, a cloud-based computing platform according to the present disclosure provides resources for performing cloud-based distributed computing; users may utilize the platform to carry out various tasks using distributed computer resources. Embodiments of the cloud-based computing platform described herein thus enable the functionality of performing computations on distributed computer resources to be performed.

As used herein, a "module" refers to software and/or hardware that is operable to perform a specified set of operations. A module may in some instances refer to a set of software instructions that are executable by a computer system to perform the set of operations. Alternatively, a module may refer to hardware that is configured to perform the set of operations. A hardware module may constitute general-purpose hardware as well as a non-transitory computer-readable medium that stores program instructions, or specialized hardware such as a customized ASIC.

Various third-party testing computer systems exist to facilitate security scans of vulnerable computer systems that may be subject to attacks by malicious actors. Some of such testing computer systems scan the vulnerable computer systems directly, but other testing computer systems leverage the capabilities of the vulnerable computer systems to test themselves. In the latter case, a testing computer system may send information to its client vulnerable computer systems that the client computer systems used to perform the test. In some instances, a client computer system is a platform that implements or communicates with other computer systems, and the testing computer system sends information to the client computer system, which in turn sends information to the other computer systems. In a prior patent application (<CIT> which is assigned to the same applicant as this application), the inventors described techniques relating to conducting a security scan of a client computer system by sending test payloads from a testing computer system to the client computer system, which in turn sends the test payloads to one or more target systems coupled to the client computer system. The techniques in <CIT> enable security scans to be performed on target systems that are (a) not directly accessible to the testing computer system (e.g., because they do not have a network address that is accessible to the testing computer system), and/or (b) without having to install software that persists on the client computer system. While these techniques are useable to identify security threats, the techniques discussed below enable responses to such identified security threats to be deployed. Moreover, the techniques discussed below also enable a testing computer system to send any of a number of commands to the client computer system and/or target systems even without running a security scan or identifying a security threat.

Referring now to <FIG>, a block diagram illustrating a computer system <NUM> including a testing computer system <NUM>, a client computer system <NUM>, and one or more target systems <NUM> is depicted.

In various embodiments, testing computer system <NUM> includes a payload generator <NUM> and a results evaluator <NUM> and communicates with client computer system <NUM> over a wide area network (WAN) <NUM>. Testing computer system <NUM> can be implemented by any of a number of suitable computer systems (e.g., testing computer system <NUM> can be a single computer or server, one or more dedicated computer systems including multiple computer systems, or implemented with a cloud computing system). In various embodiments, testing computer system <NUM> communicates with WAN <NUM> using one or more wired and/or wireless communications mediums. In various embodiments, testing computer system <NUM> is configured to determine a list of target systems <NUM> coupled to the client computer system <NUM> (e.g., using list <NUM>), generate respective test payloads <NUM> with payload generator <NUM> for a set of the target systems <NUM> (i.e., one or more target systems on list <NUM>), and to send the test payloads <NUM> to the client computer system <NUM> for forwarding to the set of target systems <NUM>. As discussed herein, the test payloads <NUM> are useable by respective target systems <NUM> to perform a security scan of the respective target systems <NUM> and send test results <NUM> to the testing computer system <NUM>. In various embodiments, the test payloads <NUM> also include instructions that cause the test payloads <NUM> to be deleted after the security scan is performed. In various embodiments, testing computer system <NUM> is configured to receive and evaluate test results <NUM> to determine whether any of the set of target systems <NUM> is implicated in a security breach.

Testing computer system <NUM> is configured to generate, using payload generator <NUM>, and send to client computer system <NUM> one or more runtime payloads <NUM> for client computer system <NUM> to perform (e.g., to respond to a security breach, to install additional software on client computer system <NUM> and/or target systems <NUM>, etc.). In some of such embodiments, runtime payload <NUM> is generated in response to test results <NUM> to ameliorate one or more security breaches (e.g., terminating one or more instances of malicious processes executing on client computer system <NUM> and/or a target system <NUM>) identified as a result of client computer system <NUM> and/or target systems <NUM> performing a scan using test payloads <NUM>. In various embodiments, runtime payload <NUM> is generated in response to a user selection of one or more responses to the security breach. In various embodiments, testing computer system <NUM> is configured to receive execution results <NUM> that are indicative of the results of client computer system <NUM> performing runtime payloads <NUM> (e.g., confirmation that a malicious process has been terminated). Testing computer system <NUM>, payload generator <NUM>, and results evaluator <NUM> are discussed in further detail herein with reference to <FIG>.

In various embodiments, client computer system <NUM> can be implemented by any of number of computer systems (e.g., client computer system <NUM> can be implemented with a single computer or server, one or more dedicated computer systems including multiple computer systems, or implemented with a cloud computing system). In some embodiments, client computer system <NUM> is implemented on any of a number of cloud-based computing platforms (e.g., Amazon Web Services®, Microsoft Azure®, Google® Cloud). In some instances, client computer system <NUM> is a commercially available, pay-as-you-go platform that enables users to provision and use one or more virtual machines. In various embodiments, client computer system <NUM> has installed on it an agent module <NUM> (also referred to herein as an agent program) that is operable to facilitate the interaction between testing computer system <NUM> and client computer system <NUM> as discussed herein. In various embodiments, agent module <NUM> is software running on client computer system <NUM> that is installed by testing computer system <NUM>. In particular, in various embodiments agent module <NUM> is operable to receive test payloads <NUM> and forward them to one or more target systems <NUM> for execution, assemble results from the target systems <NUM> into test results <NUM> for transmission to testing computer system <NUM>, receive runtime payloads <NUM> and execute them on client computer system <NUM> (and/or forward them to target systems <NUM> for execution), and send execution results <NUM> to testing computer system <NUM>. List <NUM>, test payloads <NUM>, test results <NUM>, runtime payloads <NUM>, and execution results <NUM> are discussed in further detail in reference to <FIG>, <FIG>, and <FIG>.

In various embodiments, application programming interface (API) <NUM> is a set of subroutine definitions and communication protocols that define methods of communication between client computer system <NUM> and computer systems coupled to WAN <NUM>. Using API <NUM>, testing computer system <NUM> is able to send and receive messages containing information and/or commands to client computer system <NUM> including but not limited to list <NUM>, test payloads <NUM>, test results <NUM>, runtime payloads <NUM>, and execution results <NUM> in various embodiments.

In various embodiments, client computer system <NUM> is coupled to one or more target systems <NUM> (e.g., target systems 124A-n). In these embodiments, target systems <NUM> are able to communicate with client computer system <NUM> (e.g., via HTTP). As discussed herein in reference to <FIG>, in various embodiments, target systems <NUM> are implemented within client computer system <NUM> and/or on one or more remote computer systems. In either instance, because the various target systems <NUM> are able to communicate with the client computer system <NUM>, these target systems <NUM> are referred to herein as being "coupled to" client computer system <NUM>. Thus, being "coupled to" client computer system <NUM> includes (a) being coupled by a communications channel between client computer system <NUM> and a remote target system <NUM>, or (b) being coupled by a communications channel within client computer system <NUM> for target systems <NUM> implemented by client computer system <NUM>. In various embodiments, target computer systems <NUM> include but are not limited to databases, user devices such as laptops or workstations, web servers, etc. Target systems <NUM> are configured to run a security scan upon receiving respective test payloads <NUM> from client computer system <NUM>, generate results of the security scan, and send the test results <NUM> to client computer system <NUM> for forwarding to testing computer system <NUM>. In some embodiments, especially with regard to target systems <NUM> that are remote from client computer system <NUM>, these target systems <NUM> are configured to run a security scan upon receiving respective test payloads <NUM> from client computer system <NUM>, generate results of the security scan, and send the test results <NUM> to testing computer system <NUM> directly (i.e., not via client computer system <NUM>). Similarly, in various embodiments, target systems <NUM> are configured to respond to runtime payloads <NUM> executed by client computer system <NUM> (e.g., a runtime payload <NUM> causes client computer system <NUM> to issue commands to one or more target systems <NUM> which the one or more target systems follow) and/or to execute runtime payloads <NUM> themselves after such runtime payloads are forwarded by agent module <NUM>. Client computer system <NUM>, API <NUM>, target systems <NUM>, and agent module <NUM> are discussed in further detail herein with reference to <FIG>.

WAN <NUM> is one or more wired and/or wireless networks via which testing computer system <NUM> and client computer system <NUM> are configured to communicate (e.g., messages including list <NUM>, test payloads <NUM>, test results <NUM>, runtime payloads <NUM>, execution results <NUM>). In various embodiments, WAN <NUM> includes the Internet. Each device that is coupled to WAN <NUM> has a publicly accessible address (e.g., a public IP address, an Internet IP) useable to differentiate each device from the other devices coupled to WAN <NUM>. As discussed herein, various target systems <NUM> are not directly coupled to WAN <NUM> and do not have publicly accessible address. Instead, the various target systems <NUM> have private addresses (e.g., private IP addresses, local IP addresses) to facilitate communication via one or more local networks, but such private addresses are not exposed to WAN <NUM>. In various embodiments, when a device (e.g., a target system <NUM>) without a publicly accessible address receives or sends information via WAN <NUM> via a message, a device (e.g., a router) translates the message through a publicly accessible address, which is useable to communicate the message with other publicly accessible addresses and eventually to other local networks. Thus, various target systems <NUM> have access to WAN <NUM>, but indirectly (e.g., though a router).

However, because private addresses are not visible on WAN <NUM>, such architecture prevents testing computer system <NUM> from identifying target systems <NUM> that do not have publicly accessible addresses. Accordingly, testing computer system <NUM> is unable to determine a list of all of the target systems <NUM> that sit behind the publicly accessible address(es) used by client computer system <NUM>. In various embodiments, testing computer system <NUM> requests list <NUM> from client computer system <NUM>. In various embodiments, list <NUM> includes indications of a plurality of target systems <NUM> as well as indications of hardware, software, and/or settings of the target systems <NUM> that testing computer system <NUM> is configured to use to determine test payloads <NUM> and/or runtime payloads <NUM> for a set of the target systems <NUM> as discussed herein. After testing computer system <NUM> sends the test payloads <NUM> to client computer system <NUM>, client computer system <NUM> forwards the test payloads <NUM> to their respective target systems <NUM> who perform security scans and generate test results <NUM>. Client computer system <NUM> sends test results <NUM> to testing computer system <NUM>, which evaluates the test results using results evaluator <NUM>.

Similarly, in various embodiments, testing computer system <NUM> may not be able to communicate with one or more target systems <NUM> to provided instructions on how to the target systems <NUM> should respond to one or more issues identified in a security scan, but testing computer system <NUM> is operable to send runtime payloads <NUM> to agent module <NUM>, which in turn can effectuate the performance of the runtime payloads <NUM> by the various target systems <NUM> using the architecture of client computer system <NUM>. For example, in various embodiments agent module <NUM> is implemented by client computer system <NUM> with administrator access privileges to client computer system <NUM> and the various target systems <NUM>. Thus, in various embodiments, through agent module <NUM> testing computer system <NUM> is operable to cause client computer system <NUM> and target systems <NUM> to perform any of a number of actions in response to the results of a security scan (e.g., terminating processes, updating software or firmware, restarting physical or virtual machines, etc.) as discussed herein. Similarly, in various embodiments, this same architecture allows testing computer system <NUM>, through agent module <NUM>, to cause client computer system <NUM> and target system <NUM> to perform any action that an administrator-level user of client computer system <NUM> is authorized to do including but not limited to provisioning or deprovisioning additional target systems <NUM>; deploying monitoring software; and installing, updating, or deleting software on client computer system <NUM> and/or target systems <NUM>.

As discussed in further detail below, in various embodiments both the test payloads <NUM> and the runtime payloads <NUM> may be performed without having to install software other than agent module <NUM> on client computer system <NUM>. Accordingly, in various embodiments the techniques disclosed herein enable testing computer system <NUM> to perform security scans of client computer system <NUM> (and target systems <NUM>) and to send to messages to client computer system <NUM> to effectuate a response to the results of the security scan without installing software other than agent module <NUM> on client computer system <NUM>. In various embodiments, the test payloads <NUM> also include instruction to delete the test payloads <NUM> after the security scan. Similarly, in various embodiments, the runtime payloads <NUM> also include instruction to delete the runtime payloads <NUM> after executing. In other embodiments, test payloads <NUM> and/or runtime payloads <NUM> do not include instructions to delete the test payloads <NUM> and/or runtime payloads <NUM>, but are scripts that are not retained in memory on target systems <NUM> or client computer system <NUM> after the scripts have been performed. In such embodiments, the test payloads <NUM> and runtime payloads <NUM> are overwritten when the target systems <NUM> load other operations (e.g., operations for programs installed on target system <NUM>) into memory. In embodiments, because these test payloads <NUM> and runtime payloads <NUM> are only delivered and executed on demand, long term performance side effects (e.g., reduced cloud performance and higher hosting costs due to overprovisioning or slower processing times) are reduced or eliminated.

Referring now to <FIG>, an expanded block diagram of testing computer system <NUM> is depicted. In the embodiments shown in <FIG>, testing computer system <NUM> includes payload generator <NUM> and results evaluator <NUM> shown in <FIG> and a user interface <NUM>. Testing computer system <NUM> is configured to communicate with client computer system <NUM> to receive list <NUM>, determine a set of target systems <NUM> to test, generate and send test payloads <NUM>, receive and evaluate results <NUM>, generate and send runtime payloads <NUM>, receive execution results <NUM>, and present information to a user via user interface <NUM>.

In various embodiments, payload generator <NUM> includes a test payload generator <NUM> and a runtime payload generator <NUM>. In various embodiments, test payload generator <NUM> includes a plurality of modules operable to generate test payloads <NUM> including but not limited to service level module <NUM>, operating system (OS) module <NUM>, running processes module <NUM>, loaded modules module <NUM>, loaded drivers module <NUM>, autostart settings module <NUM>, user information module <NUM>, log events module <NUM>, installed software module <NUM>, OS subversion module <NUM>, and fileless running code module <NUM>. In various embodiments, test payload generator <NUM> includes all of these modules, but in other embodiments payload test payload generator <NUM> includes additional modules (e.g., modules that add additional tests to the test payloads <NUM>) or fewer modules.

Service level module <NUM> is operable to determine the set of target systems <NUM> to test. In some embodiments, the set includes each of the target systems <NUM> on list <NUM>. In other embodiments, the size of the set of target systems <NUM> to test is limited by a service level (e.g., a lower level of service in which only a subset of target systems <NUM> are tested). In other embodiments, service level module <NUM> receives an indication (e.g., scan request <NUM> discussed in connection to <FIG>) and focus the test on a subset of target systems <NUM> based on the indication (e.g., scan request <NUM> indicates that database target systems <NUM> should be tested but user workstation target systems <NUM> should not be tested, scan request <NUM> indicates a potential security breach in certain target systems <NUM>).

The various modules <NUM>-<NUM> are executable to add operations to the test payloads <NUM> to gather details regarding the running state of a target system <NUM>, without the requirement to install heavy software or rely on other tooling such as logs or existing security apparati. In various embodiments, because the hardware, software, and settings of various target systems <NUM> vary, test payloads <NUM> for different types of target systems <NUM> also vary. List <NUM> includes indications of a plurality of target systems <NUM> as well as indications of hardware, software, and/or settings of the target systems <NUM>. Payload generator <NUM> and the various modules <NUM>-<NUM>, therefore, are operative to assemble different test payloads <NUM> for different configurations of target systems <NUM> based on the indications in list <NUM>. For example, in various embodiments, first test payloads <NUM> are generated for target systems <NUM> having a first OS and second, different test payloads <NUM> are generated for target systems <NUM> having a second, different OS. In another example in various embodiments, first test payloads <NUM> are generated for target systems <NUM> implemented within client computer system <NUM> and second test payloads are generated for target systems <NUM> implemented on computer systems that are remote from client computer system <NUM>.

In various embodiments, OS module <NUM> is executable to include operations for the security scan caused by target systems <NUM> executing test payloads <NUM> that: determine version and architecture information about the OS running on the target systems <NUM> and, in some embodiments, also determine patch information about the OS running on the target systems <NUM>.

In various embodiments, running processes module <NUM> is executable to include operations for the security scan caused by target systems <NUM> executing test payloads <NUM> that: determine what programs and processes are being executed on target systems <NUM>, and in embodiments, collect binary metadata along with associated user accounts and process identification data.

In various embodiments, loaded modules module <NUM> is executable to include operations for the security scan caused by target systems <NUM> executing test payloads <NUM> that: determine loadable modules running on target system <NUM>. Modern OSs share common code/functionality through loadable modules (e.g., generally. dll files on Microsoft Windows®,. so files on Linux®, and. dylib files on macOS®). Malicious applications can inject modules into running processes to change their behavior and subvert detection mechanisms. In various embodiments, the security scan collects binary metadata as well as what process(es) they are loaded in.

In various embodiments, loaded drivers module <NUM> is executable to include operations for the security scan caused by target systems <NUM> executing test payloads <NUM> that: obtain a list of low-level drivers in use by the OS. Drivers generally provide access to hardware but also serve to provide additional functionality to the OS itself. Malware can attempt to install a malicious driver to subvert detection mechanisms that take place in non-kernel managed space, called user space/user land. In various embodiments, the security scan collects binary metadata for this list of drivers.

In various embodiments, autostart settings module <NUM> is executable to include operations for the security scan caused by target systems <NUM> executing test payloads <NUM> that: obtain the list of events during various boot or login events. These are grouped together under the term "autostarts" and executed automatically and without user interaction (automatically). For example, on Microsoft Windows® autostarts can include things like scheduled tasks, login triggers, items listed in various registry locations known as "run keys", along with many others. As another example, on Unix®-like systems this can include things like cron jobs, user profile scripts, systems or upstart services, etc. In various embodiments, the security scan enumerates all of these various locations and lists the entries found along with binary metadata about the files themselves.

In various embodiments, user information module <NUM> is executable to include operations for the security scan caused by target systems <NUM> executing test payloads <NUM> that: obtain information about user accounts on target system <NUM>. In various embodiments, this includes but is not limited to (a) collecting information about the local users that have accessed the system, (b) collecting information about network users that have accessed the system, and/or (c) collecting information about access times and/or account privileges of users of local users or network users.

In various embodiments, log events module <NUM> is executable to include operations for the security scan caused by target systems <NUM> executing test payloads <NUM> that: check logs from one or more built-in logging features of the OS of the target system <NUM> or any other program running on target system <NUM>. In various embodiments, security scans may collect, for example, time and detail information for user login events and/or potentially malicious events such as log or audit clearing events.

In various embodiments, installed software module <NUM> is executable to include operations for the security scan caused by target systems <NUM> executing test payloads <NUM> that: gather a list of what software is installed on target system <NUM>. In various instances, this information is useful for auditing what the user(s) of target system <NUM> have done. In other instances, this can also identify if known malware or software vulnerable to exploit is present within a target system <NUM>.

In various embodiments, OS subversion module <NUM> is executable to include operations for the security scan caused by target systems <NUM> executing test payloads <NUM> that: check for malicious methods intended to alter the way the OS of target system <NUM> functions. For example, a technique known as "hooking" a code execution path involves modifying the way internal code functions, often to mask activity or provide incorrect results to user applications. In various embodiments, the security scan looks at how important functions are supposed to be executed, and inspects current running processes to ascertain whether or not that is how they are functioning on target system <NUM>. In such instances, discrepancies are reported as results <NUM>.

In various embodiments, fileless running code module <NUM> is executable to include operations for the security scan caused by target systems <NUM> executing test payloads <NUM> that: check for "fileless execution" of malware. Many types of malware attempt to hide their execution using a variety of techniques known as "fileless execution. " In various instances, this can be a script that is run without saving the file to disk or a full executable loaded into the memory of another running process and then executed. If a security scan's detection mechanism relies on being able to inspect files saved to a disk, such techniques might not be detected by the security scan. Operations to check for fileless execution enable the security scan to inspect the memory area of running applications on the target system <NUM> and identifies code in execution that should not be there. In various embodiments, these findings are reported in the test results <NUM> as well as the binary data of the code in question.

Results evaluator <NUM> is operable to interpret test results <NUM> and determine whether a particular target system is implicated in a security breach. In some embodiments, results evaluator <NUM> is deterministic and looks for specific behaviors or results (e.g., the name of known malware installed on target computer <NUM>, the presence of known malicious modules on target system <NUM>). Additionally or alternatively, in some embodiments, results evaluator <NUM> is statistical and looks for trends in test results <NUM> that are probabilistically associated with compromised systems. For example, results evaluator <NUM> includes a model and uses machine learning techniques to determine whether test results <NUM> from a particular target system <NUM> indicate that the particular target system <NUM> has been compromised. In either or both such embodiments, if such specific behaviors or results are detected and/or comparison to a model indicates that a target system <NUM> is compromised, then results evaluator <NUM> reports that the target system <NUM> may be compromised and identifies the area(s) of concern.

In various embodiments, runtime payload generator <NUM> is operable to generate runtime payloads <NUM> to effectuate responses to security scans. In various embodiments, runtime payload generator <NUM> includes a built-in runtime module <NUM> and a custom runtime module <NUM> useable to generate runtime payloads <NUM> to perform built-in responses useable on multiple different client computer systems <NUM> and custom responses that are specific for a particular client computer system <NUM>, respectively. While the operation of runtime payload generator <NUM> is discussed herein primarily as effectuating responses to the results of a security scan, it will be understood that in various embodiments, runtime payload generator <NUM> can generate runtimes to effectuate execution of any of a number of actions by client computer system <NUM> whether a security scan is performed or not (e.g., effectuating a patch of software on target systems <NUM> or restarting target systems <NUM> without performing a security scan as discussed herein). Built-in runtime module <NUM> includes one or more routines or scripts that are operable to be compiled and performed on any client computer system <NUM> and include but is not limited to terminating one or more particular processes, isolating target systems <NUM> from other target systems <NUM> (e.g., to keep malicious code from spreading), quarantining files, and performing further investigation of a security breach (e.g., performing a more extensive scan of one or more target systems <NUM>, recovering files on a target system <NUM>, etc.). Custom runtime module <NUM> includes one or more custom routines or scripts that are entered by a user on behalf of a particular client computer system <NUM> (and/or target systems <NUM>) and operable to be compiled and performed by the particular client computer system <NUM> (and/or target systems <NUM>). Examples of custom routines or scripts includes but are not limited to installing, updating, or patching software on client computer system <NUM>; installing, updating, or patching software on one or more target computer systems <NUM>; entering commands to software installed on client computer system <NUM> and/or target computer systems <NUM> (e.g., a command to run an instance of particular software installed on a target computer system <NUM>), etc. In various embodiments, runtime payload generator <NUM> is operable to compile source code into runtime binary (a) suitable for running on client computer system <NUM> based on the hardware and software used to implement client computer system <NUM> and/or (b) suitable for running on target systems <NUM> based on the hardware and software used to implement target systems <NUM>. For example, in embodiments where client computer system <NUM> is implemented by a particular cloud-based computing platform, runtime payload generator <NUM> is operable to generate binary runtime files suited to the particular cloud-based computing platform. In various embodiments, runtime payload generator <NUM> is operable to compile runtimes for any number of operating system, hardware and software architectures, and cloud-based computing platforms. For example, in embodiments where one or more target systems <NUM> are implemented using Windows OS, runtime payload generator <NUM> is operable to compile runtimes that are executable on Windows OS.

User interface <NUM> is operable to present information to a user of testing computer system <NUM> about security scans performed as a result of execution of test payloads <NUM>, about results <NUM> of a security scan, about available responses that may be effectuated using runtime payload generator <NUM>, and/or about the status of responses being performed. As discussed in further detail in reference to <FIG> and <FIG>, user interface <NUM> is operable to present information to and receive commands from a user computer system (e.g., user computer system <NUM> shown in <FIG>).

Referring now to <FIG>, an expanded block diagram of client computer system <NUM> is depicted. As in <FIG>, client computer system <NUM> includes API <NUM> and agent module <NUM>. The client computer system <NUM> depicted in <FIG> additionally includes a plurality of target systems <NUM> implemented by client computer system <NUM> including a target system 124C implemented with private cloud <NUM> and a runtime editor environment module <NUM>. In various embodiments, client computer system <NUM> is also configured to send or receive a scan request <NUM>, a list request <NUM>, and/or a scan start command <NUM> in addition to being configured to send or receive list <NUM>, test payloads <NUM>, test results <NUM>, runtime payloads <NUM>, and execution results <NUM> as discussed herein. In various embodiments, client computer system <NUM> does not include all of the elements depicted in <FIG>. For example, client computer system <NUM> does not include a private cloud or runtime editor environment module <NUM> in various embodiments. As another example, client computer system <NUM> might send a single message combining scan request <NUM> and list <NUM> without receiving list request <NUM>.

In various embodiments, agent module <NUM> is software that is installed on client computer system <NUM> and facilitates the running of security scans (e.g., by executing test payloads <NUM> and/or distributing test payloads <NUM> to target systems <NUM>, by assembling and communicating test results <NUM>) and the execution of runtime payloads <NUM> (e.g., by executing the runtime binary in runtime payloads <NUM> and/or by distributing runtime payloads <NUM> to target systems for execution). In some embodiments, agent module <NUM> executes the runtime binaries within client computer system <NUM> which causes the client computer system <NUM> (and the target systems <NUM> implemented by and in communication with client computer system <NUM>) to perform whatever actions are dictated by the runtime payload <NUM>. In other embodiments, runtime payloads <NUM> indicate a particular target system <NUM> and agent module <NUM> forwards such runtime payloads <NUM> to the particular target system <NUM> for execution. In some of such embodiments, agent module <NUM> gathers information about the progress of the execution of the runtime payloads <NUM> and communicates such information back to testing computer system as execution results <NUM>.

In various embodiments, agent module <NUM> is operable to provide a runtime editor environment via runtime editor environment module <NUM>. For the purposes of discussion herein, runtime editor environment module <NUM> is depicted as being a separate module from agent module <NUM>, but in various embodiments, runtime editor environment module <NUM> is a component of agent module <NUM>. As discussed in further detail in reference to <FIG> and <FIG>, the runtime editor environment provided by runtime editor environment module <NUM> is useable to provide a user with the ability to code custom scripts or routines to be executed on client computer system <NUM>. Such custom scripts or routines may be sent from client computer system <NUM> to testing computer system <NUM> (e.g., custom scripts <NUM> shown on <FIG>) and may be compiled by runtime payload generator <NUM> of testing computer system <NUM> to generate runtime binary which can be sent back to the client computer system <NUM> as one or more runtime payloads <NUM> in various embodiments. As discussed herein, in various embodiments agent module <NUM> is operable effectuate the execution of such runtime payloads <NUM> by executing the runtime payloads <NUM> and/or distributing them to target systems <NUM>.

In various embodiments, agent module <NUM> is also operable to monitor client computer system <NUM> and/or one or more target system <NUM> to determine whether to initiate a security scan. In some embodiments, agent module <NUM> receives an indication that a security breach is suspected to have occurred at one or more target systems <NUM>. In such embodiments, agent module <NUM> in turn sends scan request command <NUM> to testing computer system <NUM> to initiate the security scans as discussed herein. For example, IT personnel associated with the entity operating client computer system <NUM> could receive an indication that a security breach has occurred by seeing that confidential information from a target system <NUM> (e.g., user account information) has been posted on the internet, and the IT personnel send an indication to agent module <NUM> (e.g., via a GUI), and agent module <NUM> sends scan request message <NUM>. Additionally or alternatively, agent module <NUM> monitors for the provisioning of new target systems <NUM> or receives an indication from client computer system <NUM> that new target systems <NUM> have been provisioned. In some of such embodiments, agent module <NUM> sends scan request <NUM> to initiate a security scan of target systems <NUM> to form a baseline for subsequent security scans (i.e., a brand new target system <NUM> is unlikely to have been attacked already, so the test results <NUM> from the new target system <NUM> could be used to compare with future scans). Additionally or alternatively, agent module <NUM> causes the security scan to be initiated periodically (e.g., every week, every month, every two months, etc.).

In the embodiment shown in <FIG>, client computer system <NUM> is coupled to target system A 124A and implements target systems 124B, 124C, and 124n. In various embodiments, target system 124A is implemented on a third computer system that is remote from client computer system <NUM>. Thus, in various embodiments target system 124A is implemented on a separate client computer system <NUM> or in separate computer system (e.g., a desktop computer, a laptop, a server, or a private data center). In some embodiments, target system 124A is located on the premises of the entity that is utilizing the cloud computing platform. Target systems 124B, 12C, through 124n are implemented by client computer system <NUM>.

In various embodiments, client computer system <NUM> implements one or more target systems <NUM> (e.g., target system 124C shown in <FIG>) in a private cloud <NUM>. In various embodiments, private cloud <NUM> is a logically isolated section of client computer system <NUM> where resources can be launched in a virtual network that is defined by an entity using client computer system <NUM>. In various embodiments, the virtual networking environment of private could <NUM> is controllable down to the selection of IP address ranges, the creation of subnets, and the configuration of route tables and network gateways. As discussed herein, in some embodiments, some target systems <NUM> (e.g., target systems 124B-124n shown in <FIG>) have publicly accessible address on WAN <NUM> that can be accessed via WAN <NUM> while other target systems (e.g., target system 124A and target system 124C shown in <FIG>) do not have publicly accessible addresses on WAN <NUM>.

Accordingly, an entity (e.g., an online storefront, an online service provider) utilizing client computer system <NUM> to perform tasks (e.g., the tasks associated with the online store or online service) is able to use client computer system <NUM> to coordinate the performance of these tasks using various target systems <NUM> that are implemented within client computer system <NUM> or remote from client computer system <NUM>. In one non-limiting example, an entity providing online media streaming services can use client computer system <NUM> to provision a public-facing subnet (e.g., a website that serves the streaming media) with target systems 124B-n such that users can access the website over WAN <NUM>. In this example, target system 124C is a database storing media and is implemented within private cloud <NUM> that is not directly accessible via WAN <NUM>, but is in communication with the public-facing subnet such that media from the database can be streamed to users via the website. In this example, target system 124A is a development server located in a private datacenter belonging to the entity that is used by developers to test new features for the website. In this example, because all of target systems 124A, 124B, 124C, and 124n are in communication with (or implemented by) client computer system <NUM>, each of these target systems <NUM> can be scanned as discussed herein without implementing persistent scanning software (other than agent module <NUM>) on the target system <NUM> even though target systems 124A and 124C are not directly accessible via WAN <NUM>. Similarly, through runtime payloads <NUM> sent to client computer system and executed by (or using) agent module <NUM>, testing computer system <NUM> can effectuation the execution of various types of commands (e.g., commands to terminate a process, commands to install or update software) by any of these target systems even though target systems 124A and 124C are not directly accessible via WAN <NUM>.

In various embodiments, testing computer system <NUM> sends list request <NUM> to client computer system <NUM>. In such embodiments, list request <NUM> requests that client computer system <NUM> provide list <NUM>, which includes indications of a plurality of target systems <NUM> as well as indications of hardware, software, and/or settings of the target systems <NUM> in various embodiments as discussed herein. In some embodiments, list requests <NUM> requests a list <NUM> of all target computer systems <NUM> in communication with (or implemented by) client computer system <NUM> that are operated for a particular entity (e.g., an online storefront, an online service provider). In other embodiments, list request <NUM> requests a list <NUM> for a subset of target computer systems <NUM> in communication with (or implemented by) client computer system <NUM> that are operated for a particular entity. For example, list request <NUM> may request a list <NUM> of all target computer systems <NUM> implemented by target computer system <NUM>. In various embodiments, list request <NUM> is received by cloud computing platform via API <NUM>. In various embodiments, list request <NUM> is sent by testing computer system <NUM> in response to receiving scan request <NUM>. In other embodiments, however, no scan request <NUM> is received and the security scan process discussed herein is initialized by testing computer system <NUM> sending list request <NUM>.

In various embodiments, testing computer system <NUM> sends a scan start command <NUM> to client computer system <NUM> after sending test payloads <NUM>. In such embodiments, scan start command <NUM> is useable by the client computer system <NUM> to instruct target systems <NUM> to start their security scans using the test payloads <NUM>. In various embodiments, scan start command <NUM> is an application program interface call made to client computer system <NUM> via API <NUM>. In such embodiments, for example, API <NUM> is operable to enable computer systems remote from client computer system <NUM> (such as testing computer system <NUM>) to command client computer system <NUM> and various target systems <NUM> in communication with (or implemented by) client computer system <NUM> to execute any of a number of operations including the various operations in test payloads <NUM>. In various embodiments, scan start command <NUM> is a separate message, but in other embodiments scan start command <NUM> is incorporated with test payload <NUM> such that target systems receive their respective test payloads <NUM> and execute the security scans without further command.

Referring now to <FIG>, a block diagram illustrating the computer system <NUM> depicted in <FIG> in communication with a user computer system <NUM> is depicted in accordance with various embodiments. Testing computer system <NUM> and client computer system <NUM> include various components as discussed above, but only certain components are shown in <FIG> for clarity. As shown in <FIG>, testing computer system <NUM> is configured to communicate with user computer system <NUM> over WAN <NUM> and client computer system <NUM> is configured to communicate with user computer system <NUM> via link <NUM>. In various embodiments, communication over link <NUM> includes communication over WAN <NUM>, over a local area network, or by direct terminal access to client computer system <NUM> in various embodiments. In some embodiments, communication over link <NUM> includes user computer system <NUM> communicating with client computer system <NUM> via testing computer system <NUM> (e.g., using user interface <NUM>).

User computer system <NUM> can be implemented by any of a number of suitable computer systems (e.g., user computer system <NUM> can be a single computer or server, one or more dedicated computer systems including multiple computer systems, or implemented with a cloud computing system). In various embodiments, user computer system <NUM> is a desktop computer, laptop computer, tablet computer, or a smartphone. In various embodiments, user computer system <NUM> communicates with WAN <NUM> using one or more wired and/or wireless communications mediums. User computer system <NUM> is configured to communicate with testing computer system <NUM> over WAN <NUM> (although some embodiments, user computer system <NUM> is configured to communicate with testing computer system <NUM> over a local area network or by a direct terminal access). In the embodiments shown in <FIG>, user computer system <NUM> is configured to interface with testing computer system <NUM> using user interface <NUM>. In such embodiments, user computer system <NUM> is configured to receive alerts <NUM> via user interface <NUM> and to send indications of selected responses <NUM>. As discussed in further detail herein, user interface <NUM> causes the screens discussed in connection to <FIG> to be displayed to a user using user computer system <NUM> (e.g., on a display coupled to user computer system <NUM>) as part of presenting alerts <NUM> and receiving indications of selected responses <NUM>, in various embodiments.

In the embodiments shown in <FIG>, testing computer system <NUM> is configured to generate one or more alerts <NUM> and send such alerts <NUM> to user computer system <NUM> via user interface <NUM>. In various embodiments, alerts <NUM> are based on an analysis of test results <NUM> that indicates a security breach (e.g., a malicious process running on client computer system <NUM> or a target system <NUM>). Such alerts <NUM> are presented to a user via user interface <NUM> and user computer system <NUM>, and in some instances alongside one or more selectable responses to the alert <NUM>. Example screens of a user interface <NUM> displaying one or more alerts and one or more selectable responses is shown in <FIG> and <FIG>. In the embodiments shown in <FIG>, a user is able to select from one or more responses to respond to a given alert <NUM>. In such embodiments, the user inputs a selection via user computer system <NUM>, which sends an indication of the selected response <NUM> to testing computer system <NUM>.

As discussed herein, in various embodiments, potential selectable responses to an alert <NUM> that may be displayed to the user include by are not limited to:.

Testing computer system <NUM> is configured to generate runtime payloads <NUM> in response to the indication of the selected response <NUM>. In the instance of items (a)-(f) above, runtime payload generator <NUM> is operable (using built-in runtime module <NUM>) to compile a runtime binary tailored for the particular client computer system <NUM> and to send the runtime binary in a runtime payload <NUM>. In the instance of item (g) above, runtime payload generator <NUM> is operable (using custom runtime module <NUM> and a previously-received custom script or routine corresponding to the selected customer response) to compile a runtime binary tailored for the particular client computer system <NUM> and to send the runtime binary for the customer response in a runtime payload <NUM>. In various embodiments, agent module <NUM> running on client computer platform <NUM> is not operable to perform any of the responses identified above without the corresponding runtime payload <NUM>, and after executing of the runtime payload <NUM>, the runtime payload <NUM> is deleted from memory as discussed herein. In various embodiments, a user is able to browse a list of available actions (including build-in and custom actions) that may be performed via runtime payloads <NUM> sent to agent module <NUM> via user interface <NUM>. An example screen presented by user interface <NUM> to user computer system <NUM> of a list of such available actions is depicted in <FIG>.

In various embodiments, a user is able to utilize user computer system <NUM> to input, via link <NUM>, a custom script or routine into a runtime editor environment provided using runtime editor environment module <NUM>. An example screen presented by user interface <NUM> to user computer system <NUM> of a runtime editor environment is depicted in <FIG>. In various embodiments, such a custom script or routine may be written in any of a number of suitable high-level programming languages (e.g., Lua, PowerShell, JavaScript etc.). As discussed herein, in various embodiments the custom script or routine may define a user selectable response that can be executed in response to a security threat. In other instances, however, the custom script or routine can define one or more actions that can be performed at any time, regardless of whether a security threat has been detected. For example, the custom script or routine may define a software patch or update that may be distributed to client computer system <NUM> and/or target systems <NUM> regularly or when a new version is available. As another example, the custom script or routine may define one or more actions to perform on particular a target system <NUM> having a particular type of hardware or software (e.g., reboot a target system <NUM> running Windows OS into safe mode and collect the system log; perform one or more commands that specific to Amazon Web Services®, Microsoft Azure®, or Google Cloud® platforms). After a custom script or routine is entered into client computer system <NUM>, client computer system <NUM> is operable to send the custom script or routine to testing computer system (shown as the arrow labeled customer script(s) <NUM> in <FIG>). After receiving the custom script or routine, testing computer system <NUM> is operable to compile the custom script or routine to generate a runtime payload <NUM> for the custom script or routine as discussed herein (e.g., with runtime payload generator <NUM>). As discussed herein, client computer system <NUM> performs the runtime payloads (e.g., using agent module <NUM>) and execution results <NUM> may be sent from client computer system <NUM> to testing computer system <NUM>. Such execution results <NUM> may be presented to the user via user interface <NUM>. Example screens presented by user interface <NUM> including execution results <NUM> are depicted in <FIG> and <FIG>.

<FIG> depict flowcharts representing various disclosed methods implemented with the components depicted in <FIG>. Referring now to <FIG>, a flowchart illustrating an embodiment of a target system scanning method <NUM> is shown. In the embodiment shown in <FIG>, the various actions associated with method <NUM> are implemented by testing computer system <NUM> in communication with client computer system <NUM> and target systems <NUM>. At block <NUM>, testing computer system <NUM> determines, using API <NUM> of client computer system <NUM>, a list <NUM> of target systems <NUM> coupled to the client computer system <NUM>. At block <NUM>, testing computer system <NUM> generates respective test payloads <NUM> for a set of the target systems <NUM>. Each respective test payload <NUM> is useable by its respective target system <NUM> to perform a security scan of the target system <NUM> and send test results <NUM> to the testing computer system <NUM>. Each test payload <NUM> includes instructions that cause the test payloads <NUM> to be deleted after the security scan is performed. At block <NUM>, testing computer system <NUM> sends test payloads <NUM> to client computer system <NUM> for forwarding to the set of target systems <NUM>. At block <NUM>, testing computer system <NUM> receives test results <NUM>. At block <NUM>, testing computer system <NUM> evaluates test results <NUM> to determine whether any of target systems <NUM> is implicated in a security breach.

Referring now to <FIG>, a flowchart illustrating an embodiment of a target system scanning method <NUM> is shown. In the embodiment shown in <FIG>, the various actions associated with method <NUM> are implemented by testing computer system <NUM> in communication with client computer system <NUM> and target systems <NUM>. At block <NUM>, testing computer system <NUM> determines, using an API <NUM> of client computer system <NUM>, a list <NUM> of target systems <NUM> coupled to the client computer system <NUM>. At block <NUM>, testing computer system <NUM> generates respective test payloads <NUM> for a set of the target systems <NUM>. Each respective test payload <NUM> is useable by its respective target system <NUM> to perform a security scan including: gathering, as test results <NUM>, information about software running on the target system <NUM> and information stored at the target system <NUM>, sending test results <NUM> to testing computer system <NUM>, and deleting test payload <NUM> after completing the security scan. At block <NUM>, testing computer system <NUM> sends test payloads <NUM> to client computer system <NUM> for forwarding to the set of target systems <NUM>.

Referring now to <FIG>, a flowchart illustrating an embodiment of a system scanning and response method <NUM> is shown. At block <NUM>, testing computer system <NUM> sends, to an agent program (e.g., agent module <NUM>) running on a client computer system <NUM>, one or more test payloads <NUM> for the agent program to forward to a set of target systems <NUM>. At block <NUM>, testing computer system <NUM> receives rest results <NUM> generated by the set of target systems <NUM>. At block <NUM>, testing computer system <NUM> generates an analysis of the test results <NUM>. The analysis indicates that a particular target system <NUM> of the set of target systems <NUM> is implicated in a security breach. At block <NUM>, testing computer system <NUM> receives a user selection of a response to the security breach. At block <NUM>, testing computer system <NUM> compiles one or more runtime payloads <NUM> that are executable by the agent program (e.g., agent module <NUM>) to perform the response with the particular target system <NUM>. At block <NUM>, testing computer system <NUM> sends to the one or more runtime payloads <NUM> to the agent program. At block <NUM>, testing computer system <NUM> receives from the agent program an indication of execution of the runtime payload (e.g., execution results <NUM>).

Referring now to <FIG>, a flowchart illustrating an embodiment of a system scanning and response method <NUM> is shown. At block <NUM>, testing computer system <NUM> sends, to an agent program (e.g., agent module <NUM>) running on a client computer system <NUM>, one or more test payloads <NUM> for the agent program to forward to a set of target systems <NUM>. At block <NUM>, testing computer system <NUM> receives test results <NUM> generated by the set of target systems <NUM>. At block <NUM>, testing computer system <NUM> generates one or more security breach alerts (e.g., one or more alerts <NUM>). At block <NUM>, testing computer system <NUM> presents in a user interface <NUM>, the one or more alerts (e.g., alerts <NUM>) and a plurality of selectable responses (e.g., depicted in screens <NUM> and <NUM>). In various embodiments, the plurality of selectable responses includes terminating one or more processes running on one or more target systems, isolating one or more of the target systems from other target systems, and running an additional scan. At block <NUM>, in response to a user selection received via the user interface <NUM> of a selectable response (e.g., a selected response <NUM> input via user interface <NUM>), testing computer system <NUM> sends one or more runtime payloads <NUM> executable by the agent program to perform the selected response.

Referring now to <FIG> example screens of a user interface (e.g., presented using user interface <NUM>) presented to a user computer system <NUM> are shown. Referring now to <FIG>, a screen <NUM> is depicted. Screen <NUM> includes a window <NUM> that allows a user to enter a custom script or routine into a runtime editor environment provided using runtime editor environment module <NUM>. In the depicted embodiment, a user is able to enter a name for the customer script or routine into name field <NUM> and select a type from dropdown menu <NUM>. In the depicted embodiment, the custom routine shown in screen <NUM> labeled as a response (e.g., an action to perform in response to a security breach identified in a security scan), but other labels may be used such as "Analysis" (e.g., for actions to gather addition information from target systems <NUM>), "Install" (e.g., for actions to install software, "Update" (e.g., for actions to update software"), etc. In various embodiments, a user may apply any desired label to a custom script or routine. In text box <NUM>, a text editor is presented allowing a user to add, change, or delete code for the custom script or routine. The code shown in <FIG> relates to a script that terminates a process written in Lua, but as discussed herein, a user is able to enter any code that may be executed by client computer platform <NUM> written in any suitable language.

Referring now to <FIG>, a screen <NUM> includes a list <NUM> of the actions <NUM> (also referred to herein as "extensions") that are available. Using toggle <NUM>, a user can enable or disable an action <NUM>. Referring now to <FIG>, a screen <NUM> includes a list <NUM> of entries <NUM> corresponding to alerts <NUM>. In the embodiment shown in <FIG>, the entries <NUM> include the name of the software identified to be a threat, a type (e.g., process, artifacts, modules, drivers, memory, accounts, applications, network connections, etc.), an indicator of the host running the identified threat (e.g., a network name of the corresponding target system <NUM> running the identified threat), a status of the identified threat (e.g., whether it is currently running), a threat level <NUM> assigned to the identified threat, a timestamp, and a menu <NUM> of available actions. In the embodiment shown in <FIG>, menu <NUM> includes selections for a response (e.g., causing a runtime payload <NUM> to be sent to client computer platform <NUM> as discussed herein), analyze (e.g., see more information collected about identified threat), as well as actions to control list <NUM> such as adding the identified threat to a blacklist or whitelist, removing an entry <NUM> from list <NUM>, and archiving an entry <NUM>. In various embodiments, threat level <NUM> is determined using a combination of collected threat intelligence and analysis conducted by one or more Bayesian machine learning algorithms and data science using entropy methods. These are then weighted and used to determine the threat of the item. If a user selects "response" in the embodiment shown in <FIG>, a screen <NUM> as shown in <FIG> may be presented. Screen <NUM> include a popup <NUM> that include a list of available actions <NUM> that may be executed as a response. While <FIG> only includes a "terminate process" response, it will be understood that any of the responses discussed herein may be included in popup <NUM> in various instances (e.g., a selectable response to update software running on a particular target system <NUM>, a selectable response to reboot a particular target system <NUM>, etc.). When a user makes a selection in popup <NUM>, an indication of selected response <NUM> is sent to testing computer system <NUM>, one or more runtime payloads <NUM> corresponding to the response are generated, and such runtime payloads <NUM> are sent to client computer platform <NUM> as discussed herein.

Referring now to <FIG> and <FIG>, screens <NUM> and <NUM> showing information indicative of execution results <NUM> are shown. In screen <NUM>, a popup <NUM> is overlaid on screen <NUM> discussed in reference to <FIG>. Popup <NUM> include a list of entries <NUM> that correspond to individual selected responses that are being executed or have previously been executed by client computer system <NUM> with an indication of the status of each (e.g., in progress, completed, aborted, etc.). If a user selects "view all," screen <NUM> is presented showing additional information about the various responses. Screen <NUM> includes a list <NUM> of entries <NUM> that correspond to individual selected responses that are being executed or have previously been executed by client computer system <NUM> with an indication of the status of each (e.g., in progress, completed, aborted, etc.).

Turning now to <FIG>, a block diagram of an exemplary computer system <NUM>, which may implement the various components of computer system <NUM> (e.g., testing computer system <NUM>, client computer system <NUM>, target systems <NUM>) is depicted. Computer system <NUM> includes a processor subsystem <NUM> that is coupled to a system memory <NUM> and I/O interfaces(s) <NUM> via an interconnect <NUM> (e.g., a system bus). I/O interface(s) <NUM> is coupled to one or more I/O devices <NUM>. Computer system <NUM> may be any of various types of devices, including, but not limited to, a server system, personal computer system, desktop computer, laptop or notebook computer, mainframe computer system, tablet computer, handheld computer, workstation, network computer, a consumer device such as a mobile phone, music player, or personal data assistant (PDA). Although a single computer system <NUM> is shown in <FIG> for convenience, system <NUM> may also be implemented as two or more computer systems operating together.

Processor subsystem <NUM> may include one or more processors or processing units. In various embodiments of computer system <NUM>, multiple instances of processor subsystem <NUM> may be coupled to interconnect <NUM>. In various embodiments, processor subsystem <NUM> (or each processor unit within <NUM>) may contain a cache or other form of on-board memory.

System memory <NUM> is usable to store program instructions executable by processor subsystem <NUM> to cause system <NUM> perform various operations described herein. System memory <NUM> may be implemented using different physical memory media, such as hard disk storage, floppy disk storage, removable disk storage, flash memory, random access memory (RAM-SRAM, EDO RAM, SDRAM, DDR SDRAM, RAMBUS RAM, etc.), read only memory (PROM, EEPROM, etc.), and so on. Memory in computer system <NUM> is not limited to primary storage such as memory <NUM>. Rather, computer system <NUM> may also include other forms of storage such as cache memory in processor subsystem <NUM> and secondary storage on I/O Devices <NUM> (e.g., a hard drive, storage array, etc.). In some embodiments, these other forms of storage may also store program instructions executable by processor subsystem <NUM>.

Claim 1:
A method comprising:
sending, from a testing computer system (<NUM>) to an agent program running on a client computer system (<NUM>), test payloads (<NUM>) for the agent program to forward to a set of target systems (<NUM>);
sending, from the testing computer system (<NUM>) to the client computer system (<NUM>), a scan start command (<NUM>) after sending the test payloads (<NUM>), wherein the scan start command (<NUM>) is a separate message from the test payloads (<NUM>);
receiving, at the testing computer system (<NUM>), test results (<NUM>) generated by the set of target systems (<NUM>);
generating, with the testing computer system (<NUM>), an analysis of the test results (<NUM>), wherein the analysis indicates that a particular target system (<NUM>) of the set of target systems (<NUM>) is implicated in a security breach;
receiving, at the testing computer system (<NUM>), a user selection of a response to the security breach;
compiling, with the testing computer system (<NUM>), a runtime payload (<NUM>) that is executable by the agent program to perform the response with the particular target system (<NUM>);
sending, from the testing computer system (<NUM>) to the agent program, the runtime payload (<NUM>); and
receiving, at the testing computer system (<NUM>) from the agent program, an indication of execution of the runtime payload (<NUM>).