Patent Description:
Embodiments of the disclosure include a method for drilling a well that includes generating a plurality of proposed drilling actions using a plurality of working agents based on a working environment, simulating drilling responses to the proposed drilling actions using a plurality of validation agents in a validation environment that initially represents the working environment, determining rewards for the proposed drilling actions based on the simulating, using the validation agents, selecting one of the proposed drilling actions, and causing a drilling rig to execute the selected one of the proposed actions.

Embodiments of the disclosure include a non-transitory computer-readable medium storing instructions that, when executed by at least one processor of a computing system, cause the computing system to perform operations. The operations include generating a plurality of proposed drilling actions using a plurality of working agents based on a working environment, simulating drilling responses to the proposed drilling actions using a plurality of validation agents in a validation environment that initially represents the working environment, determining rewards for the proposed drilling actions based on the simulating, using the validation agents, selecting one of the proposed drilling actions, and causing a drilling rig to execute the selected one of the proposed actions.

Embodiments of the disclosure include a computing system including one or more processors, and a memory system including one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations. The operations include generating a plurality of proposed drilling actions using a plurality of working agents based on a working environment, receiving a manual proposed drilling action from a human user, simulating drilling responses to the proposed drilling actions and the manual proposed drilling action using a plurality of validation agents in a validation environment that initially represents the working environment, determining rewards for the proposed drilling actions and the manual proposed drilling action based on the simulating, using the validation agents, selecting one of the proposed drilling actions or the manual proposed drilling action, selecting the one of the proposed actions is based on the rewards for the proposed actions, and causing a drilling rig to execute the selected one of the proposed actions.

For example, a first object or step could be termed a second object or step, and, similarly, a second object or step could be termed a first object or step, without departing from the scope of the present disclosure. The first object or step, and the second object or step, are both, objects or steps, respectively, but they are not to be considered the same object or step.

The terminology used in the description herein is for the purpose of describing particular embodiments and is not intended to be limiting. As used in this description and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and/or" as used herein refers to and encompasses any possible combinations of one or more of the associated listed items. It will be further understood that the terms "includes," "including," "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. Further, as used herein, the term "if" may be construed to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context.

Attention is now directed to processing procedures, methods, techniques, and workflows that are in accordance with some embodiments. Some operations in the processing procedures, methods, techniques, and workflows disclosed herein may be combined and/or the order of some operations may be changed.

<FIG> illustrates an example of a system <NUM> that includes various management components <NUM> to manage various aspects of a geologic environment <NUM> (e.g., an environment that includes a sedimentary basin, a reservoir <NUM>, one or more faults <NUM>-<NUM>, one or more geobodies <NUM>-<NUM>, etc.). For example, the management components <NUM> may allow for direct or indirect management of sensing, drilling, injecting, extracting, etc., with respect to the geologic environment <NUM>. In turn, further information about the geologic environment <NUM> may become available as feedback <NUM> (e.g., optionally as input to one or more of the management components <NUM>).

In the example of <FIG>, the management components <NUM> include a seismic data component <NUM>, an additional information component <NUM> (e.g., well/logging data), a processing component <NUM>, a simulation component <NUM>, an attribute component <NUM>, an analysis/visualization component <NUM> and a workflow component <NUM>. In operation, seismic data and other information provided per the components <NUM> and <NUM> may be input to the simulation component <NUM>.

In an example embodiment, the simulation component <NUM> may rely on entities <NUM>. Entities <NUM> may include earth entities or geological objects such as wells, surfaces, bodies, reservoirs, etc. In the system <NUM>, the entities <NUM> can include virtual representations of actual physical entities that are reconstructed for purposes of simulation. The entities <NUM> may include entities based on data acquired via sensing, observation, etc. (e.g., the seismic data <NUM> and other information <NUM>). An entity may be characterized by one or more properties (e.g., a geometrical pillar grid entity of an earth model may be characterized by a porosity property). Such properties may represent one or more measurements (e.g., acquired data), calculations, etc..

In an example embodiment, the simulation component <NUM> may operate in conjunction with a software framework such as an object-based framework. In such a framework, entities may include entities based on pre-defined classes to facilitate modeling and simulation. A commercially available example of an object-based framework is the MICROSOFT®. NET® framework (Redmond, Washington), which provides a set of extensible object classes. NET® framework, an object class encapsulates a module of reusable code and associated data structures. Object classes can be used to instantiate object instances for use in by a program, script, etc. For example, borehole classes may define objects for representing boreholes based on well data.

In the example of <FIG>, the simulation component <NUM> may process information to conform to one or more attributes specified by the attribute component <NUM>, which may include a library of attributes. Such processing may occur prior to input to the simulation component <NUM> (e.g., consider the processing component <NUM>). As an example, the simulation component <NUM> may perform operations on input information based on one or more attributes specified by the attribute component <NUM>. In an example embodiment, the simulation component <NUM> may construct one or more models of the geologic environment <NUM>, which may be relied on to simulate behavior of the geologic environment <NUM> (e.g., responsive to one or more acts, whether natural or artificial). In the example of <FIG>, the analysis/visualization component <NUM> may allow for interaction with a model or model-based results (e.g., simulation results, etc.). As an example, output from the simulation component <NUM> may be input to one or more other workflows, as indicated by a workflow component <NUM>.

As an example, the simulation component <NUM> may include one or more features of a simulator such as the ECLIPSE™ reservoir simulator (Schlumberger Limited, Houston Texas), the INTERSECT™ reservoir simulator (Schlumberger Limited, Houston Texas), etc. As an example, a simulation component, a simulator, etc. may include features to implement one or more meshless techniques (e.g., to solve one or more equations, etc.). As an example, a reservoir or reservoirs may be simulated with respect to one or more enhanced recovery techniques (e.g., consider a thermal process such as SAGD, etc.).

In an example embodiment, the management components <NUM> may include features of a commercially available framework such as the PETREL® seismic to simulation software framework (Schlumberger Limited, Houston, Texas). The PETREL® framework provides components that allow for optimization of exploration and development operations. The PETREL® framework includes seismic to simulation software components that can output information for use in increasing reservoir performance, for example, by improving asset team productivity. Through use of such a framework, various professionals (e.g., geophysicists, geologists, and reservoir engineers) can develop collaborative workflows and integrate operations to streamline processes. Such a framework may be considered an application and may be considered a data-driven application (e.g., where data is input for purposes of modeling, simulating, etc.).

In an example embodiment, various aspects of the management components <NUM> may include add-ons or plug-ins that operate according to specifications of a framework environment. For example, a commercially available framework environment marketed as the OCEAN® framework environment (Schlumberger Limited, Houston, Texas) allows for integration of add-ons (or plug-ins) into a PETREL® framework workflow. The OCEAN® framework environment leverages. NET® tools (Microsoft Corporation, Redmond, Washington) and offers stable, user-friendly interfaces for efficient development. In an example embodiment, various components may be implemented as add-ons (or plug-ins) that conform to and operate according to specifications of a framework environment (e.g., according to application programming interface (API) specifications, etc.).

<FIG> also shows an example of a framework <NUM> that includes a model simulation layer <NUM> along with a framework services layer <NUM>, a framework core layer <NUM> and a modules layer <NUM>. The framework <NUM> may include the commercially available OCEAN® framework where the model simulation layer <NUM> is the commercially available PETREL® model-centric software package that hosts OCEAN® framework applications. In an example embodiment, the PETREL® software may be considered a data-driven application. The PETREL® software can include a framework for model building and visualization.

As an example, a framework may include features for implementing one or more mesh generation techniques. For example, a framework may include an input component for receipt of information from interpretation of seismic data, one or more attributes based at least in part on seismic data, log data, image data, etc. Such a framework may include a mesh generation component that processes input information, optionally in conjunction with other information, to generate a mesh.

In the example of <FIG>, the model simulation layer <NUM> may provide domain objects <NUM>, act as a data source <NUM>, provide for rendering <NUM> and provide for various user interfaces <NUM>. Rendering <NUM> may provide a graphical environment in which applications can display their data while the user interfaces <NUM> may provide a common look and feel for application user interface components.

As an example, the domain objects <NUM> can include entity objects, property objects and optionally other objects. Entity objects may be used to geometrically represent wells, surfaces, bodies, reservoirs, etc., while property objects may be used to provide property values as well as data versions and display parameters. For example, an entity object may represent a well where a property object provides log information as well as version information and display information (e.g., to display the well as part of a model).

In the example of <FIG>, data may be stored in one or more data sources (or data stores, generally physical data storage devices), which may be at the same or different physical sites and accessible via one or more networks. The model simulation layer <NUM> may be configured to model projects. As such, a particular project may be stored where stored project information may include inputs, models, results, and cases. Thus, upon completion of a modeling session, a user may store a project. At a later time, the project can be accessed and restored using the model simulation layer <NUM>, which can recreate instances of the relevant domain objects.

In the example of <FIG>, the geologic environment <NUM> may include layers (e.g., stratification) that include a reservoir <NUM> and one or more other features such as the fault <NUM>-<NUM>, the geobody <NUM>-<NUM>, etc. As an example, the geologic environment <NUM> may be outfitted with any of a variety of sensors, detectors, actuators, etc. For example, equipment <NUM> may include communication circuitry to receive and to transmit information with respect to one or more networks <NUM>. Such information may include information associated with downhole equipment <NUM>, which may be equipment to acquire information, to assist with resource recovery, etc. Other equipment <NUM> may be located remote from a well site and include sensing, detecting, emitting or other circuitry. Such equipment may include storage and communication circuitry to store and to communicate data, instructions, etc. As an example, one or more satellites may be provided for purposes of communications, data acquisition, etc. For example, <FIG> shows a satellite in communication with the network <NUM> that may be configured for communications, noting that the satellite may additionally or instead include circuitry for imagery (e.g., spatial, spectral, temporal, radiometric, etc.).

<FIG> also shows the geologic environment <NUM> as optionally including equipment <NUM> and <NUM> associated with a well that includes a substantially horizontal portion that may intersect with one or more fractures <NUM>. For example, consider a well in a shale formation that may include natural fractures, artificial fractures (e.g., hydraulic fractures), or a combination of natural and artificial fractures. As an example, a well may be drilled for a reservoir that is laterally extensive. In such an example, lateral variations in properties, stresses, etc. may exist where an assessment of such variations may assist with planning, operations, etc. to develop a laterally extensive reservoir (e.g., via fracturing, injecting, extracting, etc.). As an example, the equipment <NUM> and/or <NUM> may include components, a system, systems, etc. for fracturing, seismic sensing, analysis of seismic data, assessment of one or more fractures, etc..

As mentioned, the system <NUM> may be used to perform one or more workflows. A workflow may be a process that includes a number of worksteps. A workstep may operate on data, for example, to create new data, to update existing data, etc. As an example, a may operate on one or more inputs and create one or more results, for example, based on one or more algorithms. As an example, a system may include a workflow editor for creation, editing, executing, etc. of a workflow. In such an example, the workflow editor may provide for selection of one or more pre-defined worksteps, one or more customized worksteps, etc. As an example, a workflow may be a workflow implementable in the PETREL® software, for example, that operates on seismic data, seismic attribute(s), etc. As an example, a workflow may be a process implementable in the OCEAN® framework. As an example, a workflow may include one or more worksteps that access a module such as a plug-in (e.g., external executable code, etc.).

<FIG> illustrates a conceptual view of a system <NUM> for calculating risk of failure in a wellbore environment, e.g., during or prior to drilling, according to an embodiment. The system <NUM> may include a working environment <NUM> that includes a pool of working agents <NUM>, e.g., homogenous agents and heterogeneous agents. In this context, an "environment" is an algorithmic component in a reinforcement learning framework. It includes a simulator (or real plant, e.g., an actual field system) where an action may be applied, and a reward system for evaluating the response to this action. The working environment <NUM> may also include a representation of the current drilling operations, e.g., rig state, geology, bit location (e.g., with respect to the planned trajectory), etc. The system <NUM> may also include a validation environment <NUM> having one or more validation agents <NUM>. It will be appreciated that "agents" refers to computer-implemented software and/or hardware or parts thereof.

In some embodiments, the system <NUM> may calculate risk of failure for an action. The action may be proposed by the working agents <NUM> in the working environment <NUM>, and the risk may be calculated in the validation environment <NUM>. In some embodiments, the risk may be calculated using a DQN to evaluate the following relationship:
<MAT>.

<FIG> illustrates a flowchart of a method <NUM> for calculating risk of failure while drilling a well, according to an embodiment. The method <NUM> may include receiving an observable from the working environment, as at <NUM>. An observable may be a state observed from an environment, such as a survey point during drilling. The observable event may be the survey carried out in the field. In other embodiments, observables may be anything that may call for a drilling action in response.

As noted above, the working environment <NUM> may include multiple working agents <NUM>. Each working agent <NUM> may be employed by the method <NUM> to generate a proposed action based on (e.g., in response to) the observable, as at <NUM>. Proposed actions may include adjustments to toolface settings, sliding ratios, and/or other drilling parameters. The method <NUM> may then include synchronizing the validation environment <NUM> with the working environment <NUM>, so that the validation environment <NUM> accurately represents the current state of the drilling environment <NUM>, e.g., the position, operating parameters, and/or state of drilling equipment, the formation properties, etc..

The method <NUM> may then include stepping the proposed action with a simulator, in the validation environment <NUM>, so as to yield a new observable, as at <NUM>. Thus, after stepping the simulation, the validation environment <NUM> represents the drilling environment in a hypothetical case in which the proposed action has been implemented. The validation agent <NUM> may then execute action decisions on the new observable in the validation environment <NUM>, as at <NUM>. The validation environment <NUM> may then step the action decided upon by the validation agent <NUM> in the simulator, as at <NUM>. The worksteps of proposing action, making decisions, and stepping in the simulator may then be repeated, e.g., until the validation agent <NUM> finishes a drilling analysis, e.g., until the validation environment, executing the different steps, reaches a target location. From this analysis, the method <NUM> may include calculating a reward using the validation agent <NUM>. The preceding aspects may then be repeated for the remaining working agents and the actions proposed by these other working agents, if any, as indicated in <FIG>.

The method <NUM> may then select an action proposed by one of the working agents <NUM> based on the reward, as at <NUM>. For example, the method <NUM> may include selecting a proposed action that yields the maximum reward (R) according to the drilling analysis performed by the validation agent. The action that is selected may then be returned to the working environment <NUM>, as at <NUM>. For example, a drilling rig may be adjusted to implement the action. The process of acquiring working agent decision steps for a remainder of the drilling may then repeat, based on the new observation obtained. This may repeat, e.g., throughout the drilling process.

<FIG> illustrates an example of an architecture for a system <NUM> implementing the method <NUM>, according to an embodiment. As shown, the working agents <NUM> and working environment <NUM> may be part of an "operation" side of the architecture, while a validation projection side of the architecture may include the validation environment <NUM> and the validation agent <NUM>.

The working agents <NUM>, working environment <NUM>, validation environment <NUM>, and validation agent <NUM> interact. In particular, the working agents <NUM> may apply actions to the working environment <NUM>, and receive perceptions (e.g., sensor measurements) therefrom, e.g., a state of the working environment <NUM>. The working environment <NUM> may synchronize with the validation environment <NUM>. The validation environment <NUM> may evaluate actions by way of simulation through the validation agent <NUM>, which may provide results of the simulation back to the validation environment <NUM>. Further, the working agents <NUM> may provide action proposal sensory synchronization to the validation agent <NUM>, which may provide action selections back to the working agents <NUM>. The action may then be fed to the working environment <NUM>, which may, for example, cause a drilling rig to implement the selected drilling action.

Embodiments of the method <NUM> can implement the validation agent <NUM> to run multiple times, potentially in different configurations, for a single action proposed by one or more working agents <NUM>. For example, the validation agent <NUM> can be configured to prioritize efficiency in the drilling process, or minimization of risk, to name just two examples of different possible configurations for the validation agent <NUM>. Further, in at least some examples, two or more different (and differently configured) validation agents <NUM> may be provided and may be used to perform the drilling analysis separately, e.g., in parallel, to generate a reward associated with an action proposed by one or more of the working agents <NUM>. In some embodiments, the highest total reward may be used, but in others, the average total reward or lowest total reward may be used. The reward calculated, e.g., in one of these ways, may then be compared with the rewards, calculated the same or similarly, for other proposed actions, thereby permitting the machine-generated proposed actions to be quantitatively compared and automatically selected.

Further, by providing interaction between working agents <NUM> and validation agents <NUM>, passing proposals and selections back and forth, more stable decision making may result, because agreement between working agent <NUM> and validation agent <NUM> may prevent irrational choices by either. Additionally, the validation agent <NUM> may present an empirical evaluation through simulation.

<FIG> illustrates an example of the interaction between the system <NUM> and a human user <NUM>, according to an embodiment. As shown, the user <NUM> may have input into the working agents <NUM>, the action selection, and the validation environment <NUM>. For example, the user <NUM> may provide an action proposal, which may be added to the actions proposed by the working agents <NUM>, and may compete therewith in the drilling analysis conducted by the validation agent <NUM>. The user may also override the action selected by the validation agent <NUM>. The user may also control the formation and/or any other characteristic of the validation environment, e.g., based on offset well data and/or experience. The validation agent <NUM> may also provide feedback and alarm signals (e.g., failure of the drilling actions proposed by the working agents <NUM> to produce a viable well) to the user <NUM>, so the user <NUM> may take mitigating actions. As such, the system <NUM> may provide environmental detection, etc., and may be able to perform most tasks autonomously, but human feedback/override may still be available.

<FIG> illustrates another method <NUM> for drilling, according to an embodiment. As with the method <NUM> of <FIG>, the present embodiment may implement a bifurcated architecture in which working agents may make proposals for drilling actions and one or more validation agents may evaluate these proposals in a validation environment. Moreover, the method <NUM> may be subject to human intervention and/or override, as will be described.

As shown, the method <NUM> may include receiving an observable event and/or data in a working environment, as at <NUM>. For example, the observable may include one or more sensor measurements representing, for example, a position of a drill bit in the earth, and a comparison thereof to a planned trajectory.

The method <NUM> may then include proposing a drilling action using potentially several (e.g., <NUM>, <NUM>,. , N) working agents, as at 604A, 604B. Each working agent may be configured to interpret data differently, e.g., may be tuned for different types of environments, may implement rules-based algorithms, different machine-learning models (e.g., of different types or trained using different data specific to different situations).

In some embodiments, the method <NUM> may also include receiving a drilling action proposed by a human user, as at 604C. This drilling action, which may be based on intuition, field experience, etc. may be used as a competitive drilling action proposal, and may be evaluated alongside the machine-generated proposals.

One or more validation agents may then simulate drilling responses for the actions proposed by the working agents, as at <NUM>. In some embodiments, one validation agent may evaluate the drilling proposals from each of the working agents. In other embodiments, different validation agents may evaluate drilling actions from different working agents. For example, a different validation agent may be used for each different working agent, or there may be overlap between the drilling proposals evaluated by the different validation agents. Thus, any combination of working agents and validation agents may be provided. In a specific embodiment, several validation agents may be used, e.g., tuned to prioritize different goals, e.g., one may be tuned for efficiency, another for speed, another for risk, another for maintaining strict adherence to a planned trajectory, etc..

In some embodiments, the validation environment, e.g., drilling parameters, geological characteristics of the subsurface domain, etc., may be modified by a user before or during the simulating at <NUM>. For example, the method <NUM> may include receiving a modification to the validation environment from a human user, as at <NUM>. The validation agents may then use the modified validation environment in order to evaluate the proposed drilling actions, which may or may not include a human-proposed drilling action.

Using the validation agent(s), the method <NUM> may include determining a reward for each of the proposed drilling actions based on the drilling responses calculated using the validation environment, as at <NUM>. For example, the validation agents may simulate a drilling scenario for each of the proposed actions, determining the risk of each resulting in failure, the efficiency in the drilling process, etc. The calculation of the drilling scenario may be accomplished solely by the validation agents running through the simulation of the entire scenario, or could be accomplished by recursively pushing incremental drilling responses back to the working environment, for the working agents to then propose next actions, until the end of the drilling scenario is reached. In an embodiment, the working agents propose a set of actions, and each action is individually evaluated by the validation agent in the validation environments. When the validation agent evaluate, a projected total future reward score, or estimated total reward (ETR), is calculated for each action. The final action is the one with the maximum ETR.

In some embodiments, the reward may be a quantification of a risk of failure for a given action, e.g., the DQN equation provided above. In other embodiments, other quantifications of a reward for a proposed action may be employed. The rewards may be different as calculated between different validation agents, and thus may be combined, e.g., using an average, taking a minimum/maximum, or using any other statistical method.

The method <NUM> may again account for human intervention, e.g., in the form of an override. Accordingly, at <NUM>, the method <NUM> may provide an opportunity for a manual override of the proposed actions, e.g., in which a human operate selects an action notwithstanding, or at least not strictly adhering to, the rewards calculation by the validation agents. If an override is received (<NUM>: Yes), the override is selected, as at <NUM>. Otherwise (<NUM>: No), the method <NUM> may include selecting one of the proposed drilling actions (either a computer-generated or user-entered action) based on the calculated reward, as at <NUM>. The method <NUM> may then feed the selected drilling action back to the working agent, which may cause the working environment to be manipulated, e.g., by causing the drilling rig to execute the selected action, as at <NUM>.

In some embodiments, the methods of the present disclosure may be executed by a computing system. <FIG> illustrates an example of such a computing system <NUM>, in accordance with some embodiments. The computing system <NUM> may include a computer or computer system 701A, which may be an individual computer system 701A or an arrangement of distributed computer systems. The computer system 701A includes one or more analysis modules <NUM> that are configured to perform various tasks according to some embodiments, such as one or more methods disclosed herein. To perform these various tasks, the analysis module <NUM> executes independently, or in coordination with, one or more processors <NUM>, which is (or are) connected to one or more storage media <NUM>. The processor(s) <NUM> is (or are) also connected to a network interface <NUM> to allow the computer system 701A to communicate over a data network <NUM> with one or more additional computer systems and/or computing systems, such as 701B, 701C, and/or 701D (note that computer systems 701B, 701C and/or 701D may or may not share the same architecture as computer system 701A, and may be located in different physical locations, e.g., computer systems 701A and 701B may be located in a processing facility, while in communication with one or more computer systems such as 701C and/or 701D that are located in one or more data centers, and/or located in varying countries on different continents).

A processor may include a microprocessor, microcontroller, processor module or subsystem, programmable integrated circuit, programmable gate array, or another control or computing device.

The storage media <NUM> may be implemented as one or more computer-readable or machine-readable storage media. Note that while in the example embodiment of <FIG> storage media <NUM> is depicted as within computer system 701A, in some embodiments, storage media <NUM> may be distributed within and/or across multiple internal and/or external enclosures of computing system 701A and/or additional computing systems. Storage media <NUM> may include one or more different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories, magnetic disks such as fixed, floppy and removable disks, other magnetic media including tape, optical media such as compact disks (CDs) or digital video disks (DVDs), BLURAY® disks, or other types of optical storage, or other types of storage devices. Note that the instructions discussed above may be provided on one computer-readable or machine-readable storage medium, or may be provided on multiple computer-readable or machine-readable storage media distributed in a large system having possibly plural nodes. Such computer-readable or machine-readable storage medium or media is (are) considered to be part of an article (or article of manufacture). An article or article of manufacture may refer to any manufactured single component or multiple components. The storage medium or media may be located either in the machine running the machine-readable instructions, or located at a remote site from which machine-readable instructions may be downloaded over a network for execution.

In some embodiments, computing system <NUM> contains one or more action assessment module(s) <NUM>. In the example of computing system <NUM>, computer system 701A includes the action assessment module <NUM>. In some embodiments, a single action assessment module may be used to perform some aspects of one or more embodiments of the methods disclosed herein. In other embodiments, a plurality of action assessment modules may be used to perform some aspects of methods herein.

It should be appreciated that computing system <NUM> is merely one example of a computing system, and that computing system <NUM> may have more or fewer components than shown, may combine additional components not depicted in the example embodiment of <FIG>, and/or computing system <NUM> may have a different configuration or arrangement of the components depicted in <FIG>. The various components shown in <FIG> may be implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing and/or application specific integrated circuits.

These modules, combinations of these modules, and/or their combination with general hardware are included within the scope of the present disclosure.

Claim 1:
A method for drilling a well, comprising:
generating a plurality of proposed drilling actions (604a, 604b) using a plurality of working agents based on a working environment;
simulating drilling responses to the proposed drilling actions using a plurality of validation agents in a validation environment that initially represents the working environment (<NUM>);
determining rewards for the proposed drilling actions based on the simulating, using the validation agents (<NUM>);
selecting one of the proposed drilling actions (<NUM>); and
causing a drilling rig to execute the selected one of the proposed actions (<NUM>).