MULTI-AGENT AUTONOMOUS INSTRUCTION GENERATION FOR MANUFACTURING

Solutions are provided for multi-agent autonomous instruction generation for manufacturing. An example includes: generating, for a plurality of actor agents, a first set of instructions for performing manufacturing tasks, wherein the actor agents include a human actor accessing a user interface (UI), an autonomous actor having a first sensor, a semi-autonomous actor having a second sensor, and a non-autonomous actor having a third sensor; receiving, by a control agent from at least the plurality of actor agents, observation data regarding performance of the actor agents on the manufacturing tasks, wherein the control agent comprises an autoregressive bidirectional long-term short-term memory (LSTM) attention network; and based at least on the instructions and the observation data, generating further instructions for performing manufacturing tasks. The instructions include at least one of a role assignment, platform control, tool selection, and tool utilization.

BACKGROUND

Manufacturing of aircraft and other complex systems requires a significant amount of human touch (manual) labor, robotic-assisted manufacturing process, and coordination among the different processes. In some examples, rules-based or expert knowledge systems are used for control, including proportional-integral-derivative (PID) controllers that seek to minimize differences between observed and expected values. Even if autonomous subsystems are used, it is in a piecemeal fashion. Existing solutions typically involve high cost; require manual oversight of even autonomous vehicles, robots, and process flow; demonstrate limited flexibility in new environments and circumstances not explicitly programmed into the original design; and does do not adapt well to unplanned processes and procedures, such as the production of one-off parts or assemblies.

SUMMARY

The disclosed examples are described in detail below with reference to the accompanying drawing figures listed below. The following summary is provided to illustrate examples disclosed herein. It is not meant, however, to limit all examples to any particular configuration or sequence of operations.

Solutions are provided for multi-agent autonomous instruction generation for manufacturing. An example includes: generating, for a plurality of actor agents, a first set of instructions for performing manufacturing tasks, wherein the actor agents include a human actor accessing a user interface (UI), an autonomous actor having a first sensor, a semi-autonomous actor having a second sensor, and a non-autonomous actor having a third sensor; receiving, by a control agent from at least the plurality of actor agents, observation data regarding performance of the actor agents on the manufacturing tasks, wherein the control agent comprises an autoregressive bidirectional long-term short-term memory (LSTM) attention network; and based at least on the instructions and the observation data, generating further instructions for performing manufacturing tasks. The instructions include at least one of a role assignment, platform control, tool selection, and tool utilization.

The features, functions, and advantages that have been discussed are achieved independently in various examples or are to be combined in yet other examples, further details of which are seen with reference to the following description and drawings.

DETAILED DESCRIPTION

The various examples will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. References made throughout this disclosure relating to specific examples and implementations are provided solely for illustrative purposes but, unless indicated to the contrary, are not meant to limit all examples.

A centralized artificial intelligence (AI) or machine learning (ML) control agent is capable of collaboratively controlling multiple (a plurality of) actor agents and sensor agents to increase automation and human-machine collaboration within a manufacturing environment. The control agent learns to attend to relevant sources of situational awareness derived from various autonomous, semi-autonomous, non-autonomous, Internet of Things (IoT) devices, and human systems in order to intelligently command and control various platforms. These platforms may be robotic armatures, ground based vehicles, gimbaled measurement or vision systems and aerial vehicles (e.g., UAVs), connected via various IoT devices.

Aspects of the disclosure are able to advantageously automate manufacturing procedures and concepts operations through a centralized learning system and associated training and utilization processes through which the collective knowledge of various autonomous, semi-autonomous, non-autonomous, and human systems is collected through observations and is aggregated and utilized to intelligently inform commands issued to controllable vehicles, robots, and IoT devices. Aspects of the disclosure present novel solutions that reduce the cost and labor required to assemble and fabricate equipment and structures within a manufacturing environment.

Aspects of the disclosure present novel solutions that construct a superior situational awareness enabling intelligent autonomous control of large numbers of agents. The range of control may extend from a portion of a factory to spanning multiple factories. Actors may be controlled via roles, sub-roles, platform control, tool selection, tool utilization, and others, any of which may be masked. Lower level actions of actors are conditioned on higher level actions of actors. The queue for tasks or jobs may vary in length, as necessary.

Inputs include observations from the manufacturing environment and prior outputs. Observations (passive and interactive) are input and transformed by AI into control commands. The control agent (which may be local or remote) receives information from various devices it controls or can receive information from or provide feedback to a human via a user interface (UI). Some agents, (e.g., humans) may mask certain jobs, for example limiting the available actions for a given actor. For example, a human may mask an operation requiring the use of a particular tool if the human notices that the tool is not functioning properly.

An autoregressive, temporal and attention-based encoder-decoder deep reinforcement learning system includes a sequenced bi-directional long short-term memory (LSTM) based encoder, encoder-to-decoder attention network, and an LSTM based decoder. The decoder LSTM unrolls at each time-step per each controllable agent receiving an attention-based context vector, action masks (dictating legal actions) and other sources as input. The decoder is multi-head auto-regressive, such that actions (per each time step) may be conditioned on prior actions determined (within the same time-step). Deep reinforcement learning and system-level interaction among different data feeds, control units, and interfaces, provide an ability to improve performance through automated and human-assisted learning. AI learns from rewards, based on controlling actors to accomplish tasks properly. Rewards may result from feedback on results that correlate with control actions (instructions).

Aspects and implementations disclosed herein are directed to solutions for multi-agent autonomous instruction generation for manufacturing. An example includes: generating, for a plurality of actor agents, a first set of instructions for performing manufacturing tasks, wherein the actor agents include a human actor accessing a UI, an autonomous actor having a first sensor, a semi-autonomous actor having a second sensor, and a non-autonomous actor having a third sensor; receiving, by a control agent from at least the plurality of actor agents, observation data regarding performance of the actor agents on the manufacturing tasks, wherein the control agent comprises an autoregressive bidirectional LSTM attention network; and based at least on the instructions and the observation data, generating further instructions for performing manufacturing tasks. The instructions include at least one of a role assignment, platform control, tool selection, and tool utilization.

Referring more particularly to the drawings,FIG. 1illustrates an arrangement100that advantageously employs multi-agent instruction generation, for example, multi-agent autonomous instruction generation for manufacturing. The arrangement100includes a control agent200that issues instructions170to and receives observations180a plurality of agents110,120,130,140, and150in a manufacturing environment101, to manufacture a workpiece102. The control agent200is described in greater detail in relation toFIG. 2. In one embodiment, the arrangement100is employed during production, component and subassembly manufacturing1006during pre-production, the apparatus manufacturing and service method1000, as illustrated inFIG. 10. In one embodiment, the workpiece102becomes an apparatus1100(or a component), as shown inFIG. 11, for example a flying apparatus1101as shown inFIG. 12. Plurality of agents110,120,130,140, and150includes a plurality of actor agents110,120,130, and140and also a sensor agent150.

The human actor agent110includes a human actor112accessing a UI114. The autonomous actor agent120includes an autonomous actor122and a first sensor124that is able to collect sensor data regarding the operations or results of operations of at least the autonomous actor122, and a tool cache126. The semi-autonomous actor agent130includes a semi-autonomous actor132and a second sensor134that is able to collect sensor data regarding the operations or results of operations of at least the semi-autonomous actor132, and a tool cache136. The non-autonomous actor agent140includes a non-autonomous actor142and a third sensor144that is able to collect sensor data regarding the operations or results of operations of at least the non-autonomous actor142, and a tool cache146. The sensor agent150includes a fourth sensor154that is able to collect sensor data regarding aspects of the manufacturing environment101within its range. The collected sensor data becomes observations180(shown in three discrete time steps as first observation data181, second observation data182, and third observation data183) and is received, by the control agent200over a communication component104.

In one example, the plurality of agents110,120,130,140, and150perform manufacturing tasks that may include any of: video capture, measurement, painting, soldering, welding, moving, cutting, drilling, puncturing, and hammering. It should be understood that, although only one actor of each type (human, autonomous, semi-autonomous, non-autonomous, and sensor) is illustrated, a different number of each type of actor may be used in some examples. The manufacturing tasks are controlled using instructions170(shown in three discrete time steps as first set of instructions171, second set of instructions172, and third set of instructions173) that are generated by the control agent200and sent over the communication component104to the plurality of agents110,120,130,140, and150. In one example, the instructions170include at least one of a role assignment, platform control, tool selection, and tool utilization.

The human actor112may both receive instructions170from and provide observations180to the control agent200via the UI114. In one example, the UI114is implemented using a computing device900ofFIG. 9. For the autonomous actor agent120, the autonomous actor122, the semi-autonomous actor132, and the non-autonomous actor142each follows relevant ones of the instructions170, which may include the selection and utilization of tools from a respective one of tool caches126,136, and146.

A training component106provides pre-deployment training and/or supplemental training for the control agent200, using training data108. In one example, the training data108is synthetic training data. Pre-deployment training may be used to get the control agent200up to a minimum level of capability before being used for operations. The training data108may simulate, for example, multiple rounds of the instructions170and the observations180, along with rewards404that are leveraged for ongoing performance improvement of the control agent200. Performance improvement of the control agent200using the rewards404is described in further detail in relation toFIGS. 4 and 5.

A data store190is coupled to the control agent200and/or the communication component104and stores at least the instructions170and the observations180. This permits the control agent200to use actual historic data, including outgoing control commands (the instructions170) and performance results (embedded within the observations180) for ongoing learning and improvement. Additionally, the actual historic data stored in the data store190may become a version of training data108for another example of the control agent200that will be deployed to another manufacturing environment101.

Referring now toFIG. 2, the control agent200is shown to comprise multiple portions, including an encoder portion210comprising a plurality of input-specific LSTMs214a-214cand an attention network216, and a decoder portion220comprising a decoder LSTM222. Together, the encoder portion210and the decoder portion220form a policy network240. An LSTM is an artificial recurrent neural network (RNN) architecture used for deep learning applications, and leverages feedback connections to process sequences of data. LSTM units are units of an RNN; an RNN composed of LSTM units is often referred to as an LSTM network. In one example, the policy network240includes an autoregressive bidirectional RNN attention network, and the control agent may thus include an autoregressive bidirectional LSTM attention network. An autoregressive (AR) network predicts future behavior based on past behavior and observations. The control agent200may control multiple actions simultaneously, with some actions depending on others. An attention network is a neural attention mechanism that equip a neural network with the ability to focus on a subset of its inputs (or features).

The incoming observations180are transformed by a respective one of transform212a, transform212b, and transform212c. In one embodiment, this includes scaling measurements from a minimum and maximum of allowable input values to a range of −1 to +1. The input-specific LSTMs (LSTM214a, LSTM214b, and LSTM214c) each receives at least a portion of the (transformed) observations180. The attention network216and the decoder LSTM222receive at least a portion of a set of commands. The attention network216enables the control agent200to focus on a subset of LSTMs214a-214c. In one example, some of the observations180are passed directly to the decoder LSTM222. Thus, the decoder LSTM222receives at least a portion of the observations180(observation data) from the input-specific LSTMs214a-214cand the attention network216.

An autoregressive output230takes the output from the decoder LSTM222, observations180, and actor masks232to generate the instructions170that are routed (via the communication component104) to the proper ones of agents110,120,130,140, and150. In one example, the actor masks232include preconfigured constraints that prevent an agent from performing a task (e.g., the agent is out of supplies or has some malfunction). The actor masks232thus constrain the instructions170.

For example a set of input to the control agent200may be represented as:

Primary objectives (current or allowable jobs);

Secondary objectives (current or allowable jobs);

Masks for legal actions (provided by humans or logic-system);

List of job priorities;

Task Specific Descriptions:

Human or logic based system provided;

Role and sub-role;

Control and sub-control:

Task Specific Descriptions;

Task Ids to Suggested Actor Agents;

For example, a human may recognize attributes of the manufacturing environment101that need to be addressed by area, issues, and/or jobs. The control agent200will queue them and handle them as it has capacity.

The auto-regressive nature of the output of the control agent200may be broken down into agent role and sub-roles and agent control and sub-control. Agent sub-roles are conditioned on selected or provided roles. Agent control is conditioned on the sub-role and subsequent controls that are provided. An example is:

Role Actions: Agent Role (Embeddings/Ids)

Sub-role: Video Capture

Control Actions: Agent Control

Control: Platform Control

Target Selection: Attention to target (Points to target)—item to lift,

Drill Type (Power) and Bit

Solder

Brush

Hammer type

Strength of utilization (constrained between min and max power of tool)

Pseudocode is provided below for training the control agent, for example by the training component106ofFIG. 1, and during operations of the manufacturing environment101. For training:

For operations:

FIG. 3illustrates some step-wise operational details for the control agent200.FIG. 2was described according to a specific moment in time. As time progresses, and the workpiece102develops incrementally, the control agent200cycles through subsequent versions of the instructions170and the resulting observations180. The control agent200does not merely generate the instructions170based on the observations180, but also uses at least the prior cycle of instructions170(or more than only just the immediately prior cycle). Thus, as illustrated inFIG. 3, the decoder LSTM222and the attention network216both receive, at time step t, the output of the decoder LSTM222from time step (t−1). Similarly, the decoder LSTM222and the attention network216both receive, at time step (t+1), the output of the decoder LSTM222from time step t.

FIG. 4illustrates an operational loop400for performance improvement of the control agent200. In the operational loop, the control agent200receives the observations180and the rewards404, which are used to score the prior cycle of the instructions170. The control agent200learns from the observations180, based on the rewards404that indicate whether the assigned tasks had been accomplished properly, and produces the next cycle of the instructions170. The rewards404result from feedback on results (e.g., the observations180) that correlates with control actions (e.g., the instructions170).

FIG. 5illustrates a learning structure500that may be used to improve performance of the control agent200in the operational loop400. Gradient based optimization techniques utilize reinforcement learning (reward heuristics) for an actor and critic network topology in which the actor provides the actions and the critic assesses those actions for each autonomous agent by providing a value assessment for the quality of the action. In one example, the gradient optimization technique utilizes natural gradient techniques and stochastic gradient descent mechanisms.

Performance improvement of the control agent200may be viewed as a multi-variant optimization function. Some of the parameters to be considered include time to completion, completion of tasks according to priority, successful completion of a task (e.g., for each agent), completion of high level objectives (e.g., aggregations of tasks) which may require the successful coordination of all or some of the agents110,120,130,140, and150autonomous agents controlled by the control agent200.

As illustrated, the instructions170are generated by the control agent200as described above. In parallel, a value assessment504is also generated by a value network502. The value network502is similar to the control agent200, although not the same. A decoder LSTM522intakes the actor actions in addition to observations180and outputs from the attention network216. An autoregressive value output530intakes the outputs from the decoder LSTM522, actor masks232, observations180, and optionally the actor actions, and outputs the value assessment504rather than the instructions170. In one example, the value network502outputs a real valued number scaled between −1 and +1 (e.g., scaled from an initial calculation range that may be wider). The decoder LSTM522may be similar to the decoder LSTM322if policy network outputs are not provided at this level, or may be different in some examples.

The rewards404are derived using the value assessment504and the observations180. The rewards404may also be derived from software logic or informed by a human. Various calculations may be used in the derivation of the rewards404and the ongoing learning by the control agent200. These may include calculation of a discounted cumulative reward, Gt(at time t), from performance results:

where γ is a discount factor, and R is a reward function.

The value loss to minimize, Lvalue(w), is given by:

where s is the state, and Vwis the performance estimate by the control agent200.

An advantage, At, is a derived reward, the difference between actual and believed performance (e.g., how much better or worse the control agent200did than it believed to be the case), and is given by:

The policy loss to minimize, Lpolicy, is used for backpropagation in neural network training:

where π is a probability function and a is the actual result (outcome). The gap between self-assessment and environment feedback should decrease over time.

FIG. 6illustrates additional operational details for the control agent200. As indicated, the autoregressive output230is unrolled for each actor. That is, the decoder LSTM222receives input602(e.g., from the encoder portion210ofFIG. 2), which may be represented as a set of inputs602aand602b(plus others for other agents), while the decoder LSTM222may be represented as a set of LSTMs622aand622b(plus others for other agents). LSTM622aoutputs output604afor agent A (one of agents110,120,130,140, and150), LSTM622boutputs output604bfor agent B, and this structure is continued for the remaining ones of agents110,120,130,140, and150. The set of outputs604a,604b, and others form at least a portion of the instructions170. Another input606ais provided to output604a, and an input608is used to generate the actor masks232. The proper one of the actor masks232is also applied to the output604a. The output604ais illustrated as having multiple components: a role610, a sub-role612, a platform control614, a tool selection616, and a tool utilization618. Other outputs to other agents may be similar or may differ, based on the nature of the specific agent receiving the output.

With reference now toFIG. 7, a flow chart700illustrates a method of multi-agent instruction generation, as may be used with the arrangement100, for example during production, component and subassembly manufacturing1006ofFIG. 10. In one example, the operations illustrated inFIG. 7are performed, at least in part, by executing instructions902a(stored in the memory902) by the one or more processors904of the computing device900ofFIG. 9. Operation702includes conducting initial training of the control agent200. In one example, operation702includes training the control agent200with synthetic training data108. For operation702, training the control agent200with synthetic training data108occurs prior to generating the first set of instructions171. The control agent200is deployed in operation704.

Operation706includes receiving, by the control agent200from at least the plurality of actor agents110,120,130,140, and150, the observation data181regarding performance of the actor agents110,120,130,140, and150on the first set of manufacturing tasks, wherein the control agent200comprises an autoregressive bidirectional LSTM attention network. In one example, the receiving observation data comprises also receiving the observation data181from the sensor agent150comprising the fourth sensor154. In one example, the control agent200comprises: the encoder portion210comprising the plurality of input-specific LSTMs214a-214cand the attention network216; and the decoder portion220comprising the decoder LSTM222. In one example, the input-specific LSTMs214a-214creceive at least a portion of the observation data181, and wherein the attention network216and the decoder LSTM222receive at least a portion of the first set of instructions171.

Operation708includes applying the actor masks232, which will result in constraining the instructions171using the actor masks232, when operation708occurs. Operation710includes generating, for the plurality of actor agents110,120,130,140, and150, the first set of instructions171for performing a first set of manufacturing tasks, wherein the actor agents110,120,130,140, and150include at least one actor agent selected from the list consisting of: the human actor112accessing the UI114, the autonomous actor122having the first sensor124, the semi-autonomous actor132having the second sensor134, and the non-autonomous actor142having the third sensor144. In one example, the manufacturing tasks include at least one task selected from the list consisting of: video capture, measurement, painting, soldering, welding, moving, cutting, drilling, puncturing, and hammering.

The plurality of actor agents110,120,130,140, and150then perform their assigned manufacturing tasks in accordance with the instructions171in operation712. Rewards are generated and provided as feedback in operation714, to provide ongoing performance improvements for the control agent200in operation716. A decision operation718determines whether the control agent200would benefit from further training. If so, operation720includes conducting further training of the control agent200. In one example, operation720includes training the control agent200with synthetic training data108. For operation720, training the control agent200with synthetic training data108occurs after generating the first set of instructions171.

A decision operation722determines whether operations of the control agent200remain ongoing. If so, the flow chart700returns to operation706. However, this second time, operation708includes updating the actor masks232, which will result in constraining the instructions172using the actor masks232. In the second pass, operation706includes receiving, by the control agent200from at least the plurality of actor agents110,120,130,140, and150(and, in one example, also the sensor agent150) the second observation data182regarding performance of the actor agents110,120,130,140, and150on the second set of manufacturing tasks. Also in the second pass, operation710includes, based at least on the first set of instructions171and the observation data181, generating, for the plurality of actor agents110,120,130,140, and150, the second set of instructions172for performing a second set of manufacturing tasks. In further iterations, operation710includes, based at least on the second set of instructions172and the second observation data182, generating, by the control agent200for the plurality of actor agents110,120,130,140, and150, the third set of instructions173for performing a third set of manufacturing tasks. This use of prior instructions and observations in the generation of subsequent instructions continues as the control agent200continually improves.

FIG. 8shows a flow chart800also illustrating a method of multi-agent instruction generation that may, for example, be part of production, component and subassembly manufacturing1006ofFIG. 10. In one example, operations illustrated inFIG. 8are performed, at least in part, by executing instructions by the one or more processors904of the computing device900ofFIG. 9. In one example, operation802includes generating, for a plurality of actor agents, a first set of instructions for performing a first set of manufacturing tasks, wherein the actor agents include at least one actor agent selected from the list consisting of: a human actor accessing a UI, an autonomous actor having a first sensor, a semi-autonomous actor having a second sensor, and a non-autonomous actor having a third sensor.

Operation804includes receiving, by a control agent from at least the plurality of actor agents, observation data regarding performance of the actor agents on the first set of manufacturing tasks, wherein the control agent comprises an autoregressive bidirectional LSTM attention network. Operation806includes, based at least on the first set of instructions and the observation data, generating, by the control agent for the plurality of actor agents, a second set of instructions for performing a second set of manufacturing tasks, wherein the first set of instructions and the second set of instructions each includes at least one instruction selected from the list consisting of: a role assignment, a platform control, a tool selection, and a tool utilization.

With reference now toFIG. 9, a block diagram of the computing device900suitable for implementing various aspects of the disclosure is described. In some examples, the computing device900includes one or more processors904, one or more presentation components906and the memory902. The disclosed examples associated with the computing device900are practiced by a variety of computing devices, including personal computers, laptops, smart phones, mobile tablets, hand-held devices, consumer electronics, specialty computing devices, etc. Distinction is not made between such categories as “workstation,” “server,” “laptop,” “hand-held device,” etc., as all are contemplated within the scope ofFIG. 9and the references herein to a “computing device.” The disclosed examples are also practiced in distributed computing environments, where tasks are performed by remote-processing devices that are linked through a communications network. Further, while the computing device900is depicted as a seemingly single device, in one example, multiple computing devices work together and share the depicted device resources. For instance, in one example, the memory902is distributed across multiple devices, the processor(s)904provided are housed on different devices, and so on.

In one example, the memory902includes any of the computer-readable media discussed herein. In one example, the memory902is used to store and access instructions902aconfigured to carry out the various operations disclosed herein. In some examples, the memory902includes computer storage media in the form of volatile and/or nonvolatile memory, removable or non-removable memory, data disks in virtual environments, or a combination thereof. In one example, the processor(s)904includes any quantity of processing units that read data from various entities, such as the memory902or input/output (I/O) components910. Specifically, the processor(s)904are programmed to execute computer-executable instructions for implementing aspects of the disclosure. In one example, the instructions are performed by the processor, by multiple processors within the computing device900, or by a processor external to the computing device900. In some examples, the processor(s)904are programmed to execute instructions such as those illustrated in the flow charts discussed below and depicted in the accompanying drawings.

The presentation component(s)906present data indications to an operator or to another device. In one example, presentation components906include a display device, speaker, printing component, vibrating component, etc. One skilled in the art will understand and appreciate that computer data is presented in a number of ways, such as visually in a graphical user interface (GUI), audibly through speakers, wirelessly between the computing device900, across a wired connection, or in other ways. In one example, presentation component(s)906are not used when processes and operations are sufficiently automated that a need for human interaction is lessened or not needed. I/O ports908allow the computing device900to be logically coupled to other devices including the I/O components910, some of which is built in. Examples of the I/O components1810include, for example but without limitation, a microphone, keyboard, mouse, joystick, game pad, satellite dish, scanner, printer, wireless device, etc.

The computing device900includes a bus916that directly or indirectly couples the following devices: the memory902, the one or more processors904, the one or more presentation components906, the input/output (I/O) ports908, the I/O components910, a power supply912, and a network component914. The computing device900should not be interpreted as having any dependency or requirement related to any single component or combination of components illustrated therein. The bus916represents one or more busses (such as an address bus, data bus, or a combination thereof). Although the various blocks ofFIG. 9are shown with lines for the sake of clarity, some examples blur functionality over various different components described herein.

In some examples, the computing device900is communicatively coupled to a network918using the network component914. In some examples, the network component914includes a network interface card and/or computer-executable instructions (e.g., a driver) for operating the network interface card. In one example, communication between the computing device900and other devices occur using any protocol or mechanism over a wired or wireless connection920. In some examples, the network component914is operable to communicate data over public, private, or hybrid (public and private) using a transfer protocol, between devices wirelessly using short range communication technologies (e.g., near-field communication (NFC), Bluetooth® branded communications, or the like), or a combination thereof.

Some examples of the disclosure are used in manufacturing and service applications as shown and described in relation toFIGS. 10-12. Thus, implementations of the disclosure are described in the context of an apparatus of manufacturing and service method1000shown inFIG. 10and apparatus1100shown inFIG. 11. InFIG. 11, a diagram illustrating an apparatus manufacturing and service method1000is depicted in accordance with an example. In one example, during pre-production, the apparatus manufacturing and service method1000includes specification and design1002of the apparatus1100inFIG. 11and material procurement1104. During production, component and subassembly manufacturing1006and system integration1008of the apparatus1100inFIG. 11takes place. Thereafter, the apparatus1100inFIG. 11goes through certification and delivery1010in order to be placed in service1012. While in service by a customer, the apparatus1100inFIG. 11is scheduled for routine maintenance and service1014, which, in one example, includes modification, reconfiguration, refurbishment, and other maintenance or service subject to configuration management, described herein.

In one example, each of the processes of the apparatus manufacturing and service method1000are performed or carried out by a system integrator, a third party, and/or an operator. In these examples, the operator is a customer. For the purposes of this description, a system integrator includes any number of apparatus manufacturers and major-system subcontractors; a third party includes any number of venders, subcontractors, and suppliers; and in one example, an operator is an owner of an apparatus or fleet of the apparatus, an administrator responsible for the apparatus or fleet of the apparatus, a user operating the apparatus, a leasing company, a military entity, a service organization, or the like.

With reference now toFIG. 11, the apparatus1100is provided. As shown inFIG. 11, an example of the apparatus1100is a flying apparatus1101, such as an aerospace vehicle, aircraft, air cargo, flying car, earth-orbiting satellite, planetary probe, deep space probe, solar probe, and the like. As also shown inFIG. 11, a further example of the apparatus1100is a ground transportation apparatus1102, such as an automobile, a truck, heavy equipment, construction equipment, a boat, a ship, a submarine and the like. A further example of the apparatus1100shown inFIG. 11is a modular apparatus1103that comprises at least one or more of the following modules: an air module, a payload module and a ground module. The air module provides air lift or flying capability. The payload module provides capability of transporting objects such as cargo or live objects (people, animals, etc.). The ground module provides the capability of ground mobility. The disclosed solution herein is applied to each of the modules separately or in groups such as air and payload modules, or payload and ground, etc. or all modules.

With reference now toFIG. 12, a more specific diagram of the flying apparatus1101is depicted in which an implementation of the disclosure is advantageously employed. In this example, the flying apparatus1101is an aircraft produced by the apparatus manufacturing and service method1000inFIG. 10and includes an airframe1202with a plurality of systems1204and an interior1206. Implementations of the plurality of systems1204include one or more of a propulsion system1208, an electrical system1210, a hydraulic system1212, and an environmental system1214. However, other systems are also candidates for inclusion. Although an aerospace example is shown, different advantageous examples are applied to other industries, such as the automotive industry, etc.

The examples disclosed herein are described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program components, being executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program components including routines, programs, objects, components, data structures, and the like, refer to code that performs particular tasks, or implement particular abstract data types. The disclosed examples are practiced in a variety of system configurations, including personal computers, laptops, smart phones, mobile tablets, hand-held devices, consumer electronics, specialty computing devices, etc. The disclosed examples are also practiced in distributed computing environments, where tasks are performed by remote-processing devices that are linked through a communications network.

An exemplary method of multi-agent instruction generation comprises: generating, for a plurality of actor agents, a first set of instructions for performing a first set of manufacturing tasks, wherein the actor agents include at least one actor agent selected from the list consisting of: a human actor accessing a UI, an autonomous actor having a first sensor, a semi-autonomous actor having a second sensor, and a non-autonomous actor having a third sensor; receiving, by a control agent from at least the plurality of actor agents, observation data regarding performance of the actor agents on the first set of manufacturing tasks, wherein the control agent comprises an autoregressive bidirectional LSTM attention network; and based at least on the first set of instructions and the observation data, generating, by the control agent for the plurality of actor agents, a second set of instructions for performing a second set of manufacturing tasks, wherein the first set of instructions and the second set of instructions each includes at least one instruction selected from the list consisting of: a role assignment, a platform control, a tool selection, and a tool utilization.

An exemplary system for multi-agent instruction generation comprises: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: generating, for a plurality of actor agents, a first set of instructions for performing a first set of manufacturing tasks, wherein the actor agents include at least one actor agent selected from the list consisting of: a human actor accessing a UI, an autonomous actor having a first sensor, a semi-autonomous actor having a second sensor, and a non-autonomous actor having a third sensor; receiving, by a control agent from at least the plurality of actor agents, observation data regarding performance of the actor agents on the first set of manufacturing tasks, wherein the control agent comprises an autoregressive bidirectional LSTM attention network; and based at least on the first set of instructions and the observation data, generating, by the control agent for the plurality of actor agents, a second set of instructions for performing a second set of manufacturing tasks, wherein the first set of instructions and the second set of instructions each includes at least one instruction selected from the list consisting of: a role assignment, a platform control, a tool selection, and a tool utilization.

An exemplary computer program product comprises a computer usable medium having a computer readable program code embodied therein, the computer readable program code adapted to be executed to implement a method of multi-agent instruction generation, the method comprising: generating, for a plurality of actor agents, a first set of instructions for performing a first set of manufacturing tasks, wherein the actor agents include at least one actor agent selected from the list consisting of: a human actor accessing a UI, an autonomous actor having a first sensor, a semi-autonomous actor having a second sensor, and a non-autonomous actor having a third sensor; receiving, by a control agent from at least the plurality of actor agents, observation data regarding performance of the actor agents on the first set of manufacturing tasks, wherein the control agent comprises an autoregressive bidirectional LSTM attention network; and based at least on the first set of instructions and the observation data, generating, by the control agent for the plurality of actor agents, a second set of instructions for performing a second set of manufacturing tasks, wherein the first set of instructions and the second set of instructions each includes at least one instruction selected from the list consisting of: a role assignment, a platform control, a tool selection, and a tool utilization.

Alternatively, or in addition to the other examples described herein, examples include any combination of the following:receiving, by the control agent from at least the plurality of actor agents, second observation data regarding performance of the actor agents on the second set of manufacturing tasks;based at least on the second set of instructions and the second observation data, generating, by the control agent for the plurality of actor agents, a third set of instructions for performing a third set of manufacturing tasks;an encoder portion comprising a plurality of input-specific LSTMs and an attention network;a decoder portion comprising a decoder LSTM;the input-specific LSTMs receive at least a portion of the observation data, and wherein the decoder LSTM receives at least a portion of the observation data from the input-specific LSTMs and attention network;constraining instructions using actor masks;receiving observation data comprises receiving observation data from a sensor agent comprising a fourth sensor;training the control agent with synthetic training data;the manufacturing tasks include at least one task selected from the list consisting of: video capture, measurement, painting, soldering, welding, moving, cutting, drilling, puncturing, and hammering;training the control agent with synthetic training data occurs prior to generating the first set of instructions;training the control agent with synthetic training data occurs after generating the first set of instructions;a learning structure for improving performance of the control agent;the learning structure comprises decoder LSTM that receives outputs from the encoder portion and the decoder portion; andthe learning structure outputs a value assessment;rewards are derived using the value assessment and the observations; andthe rewards are generated and provided as feedback to provide ongoing performance improvements for the control agent.

When introducing elements of aspects of the disclosure or the examples thereof, the articles “a,” “an,” “the,” and “said” are intended to mean that there are one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that there could be additional elements other than the listed elements. The term “implementation” is intended to mean “an example of” The phrase “one or more of the following: A, B, and C” means “at least one of A and/or at least one of B and/or at least one of C.”