Patent Description:
In various technical fields, there is a need to control a machine or a system of machine. For example, in the industrial automation context autonomous machines gain more and more of importance. There are continuous efforts made in the process and manufacturing industry to efficiently implement a lot size one production based on factories equipped with modular and autonomous machines acting on the shop floor. At the same time, modular and in particular autonomous machines are hard to program and coordinate.

In the case of structured text, programming languages of the IEC <NUM> standard for programmable logic controller, short PLC, follow the imperative programming paradigm. They define the behavior of a concrete machine as a rigid sequence with explicit decision points. In the case of function block diagrams, a logical circuit is emulated. This is a good fit for the classical factory where precisely defined repetitive movements are desired, reactions depend on simple, often binary sensor input and the machine has a limited number of states, like for example running, idle, off, etc..

Behavior trees are a powerful programming paradigm to overcome the limitations of classical PLC programming approaches. A behavior tree describes switchings between a finite set of tasks in a modular fashion. Their strength results from their ability to create very complex tasks composed of simple tasks, without worrying how the simple tasks are implemented. A task describes single behavior steps, several tasks are combined into compositions defining in which order and under which condition the single tasks shall be executed. Behavior trees present some similarities to hierarchical state machines with the key difference that the main building block of a behavior is a task rather than a state. Its ease of human understanding make behavior trees less error prone. The expression behavior tree and behavior tree program are used as synonyms throughout this document.

Nowadays, behavior trees are manually created and entered to a controller of the machine or system of machine by automation engineers. While behavior trees are easier to handle than classical PLC programming languages, they still need some amount on training for the automation engineer.

<NPL> describes a learning from demonstration (LfD) approach to program a robot for an arbitrary task. The robot learns from these demonstrations a behavior tree representing the task. The transitions between learned actions are modeled as a finite state machine. A finite state machine is generated using a nearest neighbors' classifier for state transitions. Every action that the robot is told to perform and current state pair is recorded. Finally, the decision tree is converted to a behavior tree. The behavior tree is then used to execute the task without human input.

Therefore, it is one object of this invention to support the automation engineer in generating of a behavior tree.

These objects are addressed by the subject matter of the independent claims. Advantageous embodiments are proposed in the dependent claims.

According to a first aspect, a computer-implemented method for automatically generating a behavior tree program for controlling a machine is proposed, the method comprising the steps of.

Machine commands are entered by the user to control the machine. The machine command is executed in the controller and invokes one or several extensive closed program sections executing an action of the machine or an action of a component of the machine. Machine commands are structured according to, e.g., programming languages of the IEC <NUM> standard. Instead of programing the behavior tree, the behavior tree is automatically generated based on the interaction of the user with the machine. The user's intention with respect to the operation of the machine, which is not available explicitly, gathered without asking the user directly but by observation, prediction and extrapolation of the user behavior. Statistical inference is a process of using data analysis to infer properties of an underlying distribution of probability. The structure of the behavior tree is derived using statistical inference. Methods applied for statistical inference can be, e.g., association rule mining, an apriori algorithm, or methods for sequential pattern mining. The machine command is transmitted in a communication connection between the user interface and the controller via a communication protocol, e.g. OPC UA, Profinet.

According to an embodiment of the invention, the machine command activates a machine behavior, a functionality or an operational state in the machine.

Machine commands define the machine behavior, the functionality or an operational state on different level of details from high level describing a complete action to low level comprising very precise and detailed components of an action. This allows a detailed analysis considering the substructures of the machine commands and enables a prediction of the structure of the behavior tree close to the user's intended machine operation. In this description the expressions "machine command" and "high-level machine command" are used as synonyms.

According to an embodiment of the invention, the supervision data comprises status information and/or sensor data of the machine when controlled by the controller executing the high-level machine commands.

Status information comprise information whether the execution of the machine command and/or the action activated by the machine command was executed successfully of failed. Status information comprises also senor data values collected by sensors in the machine or in components of the machine providing information on the condition of the machine or conditions of the performed automation process. The supervision data can be provided independent of the execution of a machine command. Therefore, the status information provides a comprehensive overview on the operation of the machine.

According to an embodiment of the invention, at least one task of the behavior tree is derived from at least one machine command at the inference unit.

Tasks provide the basic components of a behavior tree and when executed preform an action of the machine. The inference unit provides the basic components of behavior tree and therefore generates the behavior tree from scratch, i.e., without any input of programming elements of the behavior tree.

According to an embodiment of the invention, the structure of the behavior tree is inferred by analyzing.

The inference unit derives by the described analyzing steps further structure of the behavior tree, especially composites and decorators defining rules with respect to condition and sequence the tasks are executed. Especially a decorator is inferred by analyzing the co-occurrence of supervision data values and changes of supervision data value with task execution, task cancellations and task parametrizations. Co-occurrence means analyzing which supervision data values or value chances occur at the same time as or timely related to the execution of one or several tasks or the cancellation of one or several tasks or the parameter settings of the task.

According to an embodiment of the invention, a composite combining several tasks of the behavior tree, is inferred by analyzing a sequence of successfully performed tasks and failed tasks and a transition probability for each pair of tasks in the sequence.

According to an embodiment of the invention, the inference unit provides suggestions for further machine commands to the user interface to be entered by the user.

This supports the user to automate machine operation. The user requires less knowledge of the machine command programming of the machine. Suggestions of the inference unit can be checked and reviewed and can used to train the inference unit continuously.

According to an embodiment of the invention, the inference unit provides a suggestion for a further behavior tree structure element to the user interface to be entered by the user. The structure of the behavior tree comprises at least on structure element, the structure element comprises a composition of behavior tree components. Behavior tree components are, e.g., tasks, composites, and decorators. The suggestion assists the user to improve in programming the behavior tree and improves the quality of the resulting behavior tree.

According to an embodiment of the invention, the inference unit comprises a semantic model providing domain knowledge on at least one of a sequence of tasks, conditions of the tasks and a transition probability for each pair of tasks in the sequence related to a specific machine operation scenario.

The semantic model comprises a collection of behavior tree components related to specific machine operation scenarios, wherein the behavior tree components define typical sequences of tasks and conditions under which the tasks shall be executed. The semantic model can provide information about tasks that can fail, and which fallback solutions are possible in this case. This accelerates the generation of the behavior tree for a new operation scenario and the generation of variants of the operation scenario. It further allows the inference unit to provide extensive suggestions
According to an embodiment of the invention, the inference unit provides a recommendation for the behavior tree structure element to the user interface based on an assumption with respect to a further intended machine operation for verification or modification of the recommendation.

This provides the inference unit the possibility to receive instant feedback of the user and early alignment of the behavior tree program with the machine operation intended by the user
According to an embodiment of the invention, the generated behavior tree program is provided from the inferring unit to the controller for controlling the machine.

The generated behavior tree program can be provided directly from the inferring unit to the controller or the generated behavior tree program can be transferred from the inferring unit to the user interface which forwards it from the user interface to the controller when an acknowledgement of the user is received. The direct provision allows a fast implementation of the behavior tree. The indirect provision via user interface allows the user to approve, correct or prohibit the implementation, i.e., the execution of the automatically generated behavior tree in the controller.

According to a second aspect, an assistance system for assisted generating a behavior tree program for a machine is proposed, comprising a programming interface, an observer unit, a logging unit and an inference unit, wherein.

According to an embodiment of the invention, the user interface comprises a graphical user interface configured to display the supervision data, a suggestion or a recommendation to the user and/or input the sequence of machine commands or feedback to the suggestion and/or recommendations by the user.

This provides a flexible communication between the user and the controller.

According to an embodiment of the invention, the logging unit is configured as a knowledge base.

The knowledge base is a technology used to store complex structured and unstructured information used by a computer system providing an expert system. A knowledge base represents facts and ways of reasoning about those facts to deduce new facts or highlight inconsistencies. Preferably, the knowledge base is structured as a knowledge graph.

According to a third aspect, a computer program product directly loadable into the internal memory of a digital computer is proposed, comprising software code portions for performing the steps according to one of the preceding claims when said product is run on said digital computer.

In the following, embodiments of the present disclosure will be described in more detail and with reference to the accompanying drawings.

It is noted that in the following detailed description of embodiments the accompanying drawings are only schematic and that the illustrated elements are not necessarily shown to scale. Rather, the drawings are intended to illustrate functions and cooperation of components. Here, it is to be understood that any connection or coupling of functional blocks, devices, components, or other physical or functional elements could also be implemented by an indirect connection or coupling, e.g., via one or more intermediate elements. A connection or coupling of elements or components can for example be implemented by a wire-based, a wireless connection, and/or a combination of a wire-based and a wireless connection. Functional blocks can be implemented by dedicated hardware, by firmware and/or software installed on programmable hardware, and/or by a combination of dedicated hardware and firmware or software.

<FIG> schematically illustrates a programming system <NUM> for programming a machine <NUM>, e.g., in classical automation industry.

The programming system comprises an operating interface <NUM>, a programming interface <NUM> automation, a controller <NUM> and a machine <NUM>. The machine <NUM> can be any kind of machine in manufacturing, e.g., automation cells, factory cells, production lines, process lines, assembly lines, or autonomous guided vehicles, short AGVs, Robots, and further.

The programming interface <NUM> is the interface between a user <NUM>, e.g., an automation engineer and the controller <NUM>. The automation engineer uses the programming interface <NUM> to define an automation program <NUM>. The automation program <NUM> can be structured according to any kind of programming language which is able to control a machine. Examples for programming languages structured text programming languages of the IEC <NUM> standard for programmable logic controller, short PLC, like Step7 comprising contact plan KOP, function plan FUB and Graph7. The programming interface <NUM> transmits the automation program <NUM> to the controller <NUM>. The automation program <NUM> is executed on the controller <NUM>. The controller <NUM> can be an automation system, a personal computer, an edge device, a PLC or any other kind of logical machine. During operation of the machine, the user <NUM>, e.g., a machine operator uses the operating interface <NUM> to monitor and control the machine <NUM>.

<FIG> illustrates an assistance system <NUM> generating a behavior tree program <NUM> for controlling a machine <NUM> according to an embodiment of the present disclosure. The machine <NUM> can be any kind of machine in manufacturing, e.g., automation cells, factory cells, production lines, process lines, assembly lines, or autonomous guided vehicles, short AGVs, Robots, and further, like machine <NUM>. The controller unit <NUM> controls the machine <NUM> by executing the behavior tree program <NUM>. The assistance system <NUM> assists in generating a behavior tree program for a machine <NUM>, comprising a programming interface <NUM>, an observer unit <NUM>, a logging unit <NUM> and an inference unit <NUM>.

The user interface <NUM> is the interface between a user <NUM>, e.g., a machine operator or an automation engineer and the controller unit <NUM>. The user interface <NUM> is configured as a Graphical User interface, a display, a keyboard, a mouse, control units, and the like. The user interface <NUM> is configured to receive input from a user in a role as machine operator, i.e., entering machine commands during operation of the machine. During a training phase of the assistance system <NUM>, especially of the inference unit <NUM>, the user <NUM> controls the machine <NUM> by entering machine commands into the user interface <NUM>. Machine commands, especially high-level machine commands, define a production step, a transportation step, a material flow, material handling, material processing, assembly steps, and so on.

The user <NUM> uses the user interface <NUM> to monitor and control the machine. To control the machine, e.g., a conveyor belt, the user enters a first machine commands which, when executed at the controller, instructs the machine to put a block on the conveyor belt. By a second machine command the user <NUM> starts the conveyor belt. By a third machine command the user <NUM> stops the conveyor belt when the block hits a light Barrier.

The observer unit <NUM> intercepts the communication on a communication connection <NUM> between the user interface <NUM> and controller unit <NUM>. The observer unit <NUM> collects all user machine commands entered to the user interface <NUM>, which are intended for the controller unit <NUM>. The observer unit <NUM> is a support tool to understand the user intention. The observer unit <NUM> collects the interaction of the user <NUM> and the feedback, i.e., supervision data from the machine or from components of the machine, e.g., sensors and light barrier of a machine configured as a conveyor belt with several activities being performed while transporting an object.

The logging unit <NUM> stores the communication, i.e., machine commands and supervision data observed and copied on the connection between user interface <NUM> and controller unit <NUM>. The logging unit <NUM> can be a database, a list, a file, a knowledge graph, a knowledge base, or similar. The logging unit <NUM> is the central storage system and contains all relevant information which the user <NUM> enters to the user interface <NUM> to derive the automation program for the controller unit <NUM>.

The inference unit <NUM> derives the behavior tree from the information in the logging unit <NUM>. The inference unit <NUM> derives at least one task of the behavior tree from at least one machine command. Tasks in the behavior tree typically correspond to machine commands. The inference unit <NUM> comprises a statistical inference functionality, also called statistical inference engine, that learns from the stored machine commands and supervision data, which tasks are typically executed in parallel or in sequence, what are usual parameter settings for tasks under which conditions, which tasks usually follow certain other tasks, events, e.g., proximity alert, or conditions, e.g., light barrier interrupted? Methods applied for statistical inference can be, e.g., association rule mining, the apriori algorithm, or methods for sequential pattern mining.

In doing so, preconditions, fallback scenarios, as well as typical sequences of tasks can be learned and tasks, composites providing a logical interconnection of several tasks, like a sequence composite or a fallback composite. The inference unit <NUM> sends suggestions to the user interface <NUM> to assist the user <NUM> to automate operations.

An example for statistical inference performed in inference unit <NUM> is shown in <FIG>. The assistance system <NUM> observes and stores two log files L1, L2 containing a sequence of machine commands that have been executed. Based on the machine commands tasks T11,. , T15, T21,. , T26 are derived. The Log files L1, L2 further comprise the success status and the state of variables for each task T11,. , T15, T21,. In <FIG>, tasks T11, T14, T15, T21-T23, T25, T26 have finished successfully, tasks T12, T13, T24 have failed. Based on this information transition probabilities TP for success and failure are inferred by the inference unit <NUM>. The inference unit <NUM> can now provide suggestions S for sequence composites and fallback composites to the user <NUM>, who can confirm or reject those suggestions.

<FIG> shows a sequence <NUM> of tasks T31,. The tasks are performed with high transition probabilities, see values indicated above the arrows between two tasks. The sequence <NUM> of tasks T31,. ,T33 is a candidate for being inferred as a sequence composite in case of successful task execution.

Tasks that follow with high probability on a failed task are a candidate for a fallback composite. <FIG> shows such a fallback composite <NUM> of tasks T41,. Task <NUM>, which is a successfully executed task, follows task T41 with a high transition probability.

The identification of decorators can also be supported by analysis of the log files. Decorators are attached to either a composite or a task and define whether a branch in the behavior tree, or even a single node, can be executed. The Decorator can also modify the success or failure status of the decorated task or composite.

In <FIG>, Task T51 is followed by Task T52 or by Task T53. The probability for a sequence of tasks T51-T52-T53 correlates with the values of variable A, the probability for the sequence T51-T53 correlates with the value of variable B. That means that variable A and B are candidates for an "if" decorator.

Candidates for a "guard" decorator can be identified by inferring whether the value of a specific variable changes for failed tasks. In the example in <FIG> the task T56 fails, and the value of variable A has changed. Variable A is a candidate for a "guard" decorator.

Candidates for a "Wait" decorator can be identified by observation if the value of a specific variable changes at the same time as a new task starts. The example in <FIG> shows a value change of a variable A correlating with a transition between tasks T54 and T55.

<FIG> shows a behavior tree program <NUM> generated automatically by the assistance system <NUM>. The behavior tree <NUM> controlling the system to move to a light barrier, see <NUM>. The behavior tree component <NUM> represents a "sequence" composite. Behavior tree component <NUM> represents "wait for light-barrier = false", behavior tree component <NUM> represents "< > stop", component <NUM> represents "wait for sensor_2 = true and component <NUM> represents "< > go right. The generated program learnt that:.

The inference unit <NUM> can comprise a semantic model, which provides domain knowledge. As an example, it provides the information "task can only be executed with a certain security level, picking of a robot arm is only successful in <NUM>% of the cases and consequently "fallback" is necessary. The semantic model can provide information about tasks that can fail, and which fallback solutions are possible. The inference unit <NUM> can make recommendations to the user <NUM> based on assumptions. The user <NUM> can verify or correct the assumptions. Whenever the inference unit <NUM> is unsure about something when creating the behavior tree program, user input can be requested, e.g., parallel tasks, fallbacks, preconditions, wait conditions, loops, if-else-statements, stop conditions. An example question to user is "We noticed that task A is typically executed after the light barrier is interrupted. Do you want to save this as a precondition?".

The assistance system <NUM> is configured to perform a computer implemented method for automatically generating a behavior tree program for controlling a machine <NUM> as shown schematically in <FIG>.

In a first step S1 a sequence of machine commands input by the user <NUM> at the user interface <NUM> are transmitted to the controller unit <NUM>. While the machine commands are executed in the controller unit <NUM> supervision data are received, see step S2, in the user interface <NUM> from the controller unit <NUM>. The machine commands and supervision data are transferred on a communication connection between the user interface <NUM> and the controller unit <NUM>, e.g., via a protocol messages according to OPV UA, Profinet and the like. A Machine command activates a machine behavior, a functionality, or an operational state in the machine <NUM>. Supervision data comprises status information and/or sensor data of the machine when controlled by the controller executing the machine commands. The machine commands and supervision data are observed and copied at the communication connection between the controller <NUM> and the user interface <NUM>, see step S3. Subsequently in step S4, the machine commands and the supervision data are stored in the logging unit <NUM>.

In step S5, a behavior tree program is derived from the stored machine commands and the supervision data by machine learning, especially by statistical interference. A least one task of the behavior tree is derived from at least one machine command at the inference unit <NUM>. The structure and the components, i.e., composites and decorators of the behavior tree are inferred by analysing, which of the derived tasks are executed in which temporal relation to each other, the return value of the supervision data, the parameter settings for the task with respect to the execution of the task, the co-occurrence of supervision data values and changes of supervision data value with task execution, task cancellations and task parametrizations. the inference unit <NUM> provides suggestions for further machine commands to the user interface <NUM> to be entered by the user.

The inference unit <NUM> provides suggestions for further machine commands to the user interface <NUM>, which can in return be accepted and entered by the user. The suggestions can also be ignored and/or different machine commands or behavior task structure elements, i.e., tasks, composites or decorators can be entered. The inference unit <NUM> provides a recommendation for a behavior tree structure element to the user interface <NUM> based on an assumption with respect to a further intended machine operation for verification or modification of the recommendation. Finally, the generated behavior tree program is sent to the controller unit <NUM> to control the machine <NUM>, see step S6.

Claim 1:
A computer implemented method for automatically generating a behavior tree program (<NUM>) for controlling a machine (<NUM>), the method comprising the steps of:
- transmitting (S1) a sequence of machine commands input by a user from a user interface (<NUM>) to a controller (<NUM>),
- receiving (S2) supervision data in the user interface (<NUM>) from the controller (<NUM>) while the machine commands are executed in the controller (<NUM>) controlling the machine (<NUM>),
- observing and copying (S3) the machine commands and supervision data transmitted between the controller (<NUM>) and the user interface (<NUM>),
- storing (S4) the machine commands and the supervision data in a logging unit (<NUM>),
- generating (S5) a behavior tree program derived from the stored machine commands and the supervision data by statistical inference, and
- sending (S6) the generated behavior tree program to the controller unit (<NUM>) to control the machine.