Patent ID: 12187418

DETAILED DESCRIPTION

Aircraft systems are becoming increasingly more complex. With advances in technology, the capabilities of aircraft are increasing. As aircraft systems grow in complexity, automation is used often to alleviate an increasing burden on operators of these aircraft, such as pilots. However, developing systems to provide the automation is commensurately difficult.

Generally, when designing an aircraft control system, developers are provided with a specification. The specification defines predetermined scenarios and desired operating procedures for the aircraft control system in these predetermined scenarios. The developers will then create an aircraft control system which is designed to operate in compliance with the specification. Where scenarios that are not foreseen in the specification arise, the pilots are able to initiate operations manually in the aircraft and/or override the aircraft control system. Nevertheless, it would be desirable to have an aircraft control system that is able to apply artificial intelligence to react appropriately to scenarios that are beyond the scope of the design specification.

In a given scenario there are a plurality of ways in which the aircraft control system could operate to adequately control the aircraft. However, developers are generally more likely to design the aircraft control system to operate the aircraft in a manner dictated by their experience. Due to the increasing complexity of aircraft, in some scenarios there may be unforeseen ways to operate the aircraft, which may be desirable. For example, there may be ways of performing routines in the aircraft which require the operation of fewer components, are more efficient, and provide other beneficial characteristics. Therefore, it would be desirable to provide an aircraft control system which is able to identify valid operating procedures for controlling the aircraft which may not have been foreseen by the developers.

Generally, in a given scenario, defined by some aircraft operating inputs, the aircraft control system should be able to identify an operating procedure for achieving a desired result. Operating procedures comprise a list of tasks, or operations, which are performed for a given scenario. The list of tasks includes the operation of specific equipment such as actuators (including valves operating coils and/or pumps), and initiating sensor readings, and other equipment. In some cases, the operating procedures specify a particular manner in which specific equipment is operated. Where there are dependencies between certain tasks, in other words where operating a piece of equipment is dependent on the operation of another piece of equipment, the operating procedure may comprise an ordered list of tasks.

Certain examples described herein relate to controlling an aircraft by using a classifier to determine an operating procedure for controlling the aircraft. The aircraft control system is adapted to process received input data using a classifier to generate control outputs for operating the aircraft. The classifier has a plurality of parameters which are adapted to represent a control policy which is based on a desired operation of the aircraft. Using a classifier in the aircraft control system allows the aircraft control system to produce suitable control outputs in a wide range of scenarios potentially even beyond those initially devised by the developers of the aircraft control system.

Certain examples described herein provide a machine learning system for training a classifier for controlling an aircraft. The machine learning system is adapted to evaluate a plurality of operating procedures in each of a plurality of scenarios and to train the classifier based on the results. Rather than developers having to manually devise operating procedures they may instead specify a scenario and a desired result. The machine learning system is then adapted to evaluate possible operating procedures and to determine a desired one which can be used to train the classifier. This may allow the machine learning system to devise operating procedures which may be new and non-intuitive to developers. In this way, the machine learning system may identify more-optimum operating procedures and increase the automation capability of the aircraft control system. Multiple measures may be used to constrain a desired operating procedure in each of a plurality of scenarios. In this way, the developers may still exercise control over operating procedures which are produced and the degree to which they exercise this control is determined by the number of and specificity of the measures they use. Further, the developers may review the operating procedures generated by the machine learning system before using them to train the classifier in the aircraft control system.

FIG.1shows an example aircraft control system100. The aircraft control system100is a combination of hardware and software installed in an aircraft and used to control the aircraft. The aircraft control system100comprises computing equipment for monitoring the aircraft and processing data to produce control signals. The computing equipment is connected to other physical equipment in the aircraft including processors, actuators, sensors, aircraft control equipment for the pilots, and any other physical equipment which is used to control the aircraft.

The aircraft control system100comprises an input interface102for receiving aircraft operating inputs such as sensor outputs and user inputs associated with an operating state of the aircraft, data and/or information from other processing systems and/or physical equipment (collectively shown as106a-106n). The input interface102comprises suitable signal processors, for instance, for converting analogue and/or digital inputs from physical equipment into data for processing. The input interface102receives the aircraft operating inputs and generates input data104representing an overall operating state of the aircraft.

According to the present example, the equipment106ato106nshown inFIG.1includes, but is not limited to, a variety of sensors, and aircraft controls.

The sensors may include a mixture of environmental sensors and equipment sensors. Environmental sensors include any of thermometers, hygrometers, altimeters, barometers, and other environmental sensors arranged to monitor specific equipment and/or regions in the aircraft or a general state of the aircraft. The equipment sensors are arranged to monitor operating states of equipment in the aircraft, which may include detecting faults in equipment of the aircraft. Although the aircraft operating inputs in this example comprise signals from sensor outputs and user inputs, it is to be appreciated that the aircraft operating inputs may come from just one of these sources.

An operating state of the aircraft may comprise one or more of environmental conditions, operating states of specific equipment within the aircraft, a goal to be achieved, and other suitable information. A goal to be achieved may be determined by an input provided by the pilots, for example, to apply the brakes, to increase throttle, etc. Alternatively, the goal to be achieved may be an automatic goal generated in response to a change in state of the aircraft, for example a reduction in altitude, a reduction in engine power, etc. To this end, the input data104may comprise a plurality of variables indicative of the operating state of the aircraft.

The input data104represents an instantaneous state of the aircraft at a given moment. Alternatively, the input data104may represent a dynamic state of the aircraft, wherein the input data104indicates a trend in one or more variables which represent sensor outputs and/or user inputs. The input data104may be generated based on a subset of the plurality of aircraft operating inputs provided to the input interface104and may be dependent on a change of one or more specific aircraft operating inputs. In this case, the aircraft operating inputs are continually provided to the input interface102and input data104is generated in response to a change in one or more of the aircraft operating inputs.

A processing engine108in the aircraft control system100comprises a classifier110having parameters θ1to θnwhich are adapted to represent a control policy fθ(Vn) for operating the aircraft. The processing engine108is arranged to apply the input data104representing the operating state of the aircraft to the classifier110to generate control output data112. An output interface114is used to generate control outputs from the control output data112to control the aircraft. The control outputs are used to operate the equipment116ato116nin the aircraft. The output interface114is capable of producing both analogue and digital signals for control outputs using any suitable number and combination of signal processors. The analogue and digital signals are then used to operate relevant equipment such as actuators, pumps, and to instruct processors and the like.

In an example, the processing engine108is implemented using at least one processor and at least one memory. The at least one processor used to implement the processing engine may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), an application specific instruction set processor (ASIP), or any other suitable processing device. The at least one memory may be any suitable combination of volatile and non-volatile memories including random-access memory (RAM), read-only memory (ROM), synchronous dynamic random-access memory (SDRAM), or any other suitable type of memory. The at least one memory comprises instructions which, when executed by the at least processor, cause the processor to obtain the input data104and apply the input data104to the classifier to generate control output data112. The at least one memory stores the parameters θ1to θnfor use in processing the input data104. The classifier110which is used may be any suitable classifier, some examples of which include: decision trees, naïve bayes classifiers, artificial neural networks, and k-Nearest Neighbour classifiers.

FIG.2shows an example wherein the classifier110comprises a neural network having an input layer202a, two hidden layers202band202c, and an output layer202d. In the example shown inFIG.2, aircraft operating inputs I1to I9are received by the input interface102. The input interface102generates input data104comprising variables V1to V3which are applied to the neural network by introducing them at the input layer202aand processing them according to the parameters θ1to θnof the neural network. The variables V1to V3may for instance include scalar values or vectors. According to this example, the parameters θ1to θnare the weights of nodes defining the neural network and which are used to process the input data104. Although two hidden layers202band202care shown in this example, the neural network may comprise any suitable number of hidden layers for performing the functions described herein. Neural Networks have been found to be particularly suitable for the present application in aircraft control systems100due to their ability to process higher dimensional data as inputs, and due to the high number of possible scenarios and operating procedures in aircraft.

Values V4and V5are extracted from the output layer202dof the neural network and used to generate control output data112. The control output data112is then used to generate control outputs O1to O7at the output interface114, which are used to control the aircraft, in particular by operating equipment116ato116n.

The control policy fθ(Vn) represents a desired operation of the aircraft under a plurality of scenarios. The parameters θ1to θnare generated based on training data representing the control policy fθ(Vn) for each of the plurality of scenarios. By training the classifier110using training data which represents the control policy fθ(Vn) the classifier110is able to be trained to act suitably under the plurality of scenarios. In an example, the training data is generated by selecting an operating procedure for the aircraft control system100for each of the plurality of scenarios.

Using a classifier110which is trained from this data may also allow the classifier110to generate output control data112which estimates a desired operation of the aircraft in scenarios which are not defined in the training data. Thus, the aircraft control system100may be able to control the aircraft in wider range of scenarios than other, known aircraft control systems.

As the parameters θ1to θnare trained using the training data, the control policy fθ(Vn) is dependent on the operating procedures which are selected for each scenario. In an example, there are a plurality of different operating procedures for each scenario and one operating procedure is selected from the plurality based on how well it satisfies the desired operation of the aircraft. Steps for selecting an operating procedure in a given scenario are illustrated by a flow chart300inFIG.3.

At block302of the flow chart300, at least one target outcome measure is determined for the scenario. For example, for a scenario comprising regulating the temperature of a cabin, the at least one target outcome measure may include the desired temperature of the cabin and may also include a measure relating to power used to control the temperature in the cabin. Another example of a scenario is where the aircraft has touched-down during landing, and the brakes are to be applied. In this case, the at least one target outcome measure may include an amount of braking force applied to achieve braking safely and within a shortest desirable distance, taking account of an amount of deceleration that is comfortable to passengers, a temperature limit for certain components in the brakes, and any other related measure which may be applicable in this case.

At block304, a plurality of operating procedures is generated for the scenario. These operating procedures are based on an environment representing at least part of an aircraft. The environment may be a simulated version of the aircraft, or at least part of it, with which operating procedures can be generated and assessed. The environment represents a plurality of components or equipment in the aircraft and ways in which they can be operated to control the aircraft. In some cases, the environment represents a specific system within the aircraft such as a braking system, a fuel system or an environmental control system for the cabin. An example of an environment will be described in detail hereafter with reference toFIGS.4to7.

The number of operating procedures which are generated for each scenario may depend on the complexity of the environment and the specific scenario which is being assessed. In the present example of a general case, a suitable number of operating procedures is 1000. However, there may be fewer operating procedures, such as 100 or even 10, or more, such as 10000. In some examples, techniques are applied to limit the total number of operating procedures which are used. Such techniques include producing a first set of operating procedures, and subsequently producing a second set of operating procedures based on the results of the first set. This will be described further with reference toFIG.9.

At block306, a plurality of outcome measures are determined based on the plurality of generated operating procedures, at least one outcome measure being determined for each operating procedure. The number of outcome measures depends on the scenarios and the aircraft control system to be operated. Determining the at least one outcome measure includes simulating a result of performing the respective operating procedure in the aircraft for the scenario and assessing the result. The environment may include a feedback mechanism wherein the operating procedures can be applied to the environment and at least one outcome measure per operating procedure is determined by the feedback mechanism.

At block308, an operating procedure is selected based at least on a comparison of the at least one target outcome measure with the plurality of outcome measures. In this way, the operating procedure which is selected satisfies the at least one target outcome measure and so represents a desired operation of the aircraft. Once an operating procedure has been selected for a given scenario using the graph model400, training data may be generated and used to train the classifier110.

The target outcome measures which are used to assess the performance of the operating procedures influence the way in which the aircraft is operated. A human developer of the aircraft control system may have a preconception regarding what a particular operating procedure should include, and this will influence their decisions when designing said operating procedure. However, by selecting a target outcome measure, or several, and performing an assessment of a plurality of operating procedures based on this, certain operating procedures, of which the human developer would not have conceived, may be evaluated and even selected. In some examples, a selected operating procedure is reviewed by a human operator and/or developer in order to determine its suitability before being used to train the classifier110in the aircraft control system100.

FIG.4shows an example of at least part of an environment. The environment comprises a graph model400representing the at least part of an aircraft. The graph model400comprises a plurality of nodes402ato402sand a set of directed edges404ato404uconnecting the plurality of nodes402ato402s. The plurality of nodes402ato402scomprises a start node402aand at least one end node402nto402s. In the example shown inFIG.4one node402sis shown in broken lines to indicate that a single end node402smay be provided. In some examples, different pathways through the graph model400may end at respective end nodes402nto402r. A common end node402scan be used to end the pathways through the directed graph400at a common point and trigger an evaluation of the performance of the pathways. The plurality of operating procedures is generated by determining pathways in the graph model400from the start node402ato at least one end node402nto402s.

The nodes402ato402seach represent a step in an operating procedure for controlling the aircraft. In some examples, the plurality of nodes402ato402scomprises a first subset of nodes and a second subset of nodes. The nodes in the first subset of nodes each represent a piece of equipment being operated in the at least part of an aircraft. The nodes in the second subset of nodes each represent a manner in which a piece of equipment is operated in the at least part of an aircraft. The operation of some equipment in the aircraft may be simple, for example a binary choice of on or off. Operating this equipment is represented by the first subset of nodes. The operation of other equipment in the aircraft may be more complex, for example involving a more granular choice of operation, and is represented by the second subset of nodes. Representing the equipment in the aircraft in this way in the graph model400greatly assists in identifying which potential operating procedures are feasible. If the graph model400were to model all equipment used and their interdependencies, then the resulting graph model would be extremely complex, and it may not be practical to identify which potential operating procedures are feasible. The distinction between the first subset and second subset of nodes will be described further with reference to the specific example ofFIG.7.

The environment representing the at least part of an aircraft may be dependent on the scenario which is being evaluated. For example, modifications are made to the environment to represent the scenario. In a scenario which includes a reduction in performance of a specific piece of equipment, the environment may be modified to reflect the reduction in performance of that piece of equipment. In this way, the plurality of scenarios used in the training data can be more complex and involve specific scenarios which go beyond environmental states and goals to be achieved and can involve operating states of equipment in the aircraft.

Depending on the complexity of aircraft, and the respective graph models400which represent them, it may not be feasible to evaluate all operating procedures, represented by pathways in the graph models400. In other words, brute force techniques for evaluating operating procedures would take too long to run and/or require more computing power than is feasible to use. Consequently, techniques to increase the efficiency of the evaluation of operating procedures are utilised in order to allow the application of the present disclosure to large complex aircraft with many systems and subsystems. Selecting an operating procedure is a multistage process which involves refining the operating procedures by iteratively generating pluralities of operating procedures based on a selected subset of operating procedures in a previous plurality.

Starting with a plurality of operating procedures, a subset of this plurality of operating procedures is selected based on a comparison of the at least one target outcome measure with the plurality of outcome measures, for example, the highest performing operating procedures. In other words, this selected subset includes operating procedures which were closest to achieving, or exceeded, the at least one target outcome. A further plurality of operating procedures is generated based on the selected subset of the plurality of operating procedures. Each operating procedure in the selected subset of operating procedures comprises a plurality of operations, each of which is selected from a number of possible operations. When generating the further plurality of operating procedures, some of the operations in the selected subset of operating procedures are replicated in the further plurality of operating procedures. However, the selection of some of the operations are evaluated by selecting alternatives when generating the further plurality of operating procedures. A number of different techniques may be used to perform this evaluation, including metaheuristic optimization algorithms such as genetic algorithms, and heuristic search algorithms such as Monte Carlo Tree Search.

In a specific example of the present disclosure, the aircraft control system is a braking control system. Braking control systems comprise a variety of components including physical actuation equipment and computing hardware. An example of a braking control system is shown inFIGS.5and6.FIG.5shows a hydraulic system500comprised in the braking control system.FIG.6shows an avionic system600comprised in the braking control system. In the examples described herein the braking control system is a hydraulic braking control system. However, it will be appreciated that in other examples, the braking control system may be an alternate braking control system, such as an electrical braking system in which the brakes are operated by electrical actuation equipment.

The hydraulic system500shown inFIG.5comprises components which are operated in order to control the brakes502a,502b. The brakes502aand502bcan be operated either using a primary power supply or an alternative power supply. In this example, the primary power supply is provided by a hydraulic pump504, and the secondary power supply is provided by a hydraulic accumulator506. A selector valve508selects operation using the primary power supply504in a primary selector system510a. Under normal operation, the primary power supply504is used and so the selector valve508may be referred to as the normal selector valve508. The primary selector system510amay also comprise other components including monitoring equipment (not shown). Operating the normal selector valve508comprises providing an electrical signal to a coil which actuates the valve. The normal selector valve508may comprise two such coils for redundancy purposes.

The hydraulic accumulator506is a pressurised container. When used to operate the brakes502aand502b, the pressure which is delivered to the rest of the braking system from the accumulator506is monitored and controlled to ensure the safe operation of the brakes502aand502b. To this end, there is provided a secondary selector system510b. The secondary selector system510bcomprises a relief valve512, to control the pressure of fluid released from the accumulator506, a pressure transducer514for monitoring the pressure from the hydraulic accumulator506, and a secondary selector valve516to select the hydraulic accumulator506as the power supply for operating the brakes502aand502b.

After the operation of either the normal selector valve508or the secondary selector valve516, power, in the form of pressurised fluid, is provided in the brake operation system510c. A first servo valve518and a second servo valve520are used to control the supply of hydraulic fluid from the hydraulic pump504to the first502aand second502bbrakes respectively. A third servo valve522is used to control the supply of hydraulic fluid provided by the accumulator506to the first502aand second502bbrakes. The hydraulic system500comprises two pressure transducers524and526for monitoring the pressure of hydraulic fluid being provided to each of the brakes502aand502b.

Shuttle valves532,534are included between the first518, second520, and third522servo valves and the brakes524,520to ensure that the highest-pressure input is fed through to the brakes. The use of shuttle valves prevents the pressurised fluid from the accumulator506from being fed back into the primary power supply, and vice versa. In other words, these shuttle valves ensure that fluid delivered from the accumulator goes to the brakes and not backwards through the first518and second520servo valves.

Two tachometers528and530are used to monitor the speed of the wheels during braking to provide feedback in the braking control system. It will be appreciated that other components not shown may also be included and used in the hydraulic system500, such as further sensors, actuators, and the like. It is also to be understood that while specific examples of equipment have been described herein, other equipment may also be used. For example, a wheel speed sensor other than a tachometer may be used to monitor the speed of the wheels during braking. Similarly, other pressure sensors rather than pressure transducers may be used to monitor the pressure in the hydraulic system500.

In the example shown inFIG.5, equipment is used to produce at least some of the aircraft operating inputs I1to I9which are received at the input interface102. The input interface102is configured to receive inputs from the power supplies504and506for operating the brakes502aand502b, at least one of a plurality of coils used to operate respective components for actuation equipment, pressure sensors, such as514,524, and526, and wheel speed sensors, such as tachometers528and530. It will be appreciated that the input interface102may be configured to receive inputs from all of these components or from any subset of these components.

Signals generated by the pressure transducers514,524,526may be received at the input interface102and may be used to determine an operating state of the aircraft. The tachometers528and530also provide aircraft operating inputs to the input interface102.

At least some of the aircraft control outputs O1to O7are used to control equipment in the hydraulic system500such as the selector valves508,516and the servo valves518to522. In an example, aircraft control outputs used to operate the selector valves508,516and the servo valves518to522are used to deliver power to operate these pieces of equipment. Alternatively, the aircraft control outputs are control signals used to instruct the equipment wherein the equipment comprises its own power supply. Control signals may be analogue or digital signals. The valves508,512,516,518,522, and520may comprise respective control units configured to operate their respective valve in a predetermined manner based on a received control signal.

The avionic system600shown inFIG.6comprises two remote braking control units602aand602b. These remote braking control units602aand602beach comprise memory, in the form of any suitable combination of volatile and non-volatile memory, and at least one processor.

Two remote braking control units602aand602bare provided for redundancy purposes, however it will be appreciated that the braking control system may have only one remote braking control unit602a. The remote braking control unit602acomprises two modules604aand604bwhich are adapted to control operation of the equipment in the hydraulic system500. The first module604ais connected to: primary coils for operating the normal selector valve508, the servo valves518,520, and to the sensor equipment including the tachometers528,530, and the pressure transducers514,524,526. The second module604bis connected to: secondary, or redundant, coils for the normal selector valve508, the servo valves518,520, and to the sensor equipment528,530,514,524,526. The second module604bis also connected to the selector valve516for selecting the accumulator506, and to a primary coil of the servo valve522. The second remote braking control unit602bsimilarly comprises first and second modules604cand604d. However, only the connections to the second module604dhave been shown for clarity inFIG.6.

In the present example, the remote braking control units602aand602bare part of the input and output interfaces102,114and so provide the control outputs O1to O7to the valves508,512,516,518,522, and520. However, in other examples, the remote braking control units602aand602bmay be separate from and communicatively coupled to the input and output interfaces102,114. In other words, the input interface102may be configured to receive inputs from avionic equipment in the braking control system. In such an example, the remote braking control units602aand602bprovide the operating inputs I1to I9and receive the control outputs O1to O7.

FIG.7shows an example of a graph model700representing the avionic equipment and some physical actuation equipment in the braking control system. The graph model comprises a plurality of nodes702to768connected by a set of directed edges. Due to the relatively large number of components in the braking control system, and the plurality of ways in which the braking control system can operate, the graph model700does not simply represent an overview of the equipment in the braking control system and the relevant connections. Rather, the graph model700also represents potential operating procedures in the form of pathways from a first node702to an end node760.

The graph model700comprises three sections separated inFIG.7by dashed vertical lines. The first section, located on the left and comprising nodes762to768, represents an operating procedure where no brakes are applied. The second, middle section represents operating procedures for operating the brakes using the hydraulic pump504and the first remote braking control unit602a. The third section, on the right, represents operating procedures for operating the brakes using the hydraulic pump504and the second remote braking control unit602b. It will be appreciated that when generating training data according to examples described herein, the graph model which is likely to be used would also include the possibility of operating the brakes502aand502busing the hydraulic accumulator. There will be further operating procedures which are possible, beyond those shown here.

A first subset of nodes704to718and752to758represent operation of physical actuation equipment in the braking control system. Nodes704and706represent operation of the normal selector valve508using either a primary or secondary coil respectively. Nodes712and714represent operating the servo valve518using either a primary or secondary coil respectively.

Nodes720to750of a second subset of nodes represent a manner in which a brake is operated. The manner in which a brake is operated may include specifying avionic equipment used to operate said brake, operating the brake in an environmental compensation mode, such as an antiskid mode, and operating the brake not in an environmental compensation mode. In this example, node720represents operation of the first brake502ausing the first module604ain an environmental compensation mode. Where the environmental compensation mode is an antiskid braking mode, the antiskid mode involves using feedback from the tachometer528to adjust the braking provided to the first brake502a, so as to prevent skidding. Node722, in contrast, represents operating the first brake502ausing the first module604a, not in an environmental compensation mode. Nodes724and726represent similar functionality to nodes720and722respectively, except that they are operated using the second module604b. Nodes728to734represent similar function to nodes720to726from which they follow but represent a manner of operation of the second brake502b. Nodes752and754both represent operation of servo valve520but using either a primary or secondary coil respectively.

Nodes708,710,716,718,736to750,756, and758represent similar operations and functions as the nodes previously described. Nodes in corresponding positions in the second and third sections represent the same function except that the nodes708,710,716,718,736to750,756, and758are dependent on the operation of the second remote braking control unit602brather than the first remote braking control unit602a.

FIG.8shows an aircraft800according to an example. The aircraft8comprises an aircraft control system as described in relation to any ofFIGS.1to7. In the present example, the aircraft800comprises a braking control system for operating the brakes810in the aircraft800. The aircraft800comprises other aircraft control systems, having classifiers110, such as environmental regulation systems, avionics systems, navigation systems, communication systems, flight control systems, and any other suitable systems in the aircraft800. Alternatively, the aircraft800comprises a single aircraft control system, which is adapted to provide control outputs for a plurality of systems in the aircraft based on aircraft operating inputs using a classifier110. This may allow, dependencies between different systems in the aircraft to identified and accurately handled.

In another example of the present disclosure, as shown inFIG.9, there is provided a machine learning system900for training a classifier110of an aircraft control system100. The machine learning system900is implemented using at least one processor and at least one memory comprising any suitable combination of volatile and non-volatile memory. The machine learning system may also comprise one or more user interfaces and data inputs such that a user can configure and/or operate the machine learning system900to achieve a desired result. The machine learning system900comprises an environment902, a pathway evaluation engine904, storage906, and a training engine908. The environment902, pathway evaluation engine904, and the training engine908are implemented as software modules in the machine learning system900. The at least one memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to implement the modules902,904, and908.

The environment902comprises a graph model representing at least part of an aircraft, such as the graph model shown in and described with reference toFIG.7. The pathway evaluation engine904is adapted to obtain data910representing at least one target outcome measure for a given scenario. The data910shown inFIG.9is received by the machine learning system900, for example, as an input from a user via a user interface and/or by receiving the data910over a network interface. However, it will be appreciated that the data910may be stored in storage906and retrieved by the pathway evaluation engine904.

The pathway evaluation engine904is then configured to select a pathway from the start node702to at least one end node760based on a comparison of at least one outcome measure determined for the selected pathway and the at least one target outcome measure. For example, where the aircraft control system100is a braking control system, the at least one target outcome measure includes a desired amount of braking force applied. The at least one target outcome measure may also include other variables, such as an amount of deceleration of the aircraft, a temperature value for one or more components in the brakes, and other suitable variables.

The pathway evaluation engine904is then configured to generate training data912representing a control policy fθ(Vn) for the given scenario based on the selected pathway. The training data912associates the selected operating procedure with the given scenario. Where the classifier110comprises a neural network, the given scenario is defined by one or more input variables V1to V3and these input variables are associated with specific output variables V4and V5which are to be produced by the classifier110to control the aircraft. This training data912is then stored in the storage906to be used for training the classifier110of the aircraft control system100. Storing the training data912in the storage906allows a cumulative set of training data to be generated. In this way, as the systems develop and/or as more scenarios are determined and assessed, the set of training data can grow and so future classifiers110may be trained based on larger and more extensive data sets. The training engine908is adapted to train the classifier110of the aircraft control system using the training data912by updating and/or generating parameters θ1to θn.

In the present example, the plurality of scenarios which are used to generate training data912for training the classifier110to control the aircraft are representative of scenarios which are known to operators and developers of the aircraft. In this way, the training data912may be extensive and cover the general operation of the aircraft such that when the aircraft control system100is in operation it will react predictably under scenarios based on which it has been trained. Further, by selecting the operating procedure based on a comparison of at least one outcome measure and at least one target outcome measure rather than by manually determining an operating procedure, the machine learning system900may identify superior ways of controlling the aircraft. For example, due to increased efficiency, higher reliability, requiring the operation of fewer components, providing increased comfort, and other suitable variables.

In some cases, due to the complexity of the environment902which represents the at least part of an aircraft, a large number of pathways may need to be evaluated before a suitable one can be selected. The number of pathways which may be generated before a suitable one is found may be prohibitively high and hence may be infeasible based on available computing power. In this case it is desired to determine pathways more efficiently so as to converge on a suitable pathway more quickly.

To this end, selecting a pathway may involve a multistage process. A first plurality of pathways from the start node702to at least one end node760are generated. This involves using the pathway evaluation engine904to randomly select routes in the graph model700. The number of pathways in the first plurality will be dependent on the at least part of an aircraft which is being modelled and on the computing power available to the machine learning system900. A second plurality of pathways are then evaluated from the start node702to at least one end node760. Evaluating the second plurality of pathways involves generating the second plurality of pathways and determining respective outcome measures. The second plurality of pathways are generated based on a selected subset of the first plurality of pathways. A pathway is then selected from the second plurality of pathways based on a comparison of at least one outcome measure determined for the selected pathway and the target outcome measure, or a comparison of a plurality of outcome measures and target outcome measures.

By generating the second plurality of pathways based on a selected subset of the first plurality of pathways it may be possible to exclude and not re-simulate pathways which relate to non-viable operating procedures. The subset of the first plurality of operating procedures are selected based on a comparison of outcome measures determined for the first plurality of pathways with the at least one target outcome measure.

In the specific example of a braking control system as shown inFIGS.5,6and7, a first plurality of pathways may be generated from node702to760. Consider a scenario in which the first remote braking control unit602adevelops a fault and a user input instructing the braking control system to apply braking is received. The first plurality of pathways might include the four pathways P1to P4which traverse the nodes as indicated below:
P1=[702, 704, 712, 722, 732, 754, 760]
P2=[702, 706, 714, 726, 732, 754, 760]
P3=[702, 708, 716, 738, 746, 756, 760]
P4=[702, 710, 718, 740, 750, 758, 760]

Based on a comparison of an amount of braking resultant from operating procedures P1to P4with a target amount of braking, it may be identified that pathways P1and P2are unsuitable as they rely on remote braking control unit602a.

The pathway evaluation engine904may then generate a second plurality of pathways which are based on the pathways, P3and P4. Consequently, the second plurality of pathways which are generated may include the pathways outlined below:
P5=[702, 708, 716, 736, 744, 756, 760]
P6=[702, 710, 718, 738, 748, 758, 760]
P7=[702, 708, 718, 740, 746, 758, 760]
P8=[702, 710, 716, 742, 750, 756, 760]

This second plurality of pathways have been generated based on the subset of the first plurality of pathways P3, P4and so includes operating procedures which rely on the operation of the second remote braking control unit602bwhich is functional in this scenario. In this way, the evaluation of pathways may converge on solutions which are more promising while preventing the machine learning system900from continuing to evaluate pathways which provide poor results and are unlikely to contain rewards. This increases the scalability and applicability of the machine learning system to classifiers110in aircraft control systems100which are adapted to control complex modern aircraft.

It will be appreciated that the example of the first and second pluralities of pathways described above are relatively small samples and that pluralities of pathways may each involve hundreds or even thousands of pathways.

When generating the second plurality of pathways, it is desired to identify and evaluate pathways which provide high rewards with respect to the at least one target outcome measure. In order to accomplish this, diversity may be introduced into the second plurality of pathways when they are generated. Metaheuristic optimization algorithms are used in order to increase the diversity in subsequent pluralities of pathways. These algorithms may involve stages of selection, reproduction, mutation, and recombination. Pathways from the first plurality of pathways may be selected based on their results. These selected pathways may then be combined and mutated according to techniques in evolutionary computing. These are then used in generating the second plurality of pathways. Metaheuristic optimization algorithms may be used in conjunction with suitable search algorithms, for example Monte Carlo Tree Search.

In some example, these metaheuristic optimization algorithms may also include selecting some pathways which do not receive large rewards (based on outcome measures), but which can be used to increase the diversity in the subsequent plurality of pathways. The processes described above may be iteratively repeated, for example generating a third plurality from the second, and fourth plurality from the third and so on. After many iterations, the highest ranked pathways, based on outcome measures, may be selected for use in generating the training data.

A method corresponding to the function of the machine learning system900is also provided and illustrated inFIG.10by a flow diagram1000. At a first block1002, the method for training a classifier110of an aircraft control system100comprises generating training data912representing a control policy fθ(Vn) for a plurality of scenarios. At block1004the method comprises training the classifier110of the aircraft control system100to generate output control data112according to the control policy fθ(Vn) using input data104representing an operating state of the aircraft800.

A process for generating the training data for each scenario is illustrated as a series of sub steps illustrated by blocks1002ato1002c. At block1002a, at least part of the aircraft800is represented using a graph model400comprising a plurality of nodes402ato402sand a set of directed edges404ato404uconnecting the plurality of nodes402ato402s. The plurality of nodes402ato402comprises a start node402aand at least one end node402nto402s.

At block1002b, data910representing at least one target outcome measure for the scenario is obtained, for example, being received over a network interface, retrieved from storage, or specified by a user through a user interface.

At block1002c, a pathway in the graph model400is selected from the start node to at least one end node402nto402sis selected. The selection is based on a comparison of at least one outcome measure determined for the selected pathway and at least one target outcome measure. The pathway which is selected represents an operating procedure for the aircraft control system100.

The above embodiments are to be understood as illustrative examples of the invention. Further embodiments of the invention are envisaged. It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the invention, which is defined in the accompanying claims. It is to be noted that the term “or” as used herein is to be interpreted to mean “and/or”, unless expressly stated otherwise.