Systems and methods for command management

Systems and methods capable of increasing reliability of received commands across a variety of different kinds of input devices and modalities are provided. The provided systems and methods easily expand to support additional input devices, and easily adapt to a wide variety of command destinations, such as subsystems and components. The provided systems and methods employ command specific verification strategies before transmitting the command to its destination. The provided systems and methods also concurrently support a wide variety of command destinations, such as subsystems and components.

TECHNICAL FIELD

The technical field generally relates to command management systems, and more particularly relates to systems and related operating methods for managing commands received via multiple input devices employing different modalities.

BACKGROUND

User input devices have proliferated into a variety of different kinds that each employ a different mode of communication, or modality. Based, at least in part, on the state of the technology used for each modality, the reliability of the different kinds of user input devices is uneven. As used herein, the reliability of a user input device is a proxy for a confidence that a command received from a user input device reflects the user's intent. As a first example, a physical switch or lever is a user input device that a user must manipulate (and sometimes even break a glass cover to access) in order to provide input. The physical switch or lever employs a first tactical modality and is generally considered a highly reliable user input device. Another tactile modality is employed by a touch sensitive screen. In another example, a speech recognition device is a user input device that a user speaks into to provide input. The speech recognition engine employs a voice modality, and is generally considered to have a lower reliability than input devices utilizing tactile modalities. As is readily appreciated, a variety of other user input devices, with corresponding modalities, are available, each having a respective reliability.

When an input device is configured to provide a command to a system and the command is an “action command,” meaning that it triggers an action by the system, the reliability of the input device may be relevant. Further, commands may be differentiated along a criticality scale between those that trigger rather trivial actions (non-critical commands) to those that trigger actions postulated to affect safety (highly critical commands). As may be apparent, a variety of activities may be considered an “action” responsive to an action command, and commands of higher criticality require more reliable input devices.

A technological problem is presented when a complex system is configured to concurrently receive user input from multiple different input devices of varying reliabilities. In this scenario, the complex system may comprise a plurality of subsystems, each responsive to multiple commands of varying criticality. To prevent these complex systems from triggering an action responsive to an unintentional command, conventional solutions often include a separate reliability component and verification strategy for each input device, which generally requires many interfaces, one dedicated to each user input device. Solutions of this type can be real estate intensive and unfavorably increase testing complexity and quality assurance procedures, any of which can be ominous for platforms that are sensitive to cost and weight.

Accordingly, systems and methods that address these technological problems are desirable. The desirable systems and methods easily expand to support additional input devices, and easily adapt to a wide variety of command destinations, such as subsystems and components. The desirable systems and methods employ command specific verification strategies before transmitting a command to its destination. The following disclosure provides an unconventional solution to these technological problems, in addition to introducing additional novel features.

BRIEF SUMMARY

A command management system is provided, the system comprising: a memory device; and a processor coupled to the memory device, the processor configured to: (a) determine a number of supported commands; (b) associate each supported command with a respective criticality level; (c) assign each input device of the plurality of input devices a respective reliability score; (d) receive a first received command from a first input device of the plurality of input devices; (e) generate a request reliability score for the received command based on the reliability score of the first input device; (f) compare the request reliability score to the criticality level associated with the first received command; (g) transmit the command responsive to determining that the request reliability score is >=the criticality level.

A method for a command management system is also provided, the method comprising: in a reliability engine, determining a number of supported commands; associating each supported command with a respective criticality level; assigning a respective reliability score to each input device of a plurality of input devices in communication with the reliability engine; receiving a first command from a first input device of the plurality of input devices; generating a request reliability score for the received command based on the assigned reliability score of the first input device; comparing the request reliability score to the criticality level associated with the command; and transmiting the command responsive to determining that the request reliability score is >=the criticality level.

In addition, an aircraft is provided, comprising: a plurality of user input devices; a command management system coupled to the plurality of input devices and to a plurality of command destinations, the command management system comprising a memory device, and a processor coupled to the memory device, the command management system configured to: (a) determine a number of supported commands associated with the command destinations; (b) associate each supported command with a respective criticality level; (c) assign each input device of the plurality of input devices a respective reliability score; (d) receive a first received command from a first input device of the plurality of input devices; (e) generate a request reliability score for the received command based on the assigned reliability score of the first input device; (f) compare the request reliability score to the criticality level associated with the first received command; (g) determine that the request reliability score is >=the criticality level, and transmit the command.

DETAILED DESCRIPTION

The following detailed description is merely illustrative in nature and is not intended to limit the embodiments of the subject matter or the application and uses of such embodiments. As used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Thus, any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments. All of the embodiments described herein are exemplary embodiments provided to enable persons skilled in the art to make or use the invention and not to limit the scope of the invention that is defined by the claims. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding technical field, background, brief summary, or the following detailed description.

As used herein, the term module refers to any hardware, software, firmware, electronic control component, processing logic, and/or processor device, individually or in any combination, including without limitation: application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and memory that executes one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality. The provided command management system and method may take the form of a command management module (FIGS. 1, 104), and may be separate from, or integrated within, a preexisting mobile platform management system or aircraft flight management system (FMS).

As mentioned, when a command is received via any user input device among a plurality of different kinds of user input devices, a technological problem to be solved is determining whether the received command matches a user's intention before transmitting the command to its destination (command destinations are generally subsystems and components). In addition, depending upon the destination for the command and the action to be triggered, the command may have varying levels of criticality (for example, a subsystem that controls climate may respond to commands that have low criticality for an overall system and a subsystem that controls speed brakes may respond to commands that have a high criticality for the overall system). The below described command management system and method employ novel real time reliability scoring and criticality matching rules and algorithms to address these technological problems. The below description provides more detail as to these functions.

Turning now toFIG. 1, a functional block diagram of an aircraft100comprising a command management system102is depicted, in accordance with various exemplary embodiments. Although the depicted embodiment realizes the command management system102within the aircraft100, the concepts presented here can be deployed in a variety of mobile and non-mobile platforms, such as vehicles, spacecraft, watercraft, buildings, open-air spaces, and the like.

In the depicted embodiment, the command management system102includes: a command management module104that receives commands from a plurality of user input devices106, via bus51and transceiver108. In operation of the aircraft100, it is desirable to command and control a variety of subsystems and components, referred to herein collectively as command destinations120. The command management module104transmits a command, via bus53, to a command destination120responsive to satisfactory processing and transformation by the command management module104. The operation of these functional blocks is described in more detail below.

As mentioned, commands are received from the user input devices106. In various embodiments, the user input devices106include any one, or combination, of various known user input device devices, including, but not limited to: any cursor control device (CCD), such as a mouse, a trackball, or joystick; a keyboard; one or more buttons, switches, or knobs; a voice input system; and a gesture recognition system. In the embodiment depicted inFIG. 1, the user input devices106include cursor control devices (CCD)110, speech recognition devices112, and touch screen devices114. Non-limiting examples of uses for the user input devices106include: entering values for stored variables164, loading or updating instructions and applications160(including program162), providing confirmations and/or cancellations for commands, and loading and updating the contents of the database156, each of which is described in more detail below.

Regardless of the user input device106, the command provided by each input device106is understood to have a time component that demarks the time at which the command was provided on bus51, and may further carry with it context information, such as whether the respective user input device106is associated with a pilot, a crew, or the like. Context information may also be detected by the command management module104. As may be readily appreciated, any combination of the variety of user input devices106may provide commands sequentially or concurrently. The command management system102is capable of continually: receiving commands, distinguishing among received commands, and transforming and processing them further, as described in more detail below.

The transceiver108may include at least one receiver and at least one transmitter that are operatively coupled to the command management module104. The transceiver108can enable the command management module104to establish and maintain the communications links to onboard components (not shown), and the user input devices106. The transceiver108can support wired and a variety of types of wireless communication, and can perform signal processing (e.g., digitizing, data encoding, modulation, etc.) as is known in the art. In some embodiments, the transceiver108is integrated with the command management module104.

The subsystems and components comprising the command destinations120are application specific. There can be just one command destination120or a plurality of command destinations120. When there is a plurality of command destinations120, there may or may not be multiple of a given kind of command destination120(for example, multiple speakers). Bus53is used to transmit a command to its destination, and may be used to return communication back to the command management module104. Bus53, like bus51, may represent wired or wireless communication. InFIG. 1, the command destinations120include, a flight management system (FMS)130, an autopilot system132, and one or more navigation systems (NAV)134(NAV134may include an area navigation RNAV, a lateral navigation LNAV, and a vertical navigation VNAV). In other embodiments, particularly those that are not avionics-related, additional destinations for commands may be added, and some of the above listed destinations may be deleted. Regardless of the type or the configuration of a given component with the command destination120, each component within the command destination120is a subsystem or component that can be triggered to perform at least one action responsive to receiving a command.

The command management module104performs the functions of the command management system102. With continued reference toFIG. 1, within the command management module104, the processor150and the memory device152form a fusion engine and a reliability handling engine that perform the processing activities. In an embodiment, the fusion engine determines when one or more received commands are related and fuses them, and the reliability handling engine manages the verification strategies described herein. The fusion engine and reliability handling engine are collectively referenced herein as a “reliability engine” for simplicity. The reliability engine provides a technological improvement over limitations of conventional command management solutions, in part, by conditioning and filtering received commands with real time reliability scoring and criticality matching prior to transmitting a command. These concepts are described in more detail below.

The command management module104also includes an interface154, communicatively coupled to the processor150and memory device152(via a bus155), database156, and an optional storage disk158. The processor150may comprise any type of processor or multiple processors, single integrated circuits such as a microprocessor, or any suitable number of integrated circuit devices and/or circuit boards working in cooperation to carry out the described operations, tasks, and functions by manipulating electrical signals representing data bits at memory locations in the system memory, as well as other processing of signals.

A computer readable storage medium, such as a memory device152, the database156, or a disk158may be utilized as both storage and a scratch pad. The memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties corresponding to the data bits. The memory device152can be any type of suitable computer readable storage medium. For example, the memory device152may include various types of dynamic random access memory (DRAM) such as SDRAM, the various types of static RAM (SRAM), and the various types of non-volatile memory (PROM, EPROM, and flash). In certain examples, the memory device152is located on and/or co-located on the same computer chip as the processor150. In the depicted embodiment, the memory device152includes the above-referenced instructions and applications160, the program162, stored variables164. Program162comprises rules and instructions sufficient to create, in cooperation with the processor150, the reliability engine and the command management module104. In various embodiments, the command management module104performs actions and functions in accordance with steps of the method200shown inFIG. 2.

The database156is a computer readable storage medium in the form of any suitable type of storage apparatus, including direct access storage devices such as hard disk drives, flash systems, floppy disk drives and optical disk drives. In one exemplary embodiment, the database156stores a lookup table that, for every command supported by the command management module104, associates a criticality level with the command.

As mentioned, commands for the various command destinations120may have various criticality levels associated with the action that they trigger. An assigned criticality level is used to distinguish safety-critical commands from non safety-critical commands; it is a value on a predefined scale, similar to various industry standard scales generated by standard setting groups. Examples of similar industry standard scales include a “Design Assurance Level” (DAL), and an Item Development Assurance Level (IDAL) scale. The criticality scale and the values along the scale may be application specific and predefined. In various embodiments, the criticality scale may be configured at an initialization step (FIG. 2at202). Non-limiting examples of a criticality scale include scales that have three levels: low (minimal effect), medium (significant impact), and high (hazardous effect), and scales that have five levels: low (no effect), medium-low (minor effect), medium (major effect), medium high (hazardous effect), and high (catastrophic effect).

As may be apparent at this point, each command may be distinguished by (i) the action that it triggers, and (ii) the destination or subsystem to which it is to be transmitted. The criticality level associated with each command may reflect (i) and/or (ii). In addition to an associated criticality level, supported commands may be further distinguished with respect to the nature of the action that they trigger. In various embodiments, each command may be further associated with one or more corresponding flags that indicate that the command has (i) immediate effect (IE flag), (ii) a significant impact (SI flag), and (iii) undo functionality (undo flag). In an embodiment, an asserted flag is a logical one and a deasserted flag is a logical zero, but other methods for differentiating these features may be employed. Therefore, in storage, in either memory device152or the database156, a command table may look like Table 1, below. These flags may be employed to further condition a command before transmitting it to a command destination120, their use is described in more detail in connection with the method steps inFIG. 3.

The bus155serves to transmit programs, data, status and other information or signals between the various components of the command management module104. The bus155can be any suitable physical or logical means of connecting computer systems and components. This includes, but is not limited to, direct hard-wired connections, fiber optics, infrared and wireless bus technologies. During operation, the program162, stored in the memory device152, is loaded and executed by the processor150.

The interface154enables communications within the command management module104, and can include one or more wired or wireless network interfaces to communicate with external systems or components. Interface154can be implemented using any suitable method and apparatus. For example, the interface154enables communication from a system driver and/or another computer system. In one embodiment, the interface154obtains a command from the user input devices106directly. The interface154may also include one or more network interfaces for communication with technicians, and/or one or more storage interfaces for connecting to computer readable storage, such as the database156.

During operation, the processor150loads and executes one or more programs, algorithms and rules embodied as instructions and applications160contained within the memory device152and, as such, controls the general operation of the command management module104as well as the command management system102. In executing the process described herein, such as the method200ofFIG. 2, the processor150loads and specifically executes the program162, to thereby realize an unconventional technological improvement to conventional command management systems. Additionally, the processor150is configured to process a received command, generate a request reliability score (RR Score) associated with the received command, reference the database156in accordance with the program162, and transmit a command via bus53to a command destination120based thereon. In addition to being demarked with a time, each received command may optionally include context information, for example a flag or other indicator that the received command is coming from a pilot or coming from a crew.

As mentioned, the processor150and the program162form a reliability engine that continually, and in real time, (i) generates a request reliability score (RR score) for each received command, (ii) searches database156for the received command, and (iii) transmits the command only when the RR score is greater than or equal to a criticality level associated with the command. A method200for command management is provided in connection withFIG. 2.

It will be appreciated that command management system102may differ from the embodiment depicted inFIG. 1. As a first example, in various embodiments, the command destinations120can be any system or sub-system supporting a pilot while operating the aircraft100. In addition, any combination of the user input devices106can be integrated, for example, as part of a console. Regardless of the state of integration of these systems, a user may control one or more features of the command management system102by providing user input via any user input device within the user input devices106.

Referring now toFIG. 2and with continued reference toFIG. 1, a flow chart is provided for a method200for a command management system102, in accordance with various exemplary embodiments. Method200represents various embodiments of a method associated with the command management system102. For illustrative purposes, the following description of method200may refer to elements mentioned above in connection withFIG. 1. In practice, portions of method200may be performed by different components of the described system. It should be appreciated that method200may include any number of additional or alternative tasks, the tasks shown inFIG. 2need not be performed in the illustrated order, and method200may be incorporated into a more comprehensive procedure or method having additional functionality not described in detail herein. Moreover, one or more of the tasks shown inFIG. 2could be omitted from an embodiment of the method200as long as the intended overall functionality remains intact.

The method starts, and at202the command management module104is initialized. As mentioned above, initialization may comprise uploading or updating instructions and applications160, program162, stored variables164, and the various lookup tables stored in the database156. Generally, predetermined variables include, for example, the reliability and criticality scale values, any default times associated with determining that commands go together and/or that commands are verified for their command destination120, and the like. In an embodiment, at202, the method200initializes a command table, such as Table 1. Initialization also comprises determining or uploading a criticality scale and a reliability scale. At204a number N, of unique supported commands is determined. Recall, the unique supported commands are associated with the various components or subsystems within the command destinations120. Determining the supported commands involve a configuration program contained within program162, may involve loading supported commands into the command management module104at initialization, or may involve the command management module104individually assessing the command destinations120that are in communication with the command management module104. At206, each supported command is associated with a respective criticality level. As mentioned, the N commands may also be associated with an IE flag, SI flag, and undo flag, resulting in, for each supported command of a plurality (N) of supported commands, a row or line in the command table, such as is shown in Table 1 above. At208, each user input device106(i.e.,110,112,114, etc.) is assigned with a respective reliability score. In some embodiments, the reliability score may match the criticality scale. In other embodiments, the reliability score is on a scale of different extent and/or demarcations from the criticality scale; in these embodiments, the program162comprises the instructions and rules to transform the reliability score for compatible use with the criticality scale.

At210, a command is received by the command management module104, from one of the user input devices106. The command management module104(specifically the fusion engine) also recognizes when a command comprises a fusion of partial commands received from more than one of the user input devices106. The command management module104recognizes that a first received command and a second received command are related, and fuses them. For example, the pilot could tap a waypoint on a touchscreen device114having a reliability score of 5, and speak “delete this waypoint” into a speech recognition engine having a reliability score of 2.

At212, a request reliability score (RR Score) is generated. The RR Score is generated to represent the reliability that the received command matches the user's actual intention. For this reason, the RR Score is based on the reliability of the input device(s) of the user input devices106from which the received command was received. As mentioned, each user input device106may have a different modality and reliability score. When more than one user input device106and modality is used, the input device having the lowest reliability score prevails. In the example above, for “delete this waypoint,” the reliability scores of the two input devices are compared. The lowest reliability score prevails for the command. The Request reliability score (RR Score) for the fused command is then 2.

At214, the received command is located in the command table, and the request reliability score (RR Score) is compared to the criticality level (CL) for the command at216. When the RRS is determined to be >=the CL, at218the command is transmitted to a command destination120(As denoted by the “A” in the method step218, transmittal at218may be further conditioned upon events described in connection withFIG. 3, below). At220(when the RR Score is determined to be <the CL), the received command is not transmitted. In the example above, for “delete this waypoint,” if the criticality level of the command “delete this waypoint” is 7, or anything over 2, the command will not be transmitted. After completion of218and/or220, at “B,” the process may return to210or end.

Turning now toFIG. 3, and with reference to Table 1, a flow chart showing an expansion “A” of the method step218is depicted, in accordance with various embodiments. As mentioned, transmittal of a command at218may be conditioned upon further events or determinations. Each of the paths depicted inFIG. 3illustrate a different verification strategy for a received command. As described, the supported commands may vary from fairly trivial to extremely significant. The provided verification strategies illustrate an exemplary embodiment that distinguishes between commands; multiple other verification strategies may be implemented without straying from the novel concepts presented herein. At302, it is determined whether or not the command has immediate effect (IE), at304, it is determined whether or not the command has significant impact (SI), and at306, it is determined whether or not the command has undo functionality. Each determination (302,304, and306) may have an associated predetermined definition that is stored in memory device152, database156, or uploaded during initialization at202. For example, IE may be defined as happening in less than a second, and SI may be defined as a criticality level of high (or a seven out of ten). Further, these definitions may vary from command to command, and/or may be based on the command's destination120. For commands determined to meet both of the conditions: (i) have IE, and (ii) do not have SI at308, a notification of the command is transmitted to a user at a user input device (110,112,114) at316, the method waits for a predetermined delay time to elapse at318, and if a cancellation is not received (via55) from the user at the end of the elapsed delay time, the command is transmitted.

At314, if the received command is determined to meet both of the conditions: (i) does not have IE, and (ii) has undo functionality, a notification of the command is generated and transmitted to the user at a user input device (110,112, and114) at328, and the command is transmitted to the command destination120at330. In various embodiments,328and330may be in reverse order, or be concurrent.

When it is determined that the received command (a) meets both of the conditions: (i) has IE, and (ii) has SI (310), or that the received command (b) meets both of the conditions: (i) does not have IE, and (ii) does not have undo functionality (312), the method300may require a confirmation from the user that the command is intentional. Coming from310or312, the type of confirmation required at322and the modality for the required confirmation for a command may vary, and variations may be command-specific or command destination120specific. In various embodiments, the confirmation may be required to be received via a high reliability user input device and/or modality, such as a hard button that has to be manipulated, rather than another voice command. A request for the confirmation from the user may be transmitted to a user input device (110,112,114). At324, a confirmation of the received command is received, and at326, if the received confirmation matches the requirements of the confirmation from322, the confirmation is considered verified; the command transmitted at330only when the command is verified.

As is readily appreciated, the above examples are non-limiting, and many others may be addressed the command management module104. Thus, command management systems and methods capable of increasing reliability of received commands over a wide variety of input devices and modalities have been provided.

While at least one exemplary embodiment has been presented in the foregoing detailed description of the invention, it should be appreciated that a vast number of variations exist. It should also be appreciated that the exemplary embodiment or exemplary embodiments are only examples, and are not intended to limit the scope, applicability, or configuration of the invention in any way. Rather, the foregoing detailed description will provide those skilled in the art with a convenient road map for implementing an exemplary embodiment of the invention. It being understood that various changes may be made in the function and arrangement of elements described in an exemplary embodiment without departing from the scope of the invention as set forth in the appended claims. It will also be appreciated that while the depicted exemplary embodiment is described in the context of a fully functioning computer system, those skilled in the art will recognize that the mechanisms of the present disclosure are capable of being distributed as a program product with one or more types of non-transitory computer-readable signal bearing media used to store the program and the instructions thereof and carry out the distribution thereof, such as a non-transitory computer readable medium bearing the program136and containing computer instructions stored therein for causing a computer processor (such as the processor150) to perform and execute the program136. Such a program product may take a variety of forms, and the present disclosure applies equally regardless of the particular type of computer-readable signal bearing media used to carry out the distribution. Examples of signal bearing media include: recordable media such as floppy disks, hard drives, memory cards and optical disks, and transmission media such as digital and analog communication links. It will be appreciated that cloud-based storage and/or other techniques may also be utilized in certain embodiments.