AUGMENTING PILOTED DISPLAY DATA WITH PROCESSED DATA

Systems and methods for providing situational awareness to pilots of an aircraft. One example system includes an electronic processor configured to receive a raw sensor value for an operational characteristic of the aircraft and generate a processed sensor value based on the raw sensor value. The electronic processor is configured to provide the processed sensor value to a flight control system. The electronic processor is configured to generate a first digital gauge object that includes a graphical representation of the raw sensor value. The electronic processor is configured to generate a second digital gauge object that includes a graphical representation of the processed sensor value and a graphical representation indicating that the processed sensor value is being displayed. The electronic processor is configured to present the first digital gauge object on a first display of the aircraft and the second digital gauge object on a second display of the aircraft.

FIELD

Embodiments described herein relate to pilot-configurable aircraft instrumentation.

SUMMARY

Aircraft are equipped with sensors to measure operational characteristics including altitude and airspeed. Applicable civil regulations and military requirements mandate that certain types of aircraft sensor data must be presented on the flight displays of the aircraft for the pilots. In some cases, this sensor data must be from a single sensor source and must be presented in a raw format. As used herein, the terms “raw,” “raw data,” and “raw sensor data” refer to sensor data that is substantially unprocessed. However, many modern aircraft incorporate Fly-by-Wire (FBW) flight control systems. In such systems, it is common for the control algorithms that control the aircraft to utilize processed sensor data to improve redundancy, ride quality, and performance. As used herein, the terms “processed,” “processed data,” and “processed sensor data” include any combination of processing applied to the sensor data that noticeably alters the data statically and dynamically. Examples of such processing include filtering, inertial filtering, sensor fusion, and selection algorithms (also known as voting). Consequently, the processed parameters utilized by the flight control system may be dynamically and statically different than the unprocessed sensor data required to be displayed.

Because of these differences between unprocessed and processed values, it may appear to the pilots as if the flight control system is not performing as expected, particularly when asked to hold to a value (e.g., an altitude, a velocity, a heading, or an attitude). For example, a helicopter that performs naval operations typically has a radar altitude hold mode that is intended to keep the aircraft at a fixed height above the surface as measured by a radar altimeter sensor. However, an ocean surface dynamically varies due to waves, causing the raw sensor data from the radar altimeter to continually fluctuate. Direct use of this raw radar altimeter data is not desirable as it would continually raise and lower the helicopter height, referred to as “chasing waves.” Therefore, the radar altitude hold mode data would be processed, and potentially have the radar altimeter data inertially filtered, to improve ride quality by having the helicopter hold height above the water without actively chasing high waves. Traditionally, the pilot may see the unprocessed radar altimeter reading on the display dynamically changing as the waves pass underneath, while the radar altitude hold mode holds the aircraft much more steady inertially. This can give the impression that the flight controls are not accurately holding radar altitude because the pilot has no insight into either the actual radar altitude sensor solution within the flight control computer or the flight control performance.

Accordingly, examples described herein provide aircraft instrumentation that explicitly display the data being used to actively control the aircraft, while maintaining the ability for the crew to display independent raw sensor data within the cockpit. In this way, if a pilot sees any unexpectedly displayed differences between the held and requested sensor data, the pilot can select the voted and processed sensor data to confirm that the flight control system is performing as expected and that the dynamic and static differences observed using an independent unprocessed sensor data is to be expected. Some examples also provide for display of such information on each side of the cockpit and indications of when both sides are displaying the same source. Some examples also provide for cross cockpit comparison of displayed data to alert to crew that the displays are diverging beyond a threshold. Some examples also provide for the display to the pilot of voted data, to provide confidence to the pilot that the processed data represents the mutual agreement of multiple processors and sensors.

The examples and instances presented herein improve pilot situational awareness, instill confidence that the flight control system is operating as expected, and retain the ability for the crew to display independent unprocessed sensor data, keeping compliance with applicable regulations.

DETAILED DESCRIPTION

One or more embodiments are described and illustrated in the following description and accompanying drawings. These embodiments are not limited to the specific details provided herein and may be modified in various ways. Furthermore, other embodiments may exist that are not described herein. Also, the functionality described herein as being performed by one component may be performed by multiple components in a distributed manner. Likewise, functionality performed by multiple components may be consolidated and performed by a single component. Similarly, a component described as performing particular functionality may also perform additional functionality not described herein. For example, a device or structure that is “configured” in a certain way is configured in at least that way but may also be configured in ways that are not listed. Furthermore, some embodiments described herein may include one or more electronic processors configured to perform the described functionality by executing instructions stored in non-transitory, computer-readable medium. Similarly, embodiments described herein may be implemented as non-transitory, computer-readable medium storing instructions executable by one or more electronic processors to perform the described functionality. As used in the present application, “non-transitory computer-readable medium” comprises all computer-readable media but does not consist of a transitory, propagating signal. Accordingly, non-transitory computer-readable medium may include, for example, a hard disk, a CD-ROM, an optical storage device, a magnetic storage device, a ROM (Read Only Memory), a RAM (Random Access Memory), register memory, a processor cache, or any combination thereof.

In addition, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. For example, the use of “including,” “containing,” “comprising,” “having,” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. The terms “connected” and “coupled” are used broadly and encompass both direct and indirect connecting and coupling. Further, “connected” and “coupled” are not restricted to physical or mechanical connections or couplings and can include electrical connections or couplings, whether direct or indirect. In addition, electronic communications and notifications may be performed using wired connections, wireless connections, or a combination thereof and may be transmitted directly or through one or more intermediary devices over various types of networks, communication channels, and connections. Moreover, relational terms such as first and second, top and bottom, and the like may be used herein solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. As used herein, the terms “graphical representation” and “display,” as they pertain to representing or displaying a value is not meant to limit the graphical form used to communicate the value. It could include, for example, numerical text, dials, bars, colors, and the like. It should also be understood that graphical representation or display of a value can communicate the value in absolute terms, such as a number corresponding to a first aircraft sensor value, or it may communicate the value in relative terms, such as the difference between the first aircraft sensor value and a second reference value.

Embodiments and examples described herein provide, among other things, aircraft instrumentation that explicitly display the voted and processed data being used to actively control the aircraft, while maintaining the ability for the crew to display independent “raw” sensor data on opposite sides of the cockpit.

In particular, one example describes a system for providing situational awareness to a pilot of an aircraft. The system includes an electronic processor. The electronic processor is configured to receive a raw sensor value for an operational characteristic of the aircraft. The electronic processor is configured to generate a processed sensor value based on the raw sensor value. The electronic processor is configured to provide the processed sensor value to a flight control system of the aircraft. The electronic processor is configured to generate a first digital gauge object based on the operational characteristic of the aircraft, wherein the first digital gauge object includes a graphical representation of the raw sensor value. The electronic processor is configured to generate a second digital gauge object based on the operational characteristic of the aircraft, wherein the second digital gauge object includes a graphical representation of the processed sensor value and a graphical representation indicating that the processed sensor value is being displayed. The electronic processor is configured to present, on a first display of the aircraft, the first digital gauge object. The electronic processor is configured to present, on a second display of the aircraft, the second digital gauge object.

Another example describes a method for providing situational awareness to a pilot of an aircraft. The method includes receiving a raw sensor value for an operational characteristic of the aircraft. The method includes generating a processed sensor value based on the raw sensor value. The method includes providing the processed sensor value to a flight control system of the aircraft. The method includes generating a first digital gauge object based on the operational characteristic of the aircraft, wherein the first digital gauge object includes a graphical representation of the raw sensor value. The method includes generating a second digital gauge object based on the operational characteristic of the aircraft, wherein the second digital gauge object includes a graphical representation of the processed sensor value and a graphical representation indicating that the processed sensor value is being displayed. The method includes presenting, on a first display of the aircraft, the first digital gauge object. The method includes presenting, on a second display of the aircraft, the second digital gauge object.

FIG.1illustrates an example system100for providing aircraft instrumentation and control. According to the example illustrated inFIG.1, the system100is integrated into an aircraft105. The aircraft105includes a controller110, an avionics system (AVS)130, a pilot display150, and a co-pilot display155. The components of the system100are communicatively coupled. The controller110is an electronic controller, which may include an electronic processor115and a memory120. The memory120may be a non-transitory computer-readable memory. The memory120may include one or more types of memory storage, such as random-access memory (RAM), flash memory, solid-state memory, or hard-drive memory. In addition, or alternatively, the controller110may communicate with a cloud-based storage system. The controller110is typically implemented using well known redundancy mechanisms to guard against equipment failure within a single controller. Redundant controllers can employ a “voting” mechanism to determine the result to be used for further processing or display.

As illustrated, the AVS130may include a plurality of sensors (e.g., sensors140A,140B,140C,140D). Sensor types include Air Data system sensors (e.g., for sensing altitude and velocity), height above terrain sensors, and attitude/heading data. Each sensor generates a signal representing a measured operating characteristic of the aircraft105(e.g., altitude, velocity, position, attitude, heading, etc.) and transmits the signal to the controller110(directly or indirectly). The controller110receives and processes the signals. As described herein, the controller displays the measured characteristics received from the sensors on the pilot display150, the co-pilot display155, or both. For example, the controller may generate and display virtual gauges as part of a graphical user interface. In some instances, the raw sensor data is fed to the controller110and displayed in parallel, rather than the display data receiving raw sensor data from the controller110. In such instances the pilot display150and the co-pilot display155are configured to display raw data, processed data, or both, regardless of the data path to the displays.

In some instances, the controller110is configured to operate the aircraft using fly-by-wire technology. In some instances (e.g., to improve redundancy, ride quality, and/or aircraft performance), the controller110is configured to utilize multiple sensors to sense a single operating characteristic of the aircraft105. The controller110is configured process the data received from the sensors to determine an operating characteristic value that will be used by the fly-by-wire (FBW) flight control system during aircraft operation. For example, the controller110may implement a voting algorithm to select a sensor signal from among multiple received sensor signals. In another example, the controller110may process the sensor signals (individually or as a group) to normalize the data, to account for environmental or other factors, or to otherwise produce a more desirable sensor output. In some instances, a combination of approaches is used to generate the operating characteristic value used by the FBW flight control system. This value may vary from the raw sensor data received from each of the sensors. As described herein, the controller110is configured to display raw sensor data, processed sensor data (used by the FBW flight control system), or both.

It should be noted that, while voting and processing of sensor data is common in FBW controllers, it is also possible that traditional mechanical-based autopilot systems will significantly process the raw data and cause similar differences between the controlled sensor values and the raw sensor data. The systems and methods disclosed herein are also applicable to such mechanical flight control system autopilots, as well as to other technologies (e.g., fiber optics based fly-by-light systems).

In some embodiments, such as the embodiment illustrated inFIG.1, the pilot display150and the co-pilot display155are integrated into the aircraft105. For example, the pilot display150and the co-pilot display155may be electrically coupled to the controller110, coupled to an instrument panel of the aircraft105, or included in the AVS130. In some instances, the pilot display150, the co-pilot display155, or both may provide a user interface for an electronic flight bag (EFB) application. In some instances, the displays include user input capabilities, such as a touch screen. In some instances, the displays may be or may include a heads up display (HUD). In some instances, the system100operates using, among other things, augmented reality technology, where live images are displayed or visible through the displays and augmented with text, graphics, or graphical user interface elements superimposed on or otherwise combined with the live images. In some embodiments, the system100operates using, among other things, virtual reality technology, where actual or simulated images are displayed (for example, on the displays) with text, graphics, or graphical user interface elements superimposed on or otherwise combined with the images.

FIG.2illustrates an alternative system200for providing aircraft instrumentation and control. Unlike the system100illustrated inFIG.1, the system200ofFIG.2illustrates an alternative distributed configuration. The system200may include the aircraft105, the AVS130, and the sensors140A-140D of the system100ofFIG.1. The system200further includes a communication network205. The communication network205may be a Wi-Fi network, a cellular network, a Bluetooth network, a satellite network, or the like. The communication network205provides communicative coupling between the aircraft105and an external device250. The external device250may be a mobile device, such as a smart phone, a tablet computer, a laptop computer, or the like. In some instances, the external device250is a device separate from the internal systems of the aircraft105, but still physically located within the aircraft105. For example, in these instances the external device250may be a tablet computer, a mobile phone, or the like. In other embodiments, the external device250is located external to the aircraft105, for example in a control tower or in a ground control station, which may support remote operations.

The external device250includes a controller260, a display280, and a device transceiver290. The device transceiver290and display280may be electrically, mechanically, and/or communicatively coupled to the controller260. The controller260is an electronic controller, which may include a processor265and a memory270. The memory270may be a non-transitory computer-readable memory. The memory270may include one or more types of memory storage, such as random-access memory (RAM), flash memory, solid-state memory, or hard-drive memory. In addition, or alternatively, the controller260may communicate with a cloud-based storage system. The device transceiver290is configured to send and receive signals to the aircraft105via the communication network205.

In some instances, such as the example illustrated inFIG.2, the display280is integrated into the external device250, and not electrically or mechanically coupled to the aircraft105. For example, the display280may be electrically and communicatively coupled to a mobile device, such as a smart phone or tablet computer. In some instances, the display280is configured to operate as a duplicate of either the pilot display150or the co-pilot display155. In some instances, the display280operates as a replacement of either the pilot display150or the co-pilot display155. In some instances, the display280operates as a supplement to the pilot display150, the co-pilot display155, or both. In some instances, the display280includes user input capabilities, such as a touch screen. In some instances, the display280may be a head-mounted display (HMD), an optical head-mounted display (OHMD), or the display of a pair of smart glasses. In some instances, the external device250operates using, among other things, augmented reality technology, where live images are displayed or visible through the display280and augmented with text, graphics, or graphical user interface elements superimposed on or otherwise combined with the live images. In some instances, the external device250operates using, among other things, virtual reality technology, where actual or simulated images are displayed (for example, on the display280) with text, graphics, or graphical user interface elements superimposed on or otherwise combined with the images.

Furthermore, other variations than the system100shown inFIG.1and the system200shown inFIG.2are possible. For example, some instances may distribute the components of the system100or200across multiple devices.

FIG.3is a flowchart illustrating a method300for providing situational awareness to a pilot of an aircraft by explicitly displaying voted and processed sensor data being used to actively control the aircraft. The method300may be implemented on the system100ofFIG.1, the system200ofFIG.2, and/or a different system. As an example, the method300is described as being performed by the electronic processor115. However, the method300may be executed on one or more electronic processors according to examples described herein.

At block302, the electronic processor115receives a raw sensor value for an operational characteristic of the aircraft. For example, the electronic processor may receive a raw sensor value from one or more of the sensors140A-140D. In some aspects, one or more of the sensors140A-140D provide a signal (e.g., a voltage) to the electronic processor, which converts the signal to a value for the operational characteristic (e.g., an altitude in feet). In some aspects, the electronic processor receives a value for the operational characteristic derived from the sensor signal by, for example, intervening circuitry. Examples of operational characteristics of the aircraft include an altitude of the aircraft, a velocity of the aircraft, an attitude of the aircraft, a position of the aircraft, and a heading of the aircraft.

At block304, the electronic processor115generates a processed sensor value based on the raw sensor value in addition to other relevant information. For example, the electronic processor115may receive multiple raw sensor values and apply a selection algorithm to select one of the raw sensor values to use as the processed sensor value. In another example, the electronic processor115may receive multiple raw sensor values and apply a voting algorithm to arbitrate the results of multiple sensors such that the resulting value may be different than any single sensor value (e.g., average data). In another example, the electronic processor115may apply digital signal processing to the raw sensor value to produce the processed sensor value. In another example, electronic processor115may produce the processed sensor value using a rolling average or another suitable method to normalize the raw sensor data. In some aspects, the electronic processor115applies both a selection algorithm and processing to produce the processed sensor value. For example, the electronic processor115may select (using a selection algorithm) from one of multiple raw sensor values and apply processing to the selected raw sensor value to produce the processed sensor value. In another example, the electronic processor115may produce the processed sensor value by applying processing to several raw sensor values and then applying a voting or selection algorithm to select from among the several processed. In yet another example, the raw data may be significantly processed for display without being voted.

At block306, the electronic processor115provides the processed sensor value to a flight control system of the aircraft. This sensor value is used by the FBW flight control system to operate the aircraft.

At block308, the electronic processor115generates a first digital gauge object, based on the operational characteristic of the aircraft, that includes a graphical representation of the raw sensor value. A digital gauge object is a computer-generated dynamic image of an aircraft instrumentation gauge to be displayed to a pilot.FIG.4illustrates an example digital gauge object400representative of the prior art. The digital gauge object400is configured to display a radar altitude reading for the aircraft. The digital gauge object400includes a dial402, which provides a numeric scale. In the illustrated example, the current value of the radar altitude is displayed using two different graphical representations. The first graphical representation is a needle404, which is generated to overlay the dial at the point representing the current raw sensor value. The second graphical representation is a numeric indicator406, which displays the current raw sensor value using, for example, Arabic numerals.

In some instances, the fly-by-wire system may be configured to hold the aircraft automatically, for example, at a particular reference value. In some aspects, the electronic processor115determines (e.g., by receiving an input from the flight control system) a reference value for the operational characteristic associated with the digital gauge and displays the reference value as well. In the illustrated example, the reference value is an altitude of 225 feet. In the illustrated example, the reference value is displayed using two different graphical representations. The first graphical representation is a notch408, which is generated to overlay the dial at the point representing the current reference value. The second graphical representation is a numeric indicator410, which displays the current reference value using, for example, Arabic numerals. In the prior art, minor variations to the raw sensor data, such as variations due to ocean waves, will cause continual variations in the display of the sensed value. Thus the needle404and indicator406may continually vary due to ocean wave height. For example, as illustrated inFIG.4, the needle404and the numeric indicator410are displaying a current raw sensor data value of 235 feet (caused, for example, by a 10 foot wave under the aircraft while hovering above water), while the reference value is 225 feet. Such variations in the displayed sensed value may cause pilot confusion of whether the altitude sensors are working properly.

At block310, the electronic processor115presents, on a first display of the aircraft (e.g., the pilot display150), the first digital gauge object.

At block312, the electronic processor115generates a second digital gauge object, based on the operational characteristic of the aircraft, that includes a graphical representation of the processed sensor value and a graphical representation indicating that the processed sensor value is being displayed.FIG.5illustrates an example second digital gauge object. The second digital gauge object is similar to the first, except that the second digital gauge object does not display the raw sensor value, but instead includes a graphical representation of the processed sensor value, which may be the needle502, the numeric indicator504, or both. The display of the processed sensor value effectively reduces the fluctuations of the raw sensor values. To improve pilot situational awareness, the second digital gauge object also includes a graphical representation indicating that the processed sensor value is being displayed. The second digital gauge object may also include a graphical representation indicating the confidence of the sensor value being displayed, to instill confidence in the pilot, the processed result may be indicated as “VOTED” to indicate it is agreed to by multiple redundant processors and sensors. In the illustrated example, the graphical representation is a textual indicator506, which informs the pilot that the displayed sensor data is “VOTED.” In some examples, the graphical representation indicating that the processed sensor value is being displayed may be an icon, a shading of the gauge in a particular color, a change in color of one or more elements of the digital gauge object, a pulsating of the graphical representation of the processed sensor value, an alternate word (e.g., “PROCESSED”, “MISMATCH”), an outline, a glow, or another suitable means of highlighting to the pilot that processed sensor data is being displayed and if there is a potential sensor error.

At block314, the electronic processor115presents, on a second display of the aircraft (e.g., the co-pilot display), the second digital gauge object.

In the example described above, a pilot may be provided a radar altimeter gauge displaying the raw sensor data, while a co-pilot is provided a radar altimeter gauge displaying the processed sensor data that is used by the flight control system. However, in some instances, a pilot or co-pilot may wish to select for themselves which data source to use. In some aspects, the avionics system130provides user inputs that allow pilots and/or co-pilots to select the data source to be displayed by a gauge. For example, the electronic processor115may generate a graphical user interface including touch-screen controls. In another example, existing physical controls (e.g., switches, buttons, knobs, and the like) may be configured such that their input reflects a pilot preference for a sensor data type.

In some instances, the electronic processor115may receive a user input indicative of either one of the first display or the second display, a sensor data type, and the operational characteristic of the aircraft. For example, a pilot may select a control presented on a graphical user interface, which indicates that they would like to see processed sensor data on their display. The electronic processor115, in response to receiving this user input, will display the second digital gauge object on the pilot's display.

In some aspects, where multiple raw sensor values are available, the electronic processor115may be configured to produce alternate digital gauge objects to display the raw sensor values from different redundant sensors. For example, the electronic processor115, where a second raw sensor value is available, may generate a third digital gauge object, based on the operational characteristic of the aircraft, which includes a graphical representation of the second raw sensor value. In some aspects, where multiple raw sensor values are available, the avionics system130provides user inputs that allow pilots and/or co-pilots to select the data source to be displayed by a gauge (e.g., a raw sensor value, a second raw sensor value, or a processed sensor value).

As noted, applicable regulations may require that at least one source of raw sensor data is presented to the pilots of an aircraft. Accordingly, in some aspects, the electronic processor115monitors the displays of the aircraft and the user inputs to determine whether at least one of the displays is displaying the raw sensor data (e.g., using the first or third digital gauge objects). In some aspects, the electronic processor115is configured to generate an alert (e.g., a visual alert, and audible alert, a haptic alert, and the like) when it determines that any of the first digital gauge object, the second digital gauge object, and the third digital gauge object is displayed on both of the co-pilot display and the pilot display. Such an alert ensures that both pilots are aware that they are viewing the same data source and may be used to enforce redundancy (e.g., by alerting the pilots that the crew is viewing the same source so they can take action if needed to view independent sensor data).

In some instances, the systems described herein may provide to a pilot or pilots awareness beyond observing the type of data displayed (e.g., voted, raw, processed, and the like). For example, in some instances the electronic processor115is configured to generate an alert indicate that there is a significant difference for that specific sensor type between what the pilot is viewing and one of the other sources of that data (e.g., other raw or processed data), for example, where the difference exceeds a threshold value (e.g., 5 knots for airspeed, 20 feet for altitude, and the like). For example, the electronic processor115may be configured to display a gauge or sensor label with an underline, a color change, in a special font, or in another way such as a dedicated indication (e.g., apart from the gauge or sensor label), an aural alert, a haptic alert, and the like. Such indications and alerts give the pilots awareness of the potential error, thereby reducing confusion were the system to react in an unexpected way (e.g., the pilot is viewing raw data, while an aircraft system is reacting to processed data). In this way, pilots need not manually compare the two values on different displays or toggle between settings. In some instances, the electronic processor115is configured to implement multiple options (e.g., providing two values and different displays, allowing pilots to toggle data sources, and presenting alerts where a significant difference is detected).

Therefore, embodiments described herein provide systems and methods for providing pilot situational awareness, which enhances safe operations of the aircraft. The examples presented herein allow each multifunction display of an aircraft to independently depict any selection of raw sensor data (from any available sensor) or a processed version of that sensor data. The examples presented herein do not in any way automatically restrict what data may be displayed at each station. However, in some instances, a system is configured to alert the crew of the aircraft when multiple stations are displaying the same source for situational awareness. In some instances, a system may be further configured to indicate when there is a significant difference between the presently displayed sensor data sources to help the crew isolate a potential sensor fault.

Various features and advantages of some embodiments are set forth in the following claims.