Patent Description:
An electronic checklist (ECL) for use in an aircraft can be integrated into onboard avionics (referred to herein as onboard ECL) or can be provided outside of the onboard avionics (referred to herein as off-board ECL) from a mobile device such as a portable EFB (electronic flight bag) or other portable electronic device. The mobile device that hosts the off-board ECL, typically does not have a communication connection with onboard avionics system. Thus, an onboard avionics system lacks a way to trigger the display of an off-board ECL. When an aircraft alert requiring the performance of tasks specified in an off-board ECL occurs, a flight crew member, such as the pilot, may have to manually search for the appropriate checklist on the mobile device, for example, via typing a search query on a virtual keyboard. Text input can be cumbersome, particularly due to the small size of many virtual keyboards. This can increase flight crew workload, particularly in what could be a high-stress time, and can increase the flight crew's response time for responding to the condition causing the alert.

Relevant background prior art is disclosed in <CIT> and <CIT>.

Hence, it is desirable to provide an improved system and method for communicating an alert to facilitate the activation of an off-board ECL. Furthermore, other desirable features and characteristics of the present invention will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and the foregoing technical field and background.

In one embodiment, a computer-implemented system for providing checklists on an aircraft is provided. The system includes a controller and a mobile device. The controller is configured by programming instructions to: encode, in an optical code, information pertaining to an aircraft alert message displayed on a cockpit display, wherein the optical code when decoded identifies the information pertaining to the displayed aircraft alert message. The controller is further configured to cause a graphical element to be displayed (e.g., adjacent to the displayed aircraft alert message on the cockpit display or at another position per cockpit design) that indicates that the optical code encoded with the information pertaining to the aircraft alert message is available for display on the cockpit display; receive the selection of the graphical element; and cause the optical code to be displayed (e.g., on the same or different cockpit display) in a dialog window responsive to the selection of the graphical element. The mobile device includes a mobile device controller that is configured by programming instructions to: capture, using an optical code reader, a visual image of the optical code displayed on the cockpit display; decode, using the visual image, the optical code to identify the information pertaining to the displayed aircraft alert message; retrieve a checklist containing instructions (e.g., for the flight crew or maintenance personnel) for responding in view of the information pertaining to the displayed aircraft alert message; and cause the retrieved checklist to be displayed on a display on the mobile device.

In another embodiment, a method in an aircraft for activating functions on external equipment based on aircraft data displayed on an aircraft cockpit display is provided. The method includes: encoding, in an optical code, information pertaining to an aircraft alert message displayed on a cockpit display, wherein the optical code when decoded identifies information pertaining to the displayed aircraft alert message; causing a graphical element to be displayed (e.g., adjacent to the displayed aircraft alert message on the cockpit display or at another position per cockpit design) that indicates that the optical code encoded with the information pertaining to the aircraft alert message is available for display (e.g., on the cockpit display or a different cockpit display); receiving the selection of the graphical element; causing the optical code to be displayed (e.g., on the same or different cockpit display) in a dialog window responsive to the selection of the graphical element; capturing, by a mobile device using an optical code reader, a visual image of the optical code displayed on the cockpit display; decoding, by the mobile device using the visual image, the optical code to identify information pertaining to the displayed aircraft alert message; retrieving a checklist containing instructions (e.g., for the flight crew or maintenance personnel) for responding in view of the information pertaining to the displayed aircraft alert message; and causing the retrieved checklist to be displayed on a display on the mobile device.

In another embodiment, non-transitory computer readable media encoded with programming instructions configurable to cause the performance of a method is provided. The method includes: encoding, in an optical code, information pertaining to an aircraft alert message displayed on a cockpit display, wherein the optical code when decoded identifies information pertaining to the displayed aircraft alert message; causing a graphical element to be displayed adjacent to the displayed aircraft alert message that indicates that the optical code encoded with the information pertaining to the aircraft alert message is available for display (e.g., on the same or different cockpit display); receiving the selection of the graphical element; and causing the optical code to be displayed (e.g., on the same or different cockpit display) in a dialog window responsive to the selection of the graphical element; wherein a mobile device captures a visual image of the optical code using an optical code reader, decodes using the visual image the optical code to identify the information pertaining to the displayed aircraft alert message, retrieves a checklist containing instructions (e.g., for the flight crew or maintenance personnel) for responding in view of the information pertaining to the displayed aircraft alert message, and displays the checklist on a display on the mobile device.

The subject matter described herein discloses apparatus, systems, techniques and articles for using an optical code, such as a QR code, as media to transmit alert information, to activate an off-board ECL. <FIG> is a block diagram depicting an example system <NUM> for providing checklists on an aerial vehicle (e.g., airplane, helicopter, vertical takeoff and landing vehicle, or other aircraft). The checklists may be provided for use during flight operations to the flight crew when certain unscheduled events occur. The checklists may be provided while the aerial vehicle is on the ground, for example, during maintenance operations. The example system <NUM> includes cockpit equipment <NUM> and a mobile device <NUM>. The cockpit equipment includes a display device <NUM> such as a Head-Up Display (HUD) device and/or Head-Down Display (HDD) device, and a controller <NUM> for generating optical codes. The mobile device <NUM> is a portable electronic device (e.g., tablet computer, smartphone, laptop, Electronic Flight Bag, or other mobile devices) carried into the aircraft cockpit by a flight crew member or maintenance personnel. The mobile device includes an optical code capture device <NUM> (e.g., a camera), a controller <NUM>, and a display device <NUM>.

Each of the optical code controller <NUM> and the mobile device controller <NUM> includes at least one processor and a computer-readable storage device or media encoded with programming instructions for configuring the controller. The processor may be any custommade or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), an auxiliary processor among several processors associated with the controller, a semiconductor-based microprocessor (in the form of a microchip or chip set), any combination thereof, or generally any device for executing instructions.

The computer readable storage device or media may include volatile and nonvolatile storage in read-only memory (ROM), random-access memory (RAM), and keep-alive memory (KAM), for example. KAM is a persistent or non-volatile memory that may be used to store various operating variables while the processor is powered down. The computer-readable storage device or media may be implemented using any of a number of known memory devices such as PROMs (programmable read-only memory), EPROMs (electrically PROM), EEPROMs (electrically erasable PROM), flash memory, or any other electric, magnetic, optical, or combination memory devices capable of storing data, some of which represent executable programming instructions, used by the controller.

The optical code controller <NUM> is configured by programming instructions to encode, in an optical code, information pertaining to an aircraft alert message displayed on the cockpit display device <NUM>, wherein the optical code when decoded identifies information pertaining to the displayed aircraft alert message. The aircraft alert message may be a trouble message generated by aircraft systems, such as a CAS message, an emergency message (e.g., engine on fire), or some other message, indicating that an unplanned or extraordinary circumstance has occurred regarding which the flight crew should be made aware during flight operations. The aircraft alert message also may be an aircraft equipment failure message that is reported to the flight crew during flight and/or ground operations and/or reported to ground personnel (e.g., maintainers), for example, during maintenance operations.

The optical code is used to represent data regarding the aircraft alert message in a visual, machine-readable form. The optical code may be a linear or one-dimensional (1D) barcode such as a Universal Product Code (UPC) barcode. The optical code may be a two-dimensional (2D) optical code (e.g., matrix code, 2D barcode, or QR code) that uses rectangles, dots, hexagons and other geometric patterns to represent data regarding the aircraft alert message in a visual, machine-readable form.

The optical code controller <NUM> is further configured by programming instructions to cause a graphical element to be displayed that indicates that an optical code encoded with the information pertaining to an aircraft alert message is available for display on a cockpit display. The graphical element may be displayed adjacent to the aircraft alert message on the same display device <NUM> or displayed on a different cockpit display device.

<FIG> are diagrams depicting example ways of displaying a graphical element that indicates that an optical code related to an aircraft message is available for display on a cockpit display. In each diagram, twelve message lines are displayed in a display window. Message Line <NUM>, Message Line <NUM>, and Message Line <NUM> contain alert messages for which an optical code is available for display on a cockpit display.

In the example of <FIG>, a symbol <NUM> in the shape of a QR code is provided as the graphical element that indicates that an optical code related to an aircraft message is available for display on a cockpit display. In this example, a symbol <NUM> is provided at the beginning of each of Message Line <NUM>, Message Line <NUM>, and Message Line <NUM> in the display window <NUM>. Selection of one of the symbols <NUM> and/or the associated message line will cause an optical code associated with the alert message indicated in the message line to be displayed on a cockpit display. The optical code may be generated prior to selection of the graphical element or after selection of the graphical element.

In the example of <FIG>, a symbol <NUM> in the shape of a QR code is provided as the graphical element that indicates that an optical code related to an aircraft message is available for display on a cockpit display. In this example, a symbol <NUM> is provided at the end of each of Message Line <NUM>, Message Line <NUM>, and Message Line <NUM> in the display window <NUM>. Selection of one of the symbols <NUM> and/or the associated message line will cause an optical code associated with the alert message indicated in the message line to be displayed on a cockpit display. The optical code may be generated prior to selection of the graphical element or after selection of the graphical element.

In the example of <FIG>, a symbol <NUM> in the form of an underline is provided as the graphical element that indicates that an optical code related to an aircraft message is available for display on a cockpit display. In this example, each of Message Line <NUM>, Message Line <NUM>, and Message Line <NUM> is underlined in the display window <NUM>. Selection of one of Message Line <NUM>, Message Line <NUM>, or Message Line <NUM> will cause an optical code associated with the alert message indicated in the message line to be displayed on a cockpit display. The optical code may be generated prior to selection of the graphical element or after selection of the graphical element.

The optical code controller <NUM> is further configured by programming instructions to receive the selection of the graphical element and, responsive thereto, cause the optical code to be displayed on a cockpit display. The optical code may be displayed, for example, in a dialog window adj acent to a display window that displayed the associated alert message. Alternatively or additionally, in some examples, the optical code controller <NUM> may be further configured by programming instructions to cause the optical code to be automatically displayed on a cockpit display without the selection of the graphical element.

<FIG> is a diagram depicting an example display window <NUM> that displays twelve message lines and a dialog window <NUM> that displays an optical code <NUM> that corresponds to an alert message in a selected message line. In this example, Message Line <NUM> has been selected as indicated by the box <NUM> around the message line and the message line header <NUM> in the dialog window <NUM>, which also indicates that the displayed optical code <NUM> corresponds to Message Line <NUM>.

The optical code controller <NUM> is further configured by programming instructions to close the dialog window <NUM> under a number of different circumstances. The dialog window <NUM> may be closed, for example, when a predetermined time period has elapsed or by flight crew action, e.g., selection of the "x" in the dialog window <NUM>. The dialog window <NUM> may be closed, for example, when a cursor used for selecting the graphical element is clicked outside the dialog window, or when the cursor hovers outside of the dialog window for a predetermined period time without click operation. Also, the dialog window <NUM> may be automatically closed, for example, when a high priority event (e.g., terrain alert or traffic alert) is sensed by an aircraft system.

<FIG> is a diagram depicting an example dialog window <NUM>. The example dialog window <NUM> includes a message line header <NUM> which identifies the alert message in the selected message line, an optical symbol <NUM> corresponding to the alert message, a selectable scroll symbol <NUM> that when selected causes the next optical symbol to be displayed when there are multiple alert messages and the dialog window was opened automatically, and a numeric indicator <NUM> that indicates the number of alert messages for which optical symbols exist.

In one example operating scenario, a window <NUM> may pop up automatically if a QR window is not currently displayed when a new CAS message with QR code is triggered. When multiple CAS messages with QR code are triggered and the pilot has not selected any specific CAS message with QR code, the highest priority Level QR code is displayed first (e.g., same priority as CAS message: Warning, Caution, Advisory, Status (from high to low)). If multiple CAS messages are triggered that have the same priority level, the latest CAS message and associated QR code will be displayed. When multiple CAS messages with QR code are triggered and the pilot selects a specific CAS message with QR code, the QR code corresponding to the selected CAS message will be displayed until the pilot select a new CAS message or scrolls to a different QR code or a new CAS message with same priority or higher priority.

The optical code controller <NUM> is further configured by programming instructions to close the dialog window <NUM> under a number of different circumstances. The dialog window <NUM> may be closed, for example, when a predetermined time period has elapsed or by flight crew action, e.g., selection of another window other than the dialog window <NUM>, like waypoint list waypoint, CDTI window, etc. Also, the dialog window <NUM> may be automatically removed and return back to the original window displayed, for example, when a high priority event (e.g., terrain alert or traffic alert) is sensed by an aircraft system.

Referring back to <FIG>, the mobile device controller <NUM> is configured by programming instructions to cause the mobile device to capture, using the optical code capture device <NUM>, a visual image of the optical code displayed on a cockpit display. The optical code capture device <NUM> may, for example, be a camera on the mobile device.

The mobile device controller <NUM> is further configured by programming instructions to decode, using the captured visual image, the optical code to identify information pertaining to the aircraft alert message displayed on the cockpit display and retrieve a checklist <NUM> containing instructions (e.g., for the flight crew or maintenance personnel) for responding in view of the information pertaining to the displayed aircraft alert message. The checklist <NUM> may be resident in a data store on the mobile device <NUM> or may be retrieved from outside of the mobile device, for example, on a cloud-based data store. The mobile device controller <NUM> is further configured by programming instructions to cause the retrieved checklist <NUM> to be displayed on the display device <NUM>.

<FIG> is a process flow chart depicting an example process <NUM> for providing checklists on an aerial vehicle. The order of operation within the process <NUM> is not limited to the sequential execution as illustrated in the figure, but may be performed in one or more varying orders as applicable and in accordance with the present disclosure.

The example process <NUM> includes encoding, in an optical code, information pertaining to an aircraft alert message displayed on a cockpit display, wherein the optical code when decoded identifies information pertaining to the displayed aircraft alert message (operation <NUM>).

The example process <NUM> includes causing a graphical element to be displayed that indicates that the optical code encoded with the information pertaining to the aircraft alert message is available for display on a cockpit display (operation <NUM>).

The example process <NUM> includes receiving the selection of the graphical element (operation <NUM>) and causing the optical code to be displayed on the cockpit display in a dialog window responsive to the selection of the graphical element (operation <NUM>).

The example process <NUM> includes capturing, by a mobile device using an optical code reader, a visual image of the optical code displayed on the cockpit display (operation <NUM>), decoding, by the mobile device using the visual image, the optical code to identify information pertaining to the aircraft alert message displayed on the cockpit display (operation <NUM>), retrieving a checklist containing instructions (e.g., for the flight crew or maintenance personnel) for responding in view of the information pertaining to the displayed aircraft alert message (operation <NUM>); and causing the checklist to be displayed on a display on the mobile device (operation <NUM>). The flight crew and/or ground personnel can view the checklist and perform the instructions contained therein. This can be a quick way of making a checklist available when an abnormal condition occurs on the aircraft.

Described herein are apparatus, systems, techniques and articles for using an optical code, such as a QR code, as media to transmit alert information, to activate an off-board ECL. In one embodiment, a computer-implemented system for providing checklists on an aircraft is provided. The system comprises a controller and a mobile device. The controller is configured by programming instructions to: encode, in an optical code, information pertaining to an aircraft alert message displayed on a cockpit display, wherein the optical code when decoded identifies the information pertaining to the displayed aircraft alert message. The controller is further configured to cause a graphical element to be displayed (e.g., adjacent to the displayed aircraft alert message on the cockpit display or on a different cockpit display) that indicates that the optical code encoded with the information pertaining to the aircraft alert message is available for display on the cockpit display; and cause the optical code to be displayed (e.g., on the same or different cockpit display) in a dialog window. The mobile device comprises a mobile device controller that is configured by programming instructions to: capture, using an optical code reader, a visual image of the optical code displayed on the cockpit display; decode, using the visual image, the optical code to identify the information pertaining to the aircraft alert message displayed on the cockpit display; retrieve a checklist containing instructions (e.g., for the flight crew or maintenance personnel) for responding in view of the information pertaining to the displayed aircraft alert message; and cause the retrieved checklist to be displayed on a display on the mobile device.

These aspects and other embodiments may include one or more of the following features. The optical code may comprise a one-dimensional or linear barcode that encodes information by varying the widths and/or spacings of parallel lines. The optical code may comprise a two-dimensional or matrix barcode that encodes information using squares, rectangles, dots, hexagons or other geometric patterns. The matrix barcode may comprise a Quick Response code (QR code). The optical code reader may comprise a camera on the mobile device. The aircraft alert message may comprise a crew-alerting system (CAS) message. The checklist may be retrieved from a data store on the mobile device. The checklist may be retrieved from a cloud-based data store. The cockpit display may comprise an integrated navigational (INAV) display or a multifunction display (MFD), or others The controller in the onboard avionics equipment may be further configured to close the dialog window when: a predetermined time period has elapsed, the dialog window has been closed by flight crew action, a cursor used for selecting the graphical element has been clicked outside the dialog window, the cursor hovers outside of the dialog window for a predetermined period time without click operation, and/or a high priority event (e.g., terrain alert or traffic alert) is sensed by an aircraft system. The controller may be further configured to receive the selection of the graphical element and to cause the optical code to be displayed responsive to the selection of the graphical element.

In another embodiment, a method in an aircraft for activating functions on external equipment based on aircraft data displayed on an aircraft cockpit display is provided. The method comprises: encoding, in an optical code, information pertaining to an aircraft alert message displayed on a cockpit display, wherein the optical code when decoded identifies information pertaining to the displayed aircraft alert message; causing a graphical element to be displayed (e.g., adjacent to the displayed aircraft alert message on the cockpit display or on a different cockpit display) that indicates that the optical code encoded with the information pertaining to the aircraft alert message is available for display (e.g., on the cockpit display or a different cockpit display); receiving the selection of the graphical element; causing the optical code to be displayed (e.g., on the same or different cockpit display) in a dialog window; capturing, by a mobile device using an optical code reader, a visual image of the optical code displayed on the cockpit display; decoding, by the mobile device using the visual image, the optical code to identify information pertaining to the displayed aircraft alert message; retrieving a checklist containing instructions (e.g., for the flight crew or maintenance personnel) for responding in view of the information pertaining to the displayed aircraft alert message; and causing the retrieved checklist to be displayed on a display on the mobile device.

These aspects and other embodiments may include one or more of the following features. The optical code may comprise a one-dimensional or linear barcode that encodes information by varying the widths and/or spacings of parallel lines. The optical code may comprise a two-dimensional or matrix barcode that encodes information using squares, rectangles, dots, hexagons or other geometric patterns. The matrix barcode may comprise a Quick Response code (QR code). The optical code reader may comprise a camera on the mobile device. The aircraft alert message may comprise a crew-alerting system (CAS) message. The checklist may be retrieved from a data store on the mobile device. The checklist may be retrieved from a cloud-based data store. The method may further comprise closing the dialog window when: a predetermined time period has elapsed, the dialog window has been closed by flight crew action, a cursor used for selecting the graphical element has been clicked outside the dialog window, the cursor hovers outside of the dialog window for a predetermined period time without click operation, and/or a high priority event (e.g., terrain alert or traffic alert) has been sensed by an aircraft system. The method may further comprise receiving the selection of the graphical element and causing the optical code to be displayed in a dialog window responsive to the selection of the graphical element.

In another embodiment, non-transitory computer readable media encoded with programming instructions configurable to cause the performance of a method is provided. The method comprises: encoding, in an optical code, information pertaining to an aircraft alert message displayed on a cockpit display, wherein the optical code when decoded identifies information pertaining to the displayed aircraft alert message; causing a graphical element to be displayed adjacent to the displayed aircraft alert message that indicates that the optical code encoded with the information pertaining to the aircraft alert message is available for display (e.g., on the same or different cockpit display); receiving the selection of the graphical element; and causing the optical code to be displayed (e.g., on the same or different cockpit display) in a dialog window responsive to the selection of the graphical element; wherein a mobile device captures a visual image of the optical code using an optical code reader, decodes using the visual image the optical code to identify the information pertaining to the displayed aircraft alert message, retrieves a checklist containing instructions (e.g., for the flight crew or maintenance personnel) for responding in view of the information pertaining to the displayed aircraft alert message, and displays the checklist on a display on the mobile device.

These aspects and other embodiments may include one or more of the following features. The optical code may comprise a one-dimensional or linear barcode that encodes information by varying the widths and/or spacings of parallel lines. The optical code may comprise a two-dimensional or matrix barcode that encodes information using squares, rectangles, dots, hexagons or other geometric patterns. The matrix barcode may comprise a Quick Response code (QR code). The optical code reader may comprise a camera on the mobile device. The aircraft alert message may comprise a crew-alerting system (CAS) message. The checklist may be retrieved from a data store on the mobile device. The checklist may be retrieved from a cloud-based data store. The method may further comprise closing the dialog window when: a predetermined time period has elapsed, the dialog window has been closed by flight crew action, a cursor used for selecting the graphical element has been clicked outside the dialog window, the cursor hovers outside of the dialog window for a predetermined period time without click operation, and/or a high priority event (e.g., terrain alert or traffic alert) has been sensed by an aircraft system.

In another embodiment, a method in an aircraft for activating functions on external equipment based on aircraft data displayed on an aircraft cockpit display is provided. The method comprises: providing, in an onboard avionics system, a code generation application configured to generate: a plurality of machine readable optical codes (e.g., QR code or bar code), wherein each machine readable optical code is specific for a different aircraft alert message generated by an aircraft cockpit system (during flight and/or on the ground) and displayed on the cockpit display (during flight and/or on the ground) and is configured to capture the information content of the aircraft alert message to which it pertains, and a graphical element to present on the cockpit display adjacent to an aircraft alert message (or at another position per cockpit design) that when selected causes the display of a generated machine readable optical code that pertains to the aircraft alert message in a dialog window on the cockpit display; generating during flight, using the code generation application, a machine readable optical code pertaining to an aircraft alert message displayed on the cockpit display, wherein the machine readable optical code when decoded provides the information content of the displayed aircraft alert message; causing a graphical element to be displayed adjacent to the displayed aircraft alert message on the cockpit display; receiving, by the onboard avionics system during flight, the selection of the graphical element; causing, by the code generation application, the machine readable optical code to be displayed on the cockpit display in a dialog window responsive to the selection of the graphical element; capturing, by a mobile device during flight, a visual image of the machine readable optical code displayed on the cockpit display; decoding, by the mobile device during flight, the machine readable optical code to retrieve the information content of the displayed aircraft alert message, wherein the mobile device retrieves the information content of the displayed aircraft alert message (during flight) from an aircraft cockpit system without a wired or wireless connection with the aircraft cockpit system; and displaying, by the mobile device, the display of a checklist.

Claim 1:
A computer-implemented system (<NUM>) for providing checklists on an aircraft, the system comprising:
a controller in onboard avionics equipment (<NUM>), the controller configured by programming instructions to:
encode, in an optical code (<NUM>), information pertaining to an aircraft alert message (<NUM>) displayed on a cockpit display (<NUM>), wherein the optical code when decoded identifies the information pertaining to the displayed aircraft alert message;
cause a graphical element (<NUM>) to be displayed that indicates that the optical code encoded with the information pertaining to the aircraft alert message is available for display; and
responsive to a selection of the graphical element, cause the optical code to be displayed in a dialog window (<NUM>); and
a mobile device (<NUM>) comprising a mobile device controller (<NUM>) configured by programming instructions to:
capture, using an optical code reader (<NUM>), a visual image of the displayed optical code;
decode, using the visual image, the optical code to identify the information pertaining to the displayed aircraft alert message;
retrieve a checklist containing instructions for responding in view of the information pertaining to the displayed aircraft alert message, wherein the checklist is retrieved from a data store on the mobile device or from a cloud-based data store; and
cause the checklist to be displayed on a display on the mobile device.