Patent Description:
In one aspect, the invention is directed to a system as defined by claim <NUM>.

In a further aspect, the invention is directed to a method as defined by claim <NUM>.

The inventive concepts disclosed herein are capable of other embodiments or of being practiced or carried out in various ways, within the scope of the invention as defined by the claims.

Broadly, embodiments of the inventive concepts disclosed herein may be directed to a system and a method for a mobile computing device (e.g., a tablet computing device, a phone computing device, or a head wearable display computing device) to provide an augmented reality (AR) view of an aircraft cabin to select an aircraft cabin display and video content for the selected aircraft cabin display.

Cabin system customers often have multiple video displays installed, and different video content can be displayed on each display. The invention in a main aspect relies on using augmented reality views of the aircraft cabin to control content displayed on one or more aircraft cabin displays. A user may look around the cabin by viewing augmented reality images of the cabin on the user's mobile computing device. As nearby displays come into view, the mobile computing device is configured to use augmented reality to identify displays and create augmented reality views of the cabin that identify (e.g., highlight or draw a box around the displays) displays that the user can interact with. The user may make a user input (e.g., a user touch input, a user voice input, a user gesture input, or a user eye input) on the mobile computing device to select one or more of the displays, and the mobile computing device presents a graphical representation of user-selectable video content options for the one or more selected displays. Once the user selects the video content for the selected display(s), the graphical representation of the user-selectable video content options may be dismissed, and the selected display(s) displays the selected video. The mobile computing device instructs the system to send video to that selected display to display the selected video content. Some embodiments let the user point their mobile computing device at the display they desire to interact with to select the monitor they are looking at. Additionally, in some embodiments, the user may use a head wearable display computing device (e.g., an AR headset or AR glasses) such that the user does not even need to use their hand(s).

In some embodiments, the system software may provide an administrative interface and an end user interface.

In some embodiments, the administrative interface could be used for configuring an airplane for use, after cabin interior equipment has been installed. Such an administrative interface may be implemented on a mobile computing device configured to wirelessly communicate with other devices on the system (e.g., a line replaceable unit (LRU) computing device, displays, and/or an aircraft interface computing device). For example, using the administrative interface, the user may: capture, via the mobile computing device, a three dimensional (3D) spatial representation of an aircraft cabin interior; identify spatial points in the aircraft cabin interior where there are displays; for each identified spatial point, pair that point with a display entry in the LRU computing device; and/or export, via the mobile computing device, positioning and pairing data to the LRU computing device for end users to retrieve when using the end user interface.

In some embodiments, the end user interface may be used by passengers after the administrative interface has been used to initialize monitor positioning and pairing. Such an end user interface may be implemented on the mobile computing device configured to wirelessly communicate with other devices on the system (e.g., a line replaceable unit (LRU) computing device, displays, and/or an aircraft interface computing device). For example, the end user may: retrieve (e.g., automatically retrieve), via the mobile computing device, the positioning and pairing data created during administrative configuration; be able to point their mobile computing device camera around the aircraft cabin interior and see visual augmented reality indicators (e.g., highlighting or boxes) of displays that the user can interact with; select a display as seen through the mobile computing device and be offered a graphical interface through which the user can select a video source to output to the selected display(s); and/or when the user applies a change or cancels the end user interface, the graphical interface may be dismissed.

Referring now to <FIG>, an exemplary embodiment of a system <NUM> according to the inventive concepts disclosed herein is depicted.

The system may host at least one user (e.g., passengers, pilots, crew, cabin configurers, and/or equipment installers), and includes at least two displays <NUM> (one shown), at least one aircraft interface computing device <NUM> (e.g., an aircraft router computing device), at least one LRU computing device <NUM>, at least one mobile computing device <NUM> (e.g., a tablet computing device, a phone computing device, and/or a head wearable display computing device), some or all of which may be communicatively coupled at any given time. some embodiments, the at least one aircraft interface computing device <NUM> and the at least one LRU computing device <NUM>, may be implemented as a single computing device or any number of computing devices configured to perform (e.g., collectively perform if more than one computing device) any or all of the operations disclosed throughout.

The at least two displays <NUM> may be implemented as any suitable display device(s). Each of the displays <NUM> may be wiredly and/or wirelessly connected to the aircraft interface computing device <NUM>, the LRU computing device <NUM>, and/or the mobile computing device <NUM>. The displays <NUM> may each include at least one display element <NUM>, at least one processor <NUM>, and/or memory <NUM>, some or all of which may be communicatively coupled. One or more of the at least one display <NUM> may be configured to display particular video content associated with a user option selection associated with a particular video content option.

The at least one aircraft interface computing device <NUM> may be implemented as any suitable computing device, such as an aircraft router computing device (e.g., a flight operation and maintenance exchange (FOMAX) computing device or a smart router). The at least one aircraft interface computing device <NUM> may be configured to route (e.g., wiredly route and/or wirelessly route) communications to other computing devices (e.g., the displays <NUM>, the mobile computing device <NUM>, and/or the LRU computing device <NUM>). The at least one aircraft interface computing device <NUM> may include any or all of the elements, as shown in <FIG>. For example, the aircraft interface computing device <NUM> may include at least one processor <NUM>, at least one memory <NUM>, and/or at least one storage <NUM>, some or all of which may be communicatively coupled at any given time. For example, the at least one processor <NUM> may include at least one central processing unit (CPU), at least one graphics processing unit (GPU), at least one field-programmable gate array (FPGA), at least one application specific integrated circuit (ASIC), at least one digital signal processor, at least one deep learning processor unit (DPU), at least one virtual machine (VM) running on at least one processor, and/or the like configured to perform (e.g., collectively perform) any of the operations disclosed throughout. For example, the at least one processor <NUM> may include a CPU and a GPU configured to perform (e.g., collectively perform) any of the operations disclosed throughout. The processor <NUM> may be configured to run various software applications or computer code stored (e.g., maintained) in a non-transitory computer-readable medium (e.g., memory <NUM> and/or storage <NUM>) and configured to execute various instructions or operations. The processor <NUM> of the aircraft computing device <NUM> may be configured to perform any or all of the operations disclosed throughout.

The at least one LRU computing device <NUM> may be implemented as any suitable computing device(s), such as a cabin system computing device configured to control various aspects of the cabin electronic devices. For example, the LRU computing device <NUM> may be configured to receive, host, and/or output at least some of the user-selectable video content and/or connect (e.g., via a universal serial bus (USB) connection) to a storage device configured to store at least some of the user-selectable video content. For example, the LRU computing device <NUM> may be configured to control content displayed by the displays <NUM>. For example, the LRU computing device <NUM> may be configured to host and output various data accessible by the mobile computing device <NUM>. The at least one LRU computing device <NUM> may include any or all of the elements, as shown in <FIG>. For example, the LRU computing device <NUM> may include at least one processor <NUM>, at least one memory <NUM>, and/or at least one storage <NUM>, some or all of which may be communicatively coupled at any given time. For example, the at least one processor <NUM> may include at least one central processing unit (CPU), at least one graphics processing unit (GPU), at least one field-programmable gate array (FPGA), at least one application specific integrated circuit (ASIC), at least one digital signal processor, at least one deep learning processor unit (DPU), at least one virtual machine (VM) running on at least one processor, and/or the like configured to perform (e.g., collectively perform) any of the operations disclosed throughout. For example, the at least one processor <NUM> may include a CPU and a GPU configured to perform (e.g., collectively perform) any of the operations disclosed throughout. The processor <NUM> may be configured to run various software applications or computer code stored (e.g., maintained) in a non-transitory computer-readable medium (e.g., memory <NUM> and/or storage <NUM>) and configured to execute various instructions or operations. The at least one processor <NUM> of the LRU computing device <NUM> may be configured to perform (e.g., collectively perform) any or all of the operations disclosed throughout.

For example, the at least one LRU computing device <NUM> may be configured to perform (e.g., collectively perform if more than one LRU computing device <NUM>) any or all of the following: store mapped data associated with a connection between the LRU computing device <NUM> and the one (e.g., <NUM>-<NUM>) of the at least one display <NUM> (e.g., <NUM>-<NUM>) for each of spatial points within an interior of the aircraft cabin <NUM>, each of the spatial points corresponding to one (e.g., <NUM>-<NUM>) of the at least one display <NUM> (e.g., <NUM>-<NUM>) having the user-selectable video content options, wherein the mobile computing device <NUM> is further configured to: retrieve the mapped data from the LRU computing device <NUM>.

The at least one mobile computing device <NUM> may be within the aircraft cabin <NUM>. The at least one mobile computing device <NUM> may be implemented as any suitable computing device(s), such as at least one tablet computing device, at least one phone computing device, and/or at least one head wearable display computing device. The at least one mobile computing device <NUM> may be wiredly and/or wirelessly communicatively coupled to at least one of the LRU computing device <NUM>, the aircraft interface computing device <NUM>, and/or the displays <NUM> at any given time.

The at least one mobile computing device <NUM> may include any or all of the elements, as shown in <FIG>. For example, the LRU mobile computing device <NUM> may include at least one display <NUM>, at least one processor <NUM>, at least one memory <NUM>, at least one storage <NUM>, at least one camera <NUM>, and/or at least one antenna, some or all of which may be communicatively coupled at any given time. For example, the at least one processor <NUM> may include at least one central processing unit (CPU), at least one graphics processing unit (GPU), at least one field-programmable gate array (FPGA), at least one application specific integrated circuit (ASIC), at least one digital signal processor, at least one deep learning processor unit (DPU), at least one virtual machine (VM) running on at least one processor, and/or the like configured to perform (e.g., collectively perform) any of the operations disclosed throughout. For example, the at least one processor <NUM> may include a CPU and a GPU configured to perform (e.g., collectively perform) any of the operations disclosed throughout. The processor <NUM> may be configured to run various software applications or computer code stored (e.g., maintained) in a non-transitory computer-readable medium (e.g., memory <NUM> and/or storage <NUM>) and configured to execute various instructions or operations. The at least one processor <NUM> of the mobile computing device <NUM> may be configured to perform (e.g., collectively perform) any or all of the operations disclosed throughout.

The at least one mobile computing device <NUM> is configured to perform (e.g., collectively perform if more than one mobile computing device <NUM>) the following: receive a user input to control video content upon receipt of the user input to control video content, capture images of the aircraft cabin <NUM> based on user manipulation of a position and an orientation of the mobile computing device <NUM> within the aircraft cabin <NUM>; upon capture of the images, identify at least one display <NUM> (e.g., see also <NUM>-<NUM> in <FIG>) of the at least two displays <NUM> within the captured images, wherein the at least one display <NUM> (e.g., <NUM>-<NUM> in <FIG>) has user-selectable video content options; upon identification of the at least one display <NUM> (e.g., <NUM>-<NUM>), display an augmented reality view of the captured images including an indicator (e.g., <NUM> in <FIG>) for each of the at least one display <NUM> (e.g., <NUM>-<NUM>) having the user-selectable video content options; upon displaying of the augmented reality view of the captured images, receive a user selection of a given display <NUM> (e.g., one of <NUM>-<NUM>) of the at least one display <NUM> (e.g., <NUM>-<NUM>), wherein the user selection is indicative of a selection to control video content displayed by the given display <NUM> (e.g., one of <NUM>-<NUM>); upon receipt of the user selection, display a graphical representation of the user-selectable video content options; upon display of the graphical representation, receive a user option selection of the user-selectable video content options from the user, the user option selection associated with a particular video content option associated with particular video content; and/or upon receipt of the user option selection, wirelessly transmit an instruction to display the particular video content on the given display <NUM> (e.g., one of <NUM>-<NUM>).

In some embodiments, the at least one mobile computing device <NUM> may be further configured to perform (e.g., collectively perform if more than one mobile computing device <NUM>) any or all of the following: pair with the LRU computing device <NUM> and with the at least one display. In some embodiments, the at least one mobile computing device <NUM> may be configured to perform (e.g., collectively perform if more than one mobile computing device <NUM>) any or all of the following: capture a three-dimensional (3D) representation of an interior of the aircraft cabin <NUM>; identify spatial points within the representation of the interior such that each of the spatial points corresponds to one (e.g., <NUM>-<NUM>) of the at least one display <NUM> (e.g., <NUM>-<NUM>) having the user-selectable video content options; for each of the spatial points, pair said spatial point with the one (e.g., <NUM>-<NUM>) of the at least one display <NUM> (e.g., <NUM>-<NUM>); output data associated with the spatial points and the at least one display <NUM> (e.g., <NUM>-<NUM>) to the LRU computing device <NUM> such that the LRU computing device <NUM> is configured to map said spatial point with the one (e.g., <NUM>-<NUM>) of the at least one display <NUM> (e.g., <NUM>-<NUM>) and such that the LRU computing device <NUM> is configured to store mapped data associated with a connection between the LRU computing device and the one (e.g., <NUM>-<NUM>) of the at least one display <NUM> (e.g., <NUM>-<NUM>) for each of the spatial points; and/or retrieve the mapped data from the LRU computing device <NUM>.

Referring now to <FIG>, an exemplary embodiment of a view of an augmented reality view of the captured images including an indicator (e.g., <NUM> in <FIG>) for each of the at least one display <NUM> (e.g., <NUM>-<NUM>) having the user-selectable video content options of the system <NUM> of <FIG> according to the inventive concepts disclosed herein is depicted. Each indicator <NUM> may be any suitable graphical indicator that indicates a location of one of the at least one display <NUM> (e.g., <NUM>-<NUM>) having the user-selectable video content options. For example, each of the indicators <NUM> may be implemented as a colored box around one of the at least one display <NUM> (e.g., <NUM>-<NUM>), may highlight one of the at least one display <NUM> (e.g., <NUM>-<NUM>), and/or may include an arrow pointing to one of the at least one display <NUM> (e.g., <NUM>-<NUM>).

Referring now to <FIG>, an exemplary embodiment of a method <NUM> according to the inventive concepts disclosed herein includes the following steps. Additionally, for example, some embodiments may include performing one or more instances of the method <NUM> iteratively, concurrently, and/or sequentially. Additionally, for example, at least some of the steps of the method <NUM> which define the provision of components of the system may be performed in parallel and/or concurrently. Additionally, in some embodiments, at least some of such steps of the method <NUM> may be performed non-sequentially.

A step <NUM> includes providing an aircraft interface computing device configured to route communications to other computing devices.

A step <NUM> includes providing at least two displays installed in an aircraft cabin.

A step <NUM> includes providing a line replaceable unit (LRU) computing device comprising at least one processor and memory, the LRU computing device communicatively coupled to the aircraft interface computing device and, optionally, to the at least two displays.

A step <NUM> includes providing a mobile computing device within the aircraft cabin, the mobile computing device comprising at least one mobile computing device processor, at least one mobile computing device camera, and a mobile computing device display.

A step <NUM> includes receiving, by the mobile computing device, a user input to control video content.

A step <NUM> includes upon receipt of the user input to control video content, capturing, by the mobile computing device, images of the aircraft cabin based on user manipulation of a position and an orientation of the mobile computing device within the aircraft cabin.

A step <NUM> includes upon capture of the images, identifying, by the mobile computing device, at least one display of the at least two displays within the captured images, wherein the at least one display has user-selectable video content options.

A step <NUM> includes upon identification of the at least one display, displaying, by the mobile computing device, an augmented reality view of the captured images including an indicator for each of the at least one display having the user-selectable video content options.

A step <NUM> includes upon displaying of the augmented reality view of the captured images, receiving, by the mobile computing device, a user selection of a given display of the at least one display, wherein the user selection is indicative of a selection to control video content displayed by the given display.

A step <NUM> includes upon receipt of the user selection, displaying, by the mobile computing device, a graphical representation of the user-selectable video content options.

A step <NUM> includes upon display of the graphical representation, receiving, by the mobile computing device, a user option selection of the user-selectable video content options from the user, the user option selection associated with a particular video content option associated with particular video content.

A step <NUM> includes upon receipt of the user option selection, wirelessly transmitting, by the mobile computing device, an instruction to display the particular video content on the given display.

A step <NUM> includes displaying, by the given display, the particular video content.

As will be appreciated from the above, embodiments of the inventive concepts disclosed herein are directed to a system and a method for a mobile computing device to provide an augmented reality view of an aircraft cabin to select an aircraft cabin display and video content for the selected aircraft cabin display.

As used throughout and as would be appreciated by those skilled in the art, "at least one non-transitory computer-readable medium" may refer to as at least one non-transitory computer-readable medium (e.g., at least one computer-readable medium implemented as hardware; e.g., at least one non-transitory processor-readable medium, at least one memory (e.g., at least one nonvolatile memory, at least one volatile memory, or a combination thereof; e.g., at least one random-access memory, at least one flash memory, at least one read-only memory (ROM) (e.g., at least one electrically erasable programmable read-only memory (EEPROM)), at least one on-processor memory (e.g., at least one on-processor cache, at least one on-processor buffer, at least one on-processor flash memory, at least one on-processor EEPROM, or a combination thereof), or a combination thereof), at least one storage device (e.g., at least one hard-disk drive, at least one tape drive, at least one solid-state drive, at least one flash drive, at least one readable and/or writable disk of at least one optical drive configured to read from and/or write to the at least one readable and/or writable disk, or a combination thereof), or a combination thereof).

Claim 1:
A system, comprising:
an aircraft interface computing device (<NUM>) configured to route communications to other computing devices;
at least two displays (<NUM>) installed in an aircraft cabin (<NUM>);
a line replaceable unit, LRU, computing device (<NUM>) comprising at least one processor (<NUM>) and memory (<NUM>), the LRU computing device communicatively coupled to the aircraft interface computing device and, optionally, to the at least two displays; and
a mobile computing device (<NUM>) within the aircraft cabin, the mobile computing device comprising at least one mobile computing device processor (<NUM>), at least one mobile computing device camera (<NUM>), and a mobile computing device display (<NUM>), wherein the mobile computing device is configured to:
receive a user input to control video content;
upon receipt of the user input to control video content, capture images of the aircraft cabin based on user manipulation of a position and an orientation of the mobile computing device within the aircraft cabin;
upon capture of the images, identify at least one display of the at least two displays within the captured images, wherein the at least one display has user-selectable video content options;
upon identification of the at least one display, display an augmented reality view of the captured images including an indicator for each of the at least one display having the user-selectable video content options;
upon displaying of the augmented reality view of the captured images, receive a user selection of a given display of the at least one display, wherein the user selection is indicative of a selection to control video content displayed by the given display;
upon receipt of the user selection, display a graphical representation of the user-selectable video content options;
upon display of the graphical representation, receive a user option selection of the user-selectable video content options from the user, the user option selection associated with a particular video content option associated with particular video content; and
upon receipt of the user option selection, wirelessly transmit an instruction to display the particular video content on the given display;
wherein the given display is configured to display the particular video content.