Patent Description:
In one aspect, embodiments of the inventive concepts disclosed herein are directed to a boom pod with an adaptable user interface that automatically reconfigures based on operation phase. Elements of the user interface may also be manually reconfigured based on operator preference. Operator reconfigurations may be recorded for use during similar phases in subsequent operations.

In a further aspect, a controller automatically records system status during a refueling operation and prepares a digital log entry.

In a further aspect, portions of the use interface are dedicated to augmented displays to assist the operator at any given operational phase.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and should not restrict the scope of the claims. The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments of the inventive concepts disclosed herein and together with the general description, serve to explain the principles.

The numerous advantages of the embodiments of the inventive concepts disclosed herein may be better understood by those skilled in the art by reference to the accompanying figures in which:.

Broadly, embodiments of the inventive concepts disclosed herein are directed to a boom pod with an adaptable user interface that automatically reconfigures based on operation phase. Elements of the user interface may also be manually reconfigured based on operator preference. Operator reconfigurations may be recorded for use during similar phases in subsequent operations. A controller automatically records system status during a refueling operation and prepares a digital log entry. Portions of the use interface are dedicated to augmented displays to assist the operator at any given operational phase.

Referring to <FIG>, a block diagram of a system according to an exemplary embodiment is shown. The system, embodied in an aircraft refueling boom pod, includes a processor <NUM>, memory <NUM> in data communication with the processor <NUM> for embodying processor executable code, and one or more displays <NUM>. The displays <NUM> may be touch sensitive to allow for input and interaction with the functionality of the processor <NUM>. In at least one embodiment, one or more cameras <NUM> are in data communication with the processor <NUM> to provide a real-time image or data derived from a real-time image to the displays <NUM>.

In at least one embodiment, a data storage element <NUM> in data commination with the processor <NUM> may store a record of refueling interactions in a log for later contemporaneous or subsequent transmission via a data communication element <NUM>. Furthermore, the data storage element <NUM> may record changes to a user interface via operator interaction with the touch sensitive displays <NUM> to maintain a persistent, desired layout for each operator during each phase of a refueling operation.

In at least one embodiment, the processor <NUM> may apply one or more augmented reality overlays to images from the cameras <NUM>. For example, a camera <NUM> at a relatively fixed location corresponding to a refueling boom arm may produce an image to assist in positioning the boom arm; the processor <NUM> may apply an augmented reality overlay to the image that corresponds to the maximum movement of the boom arm. In another example, the processor <NUM> may apply image processing algorithms such as edge detection to apply an augmented reality overlay of the aircraft being refueled to enhance the operator's view of the aircraft, possibly including low-light enhancement. In at least one embodiment, the processor <NUM> may determine an aircraft type based on a database of known aircraft from the data storage element <NUM>, and apply an augmented reality overlay to enhance important features of the aircraft. In at least one embodiment, the processor <NUM> may utilize image processing algorithms or other onboard sensors to determine the location of the boom arm with respect to the images from the cameras <NUM>; in such embodiments, the processor <NUM> may render an enhanced view of the boom arm and / or an indication of a calculated distance and direction between the boom arm and the corresponding refueling port on the aircraft.

Alternatively, an operator may desire certain of the augmented reality features, but not the underlying image. In at least one embodiment, the processor <NUM> may use an image from one or more of the cameras <NUM> to determine certain image overlays such as the boundaries of the boom arm and / or a current location of the boom, but render those overlays on a blank portion of the display without the image.

Referring to <FIG>, a perspective view of a boom pod <NUM> according to an exemplary embodiment is shown. The boom pod <NUM> may be divided into an informational panel <NUM> and a control panel <NUM>; in at least one embodiment, the control panel <NUM> may be substantially similar to existing control panels <NUM>. Previously, boom pods <NUM> may have included an ancillary control panel <NUM>; but in at least one embodiment, the ancillary control panel <NUM> is not needed. When embodiments are retrofit into existing aircraft, existing controls may be obviated and removed.

The informational panel <NUM> comprises one or more displays <NUM>, <NUM> in place of existing gauges, indicator lights, control knobs, and switches of existing boom pods <NUM>. In at least one embodiment, the displays <NUM>, <NUM> maybe stacked vertically or arranged horizontally. One or more of the displays <NUM>, <NUM> may include a touch sensitive screen for selecting inputs that generally correspond to the functionality of the existing switches and knobs. Alternatively, such inputs may correspond to sets of functions that previously required multiple switch and / or knob inputs. In at least one embodiment, one or more of the displays <NUM>, <NUM>, or some portion thereof, may include a relatively fixed layout while other displays <NUM>, <NUM> or portions of displays <NUM>, <NUM> may be reconfigurable via touch inputs. Furthermore, layouts may be operator specific or operational phase specific such that layouts may be automatically reconfigured to present inputs in an ergonomically efficient way at each operational phase.

Referring to <FIG>, a graphical user interface <NUM> according to an exemplary embodiment is shown. The user interface <NUM> may be divided into a layout display control element <NUM> and one or more informational display elements <NUM>, <NUM>. In at least one embodiment, the layout display control element <NUM> may comprise a control group for selecting layouts to render in the one or more informational display elements <NUM>, <NUM>. Each layout may be associated with one of the informational display elements <NUM>, <NUM> by default, and may be permanently or temporarily reconfigurable to another of the informational display elements <NUM>, <NUM>.

In at least one embodiment, one of the informational display elements <NUM>, <NUM> may include comms controls. Furthermore, one of informational display elements <NUM>, <NUM> may include a fuel widget or layout element <NUM> generally corresponding to fuel transfer metrics; likewise, a boom widget of layout element <NUM> may generally correspond to the disposition and orientation of the boom arm. Such fuel and boom layout elements <NUM>, <NUM> may be generally associated with a singular layout element that may be selected by the operator; alternatively, or in addition, fuel and boom layout elements <NUM>, <NUM> may be associated with an operational phase to automatically render during that operational phase and de-render outside of that operational phase to remove unneeded visual clutter and improve situational awareness. In at least one embodiment, data elements that may be displayed in one or more of the available layouts may be bounded by safe operational ranges; the processor rendering the user interface <NUM> may determine when an undisplayed data point is outside of a bounded range and provide a indicator to the operator such as illuminating a corresponding control in the layout display control element <NUM> or automatically rendering the corresponding layout.

In at least one embodiment, the layout display control element <NUM> and one or more informational display elements <NUM>, <NUM> may be repositionable by an operator such as via tap-and-hold.

In at least one embodiment, data corresponding to data being displayed, and / or actual recorded representations may be recorded in a log. Alternatively, or in addition, data and representations of undisplayed data (data corresponding to layouts not being rendered) may be recorded in the log also.

In at least one embodiment, one or more of the layouts may include lighting controls. Alternatively, or in addition, lighting controls may be persistent in one or more of the informational display elements <NUM>, <NUM> and / or the layout display control element <NUM>. Lighting controls may be associated with an operation phase such that light levels are set to predetermined levels at each operational phase, potentially based on a time of day and / or mission requirements. Increment and decrement controls may allow light levels to be set at certain predefined levels. Lighting controls may include light positioning where available. In at least one embodiment, certain lights may be dedicated to giving the pilot of the fueling aircraft directional signals; those lights may be automated according to positioning algorithms, potentially utilizing one or more onboard cameras, with the option to allow an operator to give further instructions by manipulating a touch screen control that is translated to light indicators.

Referring to <FIG>, a graphical user interface <NUM> according to an exemplary embodiment is shown. In at least one embodiment, the user interface <NUM> may be rendered as a predefined layout including boom arm position data <NUM>, boom arm controls <NUM> and indicators, and fuel transfer data <NUM>. Such user interface <NUM> may be rendered in response to a specific refueling operation phase.

In at least one embodiment, a layout display control element <NUM> comprises a control group for selecting layouts to render in the user interface <NUM>, including the boom arm position data <NUM>. In at least one embodiment, layout display control element <NUM> may be repositioned by an operator; furthermore, individual data display and control elements <NUM>, <NUM>, <NUM> may be repositioned within the larger layout. Such repositioning may be recorded along with an operational phase and an operator identify such that in subsequent refueling operations, the repositioned elements <NUM>, <NUM>, <NUM> will be rendered in those new positions.

Referring to <FIG>, a graphical user interface according to an exemplary embodiment is shown. The user interface <NUM> may be divided into a layout display control element <NUM> and one or more informational display elements <NUM>, <NUM>. In at least one embodiment, the layout display control element <NUM> may comprise a control group for selecting layouts to render in the one or more informational display elements <NUM>, <NUM>. Each layout may be associated with one of the informational display elements <NUM>, <NUM> by default, and may be permanently or temporarily reconfigurable to another of the informational display elements <NUM>, <NUM>.

In at least one embodiment, when different layouts are selected from the layout display control element <NUM>, such new layouts may be rendered in a predetermined informational display element <NUM>, <NUM>; replacing whatever layouts were previously rendered there. Such replacement may enhance situational awareness my removing unneeded or undesired layouts. Furthermore, a processor may automatically determine which layouts to render based on the operation phase of the refueling operation. Where an operator selects other than the automatically determine layout, the processor may record the altered layout paradigm for subsequent operations.

Referring to <FIG>, a graphical user interface according to an exemplary embodiment is shown. The user interface <NUM> may be divided into a layout display control element <NUM> and one or more informational display elements <NUM>, <NUM>. At least one of the informational display elements <NUM>, <NUM> may render a boom arm position graphic derived from one or more image streams from cameras on the refueling aircraft. The graphic may comprise one or more boom arm boundary indicators <NUM> defined by the maximum movement range of the boom arm as would be visible in the image stream. Furthermore, a boom arm position indicator <NUM> may also be rendered.

In at least one embodiment, the one or more boom arm boundary indicators <NUM> and boom arm position indicator <NUM> may be rendered with real-time positions in the informational display element <NUM> corresponding to where they would be displayed as an overlay to a streaming image, but the streaming image is not rendered.

Referring to <FIG>, a graphical user interface according to an exemplary embodiment is shown. The user interface <NUM> may be divided into a layout display control element <NUM> and one or more informational display elements <NUM>, <NUM>. At least one of the informational display elements <NUM>, <NUM> may render a boom arm position graphic derived from one or more image streams from cameras on the refueling aircraft. The graphic may comprise one or more boom arm boundary indicators <NUM> defined by the maximum movement range of the boom arm as would be visible in the image stream rendered over the streaming image.

In at least one embodiment, features of the refueling aircraft may be identified via reference to a database of aircraft, image processing algorithms such as edge detection, a trained neural network, etc., and called out in the rendered overlay. Upon identifying the refueling aircraft, either automatically or via selection by an operator, appropriate gauge limits may be identified and continuously or periodically monitored with corresponding indications whenever a gauge is outside such limits.

Claim 1:
An in-flight refueling boom pod comprising:
a plurality of displays (<NUM>), at least one being touch sensitive;
a data storage element and
at least one processor (<NUM>) in data communication with the plurality of displays, the data storage element, and a memory storing processor executable code for configuring the at least one processor (<NUM>) to:
determine an operation phase of a refueling operation;
determine one or more data and control layouts corresponding to the operational phase;
rendering the one or more data and control layouts on the plurality of displays during the operational phase;
and characterized by further being configured to receive an input via the touch sensitive display indicating a repositioning of a data and control layout;
associate the repositioned data and control layout with an operator identity and the operational phase; and
record a location of the repositioned data and control layout and associated operator identity and operational phase in the data storage element.