Patent Description:
A synthetic vision system (SVS) is a computer-mediated reality system for aerial vehicles, that uses a 3D perspective view to provide pilots with clear and intuitive means of understanding their flying environment. Synthetic vision provides situational awareness to the operators by using terrain, obstacle, geo-political, hydrological and other databases. A typical SVS application uses a set of databases stored on board the aircraft, an image generator computer, and a display. A navigation solution is obtained through the use of GPS and inertial reference systems. If perspective view projection per distance between an object and a view point for the object is used, the size of the displayed object at the view point may be scaled to be proportional to the distance. This can lead to the display size of the object being too large for proper visualization when the object is close to the aircraft and too small for proper visualization when the object is far away from the aircraft.

Hence, it is desirable to provide a system and method that applies a scaling algorithm for 3D display of dynamic objects for dynamic/optimal sizing of objects based on visual condition/requirement. Furthermore, other desirable features and characteristics of the present invention will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and the foregoing technical field and background.

Patent document <CIT> discloses flight display systems that present a pilot with a three dimensional view of an area within a potential flight path of an aircraft. The three dimensional view may include representations of potential obstacles and avoidance zones surrounding the potential obstacles. The view may also include a cardinal compass representation aligned with the visual horizon. Aircraft traffic obstacle symbols may be variable based on the type of aircraft represented.

The solution is provided by the features of the independent claims. Variations are as described by the dependent claims.

<FIG> is a block diagram depicting an example display system <NUM> for displaying a three-dimensional view in front of an aerial vehicle, such as an airplane, helicopter, UAV or other aircraft. The example display system <NUM> includes a display size controller <NUM>, one or more displays <NUM>, and one or more object sensors <NUM>. The one or more displays <NUM> may include a primary flight display (PFD) <NUM>, which may display a three-dimensional (<NUM>-D) synthetic vision system (SVS) <NUM> and a two-dimensional (<NUM>-D) horizontal situation indicator (HSI) <NUM>. The one or more displays <NUM> may also include a navigational display (ND) <NUM>, which may display <NUM>-D and <NUM>-D content, and a HUD <NUM>, which may also display <NUM>-D and <NUM>-D content. The one or more object sensors <NUM> are for obtaining object data regarding objects within the travel path of the aerial vehicle so that the objects may be displayed in the field of view of one or more of the displays <NUM>. The object sensors may comprise airborne vehicle sensors such as ADSB receivers, radar, lidar, cameras, and others.

The display size controller <NUM> includes at least one processor and a computer-readable storage device or media encoded with programming instructions for configuring the controller. The processor may be any custom-made or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), an auxiliary processor among several processors associated with the controller, a semiconductor-based microprocessor (in the form of a microchip or chip set), any combination thereof, or generally any device for executing instructions.

The computer readable storage device or media may include volatile and nonvolatile storage in read-only memory (ROM), random-access memory (RAM), and keep-alive memory (KAM), for example. KAM is a persistent or non-volatile memory that may be used to store various operating variables while the processor is powered down. The computer-readable storage device or media may be implemented using any of a number of known memory devices such as PROMs (programmable read-only memory), EPROMs (electrically PROM), EEPROMs (electrically erasable PROM), flash memory, or any other electric, magnetic, optical, or combination memory devices capable of storing data, some of which represent executable programming instructions, used by the controller <NUM>.

The display size controller <NUM> is configured by programming instructions on non-transitory computer readable media to: cause the display of one or more objects as graphical objects in a <NUM>-dimensional graphical representation of real space, wherein the display size of an object is determined based on a non-linear projection per distance between the object and view point that is scaled based on a continuous, non-linear scaling function, which causes the object to be displayed smaller at certain distances and larger at other distances than it would if only linear scaling (e.g., inverse linear scaling performed by an inverse linear function) were applied. This may allow the display size of the object to not be too big when the object is close to the aircraft and to not be too small when the object is far away from the aircraft.

As used herein a linear function (which includes an inverse linear function) is a function whose graph is a straight line, that is a polynomial function of degree one or zero. A non-linear function is a function whose graph is not a straight line.

The continuous, non-linear scaling function may include one or more different compensation scaling functions. In this example, the plurality of different compensation scaling functions includes a first compensation scaling function for normal conditions and a second compensation scaling function for alert or bad visual conditions. In this example, the first compensation scaling function used to calculate a scaling factor for a specific distance x is d<NUM>(a + (d*x-b)/(e*x+c)), wherein d<NUM>, a, b, c, d and e are constants and x is the distance from the object to the view point. In this example, the second compensation scaling function used to calculate a scaling factor for a specific distance x is d<NUM>(a + (d*x-b)/(e*x+c)), wherein d<NUM>, a, b, c, d and e are constants, d<NUM> is not equal to d<NUM>, and x is the distance from the object to the view point. In this example, d<NUM> > d<NUM>, thereby causing the display size of objects at a certain distance to be larger when the second compensation scaling function versus when the first compensation scaling function is used. In this example, the display size of an object is determined by a linear projection scaling function (an inverse linear scaling function in particular) plus one of the plurality of different compensation scaling functions. In this example, the inverse linear projection scaling function comprises <NUM>/x, where x is the distance from the object to the view point.

In this example, the display system <NUM> is a flight deck display system on board an aircraft. In this example, the flight deck display system <NUM> includes a PFD <NUM>. In this example, the display100 includes a synthetic vision system <NUM> displayed on a PFD <NUM>.

The controller <NUM> is further configured to detect a threat level posed by a detected object and adjust the position of the object on the display <NUM> based on the determined threat level. The controller <NUM> may adjust the position of the object further away from the center of the projected travel path of the aerial vehicle (e.g., aircraft) when the threat level is considered low. The controller <NUM> may adjust the position the object closer to the center of the projected travel path of the aircraft when the threat level is considered high. The travel path of the aircraft may be along a runway and the controller <NUM> may adjust the position of the object further away from the center of the runway when the threat level is considered low. The controller <NUM> may adjust the position of the object closer to the center of the runway when the threat level is considered high.

<FIG> is a process flow chart depicting an example process for displaying ground objects in the travel path of an aircraft. The order of operation within the process <NUM> is not limited to the sequential execution as illustrated in the figure, but may be performed in one or more varying orders as applicable and in accordance with the present disclosure.

The example process <NUM> includes obtaining object data regarding sensed ground objects around the travel path of the aircraft from one or more aircraft sensors (operation <NUM>). The object data may be obtained, for example, from an object sensor <NUM>.

The example process <NUM> includes determining if the sensed ground objects are within a field of view of a three-dimensional view in front of the aircraft displayed on a display screen in the aircraft flight deck (operation <NUM>). The display screen, for example, may be from a display system <NUM>, which may include a primary flight display (PFD) <NUM>, a three-dimensional (<NUM>-D) synthetic vision system (SVS) <NUM>, a two-dimensional (<NUM>-D) horizontal situation indicator (HSI) <NUM>, a navigational display (ND) <NUM>, and/or a HUD <NUM>.

The example process <NUM> includes selecting a graphical object display size, for each graphical object representing a ground object within the field of view of the three-dimensional view (operation <NUM>). This includes selecting, for each graphical object representing a ground object within the field of view of the three-dimensional view, a graphical object display size for the ground object, wherein the display size of a ground object is determined based on a non-linear projection per distance between the ground object and the view point that is scaled based on a continuous, non-linear scaling function, which causes the ground object to be displayed smaller at certain distances and larger at other distances than it would if only linear scaling (e.g., inverse linear scaling performed by an inverse linear function) were applied, so that the display size of the object is not too big when the ground object is close to the aircraft and not too small when the ground object is far away from the aircraft.

The continuous, non-linear scaling function includes one of a plurality of different compensation scaling functions. The plurality of different compensation scaling functions include a first compensation scaling function for normal conditions and a second compensation scaling function for alert or bad visual conditions. The first compensation scaling function used to calculate a scaling factor for a specific distance x may be di(a + (d*x-b)/(e*x+c)), wherein d<NUM>, a, b, c, d and e are constants and x is the distance from the object to the view point. The second compensation scaling function used to calculate a scaling factor for a specific distance x may be d<NUM>(a + (d*x-b)/(e*x+c)), wherein d<NUM>, a, b, c, d and e are constants, d<NUM> is not equal to di, and x is the distance from the object to the view point. d<NUM> may be > d<NUM>, thereby causing the display size of objects at a certain distance to be larger when the second compensation scaling function versus when the first compensation scaling function is used. The display size of an object may be determined by an inverse linear projection scaling function plus one of the plurality of different compensation scaling functions. The inverse linear projection scaling function may include <NUM>/x, where x is the distance from the object to the view point.

The example process <NUM> includes causing the display on the display screen of the one or more graphical objects with the selected display size (operation <NUM>).

The example process <NUM> may further include detecting a threat level posed by the ground object and adjusting the position of the ground object on the display based on the determined threat level (operation <NUM>). The adjusting may include adjusting the position of the ground object further away from the center of the projected travel path of the aircraft when the threat level is considered low. The adjusting may further include adjusting the position of the object closer to the center of the projected travel path of the aircraft when the threat level is considered high. The travel path of the aircraft may be along a runway and the adjusting further may include adjusting the position of the object further away from the center of the runway when the threat level is considered low. The adjusting may further include adjusting the position of the object closer to the center of the runway when the threat level is considered high.

Non-transitory computer readable media encoded with processor executable programming instructions is disclosed (e.g., part of controller <NUM>). When the processor executable programming instructions are executed by a processor, a method (e.g., process <NUM>) is performed. The method includes: obtaining object data regarding sensed objects around the travel path of the aircraft from one or more aircraft sensors; determining if the sensed objects are within a field of view of a three-dimensional view in front of the aircraft displayed on a display screen in the aircraft; selecting, for each graphical object representing an object, a graphical object display size for the object, wherein the display size of an object is determined based on a non-linear projection per distance between the object and the view point that is scaled based on a continuous, non-linear scaling function, which causes the object to be displayed smaller at certain distances and larger at other distances than it would if only linear scaling were applied, wherein the display size of the object is not too big when the object is nearby to the aircraft and not too small when the object is far away; and causing the display of the one or more graphical objects on the display screen.

<FIG>, <FIG>, <FIG>, <FIG>, <FIG> are diagrams depicting a detected ground object overlaid on a <NUM>-D view in front of an aircraft, with linear scaling applied for sizing the ground object. <FIG>, <FIG>, <FIG>, <FIG>, <FIG> are diagrams depicting a detected ground object overlaid on a <NUM>-D view in front of an aircraft, with non-linear scaling applied for sizing the ground object.

<FIG> illustrates the display on an SVS display of an example <NUM>-D object <NUM> that is <NUM> feet from the view point with only an inverse linear projection function applied for sizing. <FIG> illustrates the display on an SVS display of the same example <NUM>-D object <NUM> that is <NUM> feet from the view point with both an inverse linear projection function and a compensation scaling function applied for sizing. In this example, the object <NUM> is displayed smaller than the object <NUM> since the objects are closer to the aircraft.

<FIG> illustrates the display on an SVS display of an example <NUM>-D object <NUM> that is <NUM> feet from the view point with only an inverse linear projection function applied for sizing. <FIG> illustrates the display on an SVS display of the same example <NUM>-D object <NUM> that is <NUM> feet from the view point with both an inverse linear projection function and a compensation scaling function applied for sizing. In this example, the object <NUM> and <NUM> are displayed at approximately the same size.

<FIG> illustrates the display on an SVS display of an example <NUM>-D object <NUM> that is <NUM> feet from the view point with only an inverse linear projection function applied for sizing. <FIG> illustrates the display on an SVS display of the same example <NUM>-D object <NUM> that is <NUM> feet from the view point with both an inverse linear projection function and a compensation scaling function applied for sizing. In this example, the object <NUM> is displayed larger than the object <NUM> since the objects are further away from the aircraft.

<FIG> and <FIG> are diagrams depicting example graphs of scaling factor versus distance of an object to a view point for example compensation scaling functions. <FIG> and <FIG> are diagrams depicting example graphs of display size on screen versus distance of an object to a view point when (i) only an example inverse linear projection is used (at <NUM>, <NUM>), (ii) an example inverse linear projection function plus a first example compensation scaling function is used (at <NUM>, <NUM>), e.g., for normal conditions, and (iii) an example inverse linear projection function plus an second example compensation scaling function is used (at <NUM>, <NUM>), e.g., for an alert or bad visual conditions.

<FIG> and <FIG> are diagrams depicting a calculated virtual object around the travel path of an aircraft that is overlaid on a <NUM>-D view in front of the aircraft, with non-linear scaling applied for sizing the calculated virtual object. <FIG> illustrates the display on an SVS display of an example calculated virtual <NUM>-D object <NUM> that is in front of the view point with both an inverse linear projection function and a compensation scaling function applied for sizing. <FIG> illustrates the display on an SVS display of an example calculated virtual <NUM>-D object <NUM> that is in front of the view point with both an inverse linear projection function and a compensation scaling function applied for sizing.

In one embodiment, a display system for displaying a three-dimensional view in front of an aerial vehicle is disclosed. The display system comprises a display screen for displaying the three-dimensional view, one or more sensors for obtaining object data regarding objects within the travel path of the aerial vehicle, and a controller for receiving the object data from the one or more sensors and for providing input to the display screen. The controller is configured by programming instructions on non-transitory computer readable media to: cause the display of one or more objects as graphical objects in a <NUM>-dimensional graphical representation of real space, wherein the display size of an object is determined based on a non-linear projection per distance between the object and view point that is scaled based on a continuous, non-linear scaling function, which causes the object to be displayed smaller at certain distances and larger at other distances than it would if only linear scaling (e.g., inverse linear scaling performed by an inverse linear function) were applied. This may allow the display size of the object to not be too big when the object is close to the aerial vehicle and to not be too small when the object is far away from the aerial vehicle.

In one embodiment of the display system, the continuous, non-linear scaling function comprises one of a plurality of different compensation scaling functions.

In one embodiment of the display system, the plurality of different compensation scaling functions comprises a first compensation scaling function for normal conditions and a second compensation scaling function for alert or bad visual conditions.

In one embodiment of the display system, the first compensation scaling function used to calculate a scaling factor for a specific distance x comprises d<NUM>(a + (d*x-b)/(e*x+c)), wherein d<NUM>, a, b, c, d and e are constants and x is the distance from the object to the view point.

In one embodiment of the display system, the second compensation scaling function used to calculate a scaling factor for a specific distance x comprises d<NUM>(a + (d*x-b)/(e*x+c)), wherein d<NUM>, a, b, c, d and e are constants, d<NUM> is not equal to d<NUM>, and x is the distance from the object to the view point.

In one embodiment of the display system, d<NUM> > d<NUM>, thereby causing the display size of objects at a certain distance to be larger when the second compensation scaling function versus when the first compensation scaling function is used.

In one embodiment of the display system, the display size of an object is determined by an inverse linear projection scaling function plus one of the plurality of different compensation scaling functions.

In one embodiment of the display system, the inverse linear projection scaling function comprises <NUM>/x, where x is the distance from the object to the view point.

In one embodiment of the display system, the display system is a flight deck display system on board an aerial vehicle.

In one embodiment of the display system, the flight deck display system comprises a primary flight display (PFD).

In one embodiment of the display system, the display comprises a synthetic vision system displayed on a PFD.

In one embodiment of the display system, the display system is further configured to detect a threat level posed by the object and adjust the position of the object on the display based on the determined threat level.

In one embodiment of the display system, to adjust the position of the object on the display based on the determined threat level, the system is configured to adjust the position of the object further away from the center of the projected travel path of the aerial vehicle when the threat level is considered low.

In one embodiment of the display system, to adjust the position of the object on the display based on the determined threat level, the system is configured to adjust the position of the object closer to the center of the projected travel path of the aerial vehicle when the threat level is considered high.

In one embodiment of the display system, the travel path of the aerial vehicle is along a runway and wherein, to adjust the position of the object on the display based on the determined threat level, the system is configured to adjust the position of the object further away from the center of the runway when the threat level is considered low.

In one embodiment of the display system, to adjust the position of the object on the display based on the determined threat level, the system is configured to adjust the position of the object closer to the center of the runway when the threat level is considered high.

In another embodiment, a method for displaying objects in the travel path of an aerial vehicle is disclosed. The method comprises: obtaining object data regarding sensed objects around the travel path of the aerial vehicle from one or more aerial vehicle sensors; determining if the sensed objects are within a field of view of a three-dimensional view in front of the aerial vehicle displayed on a display screen in the aerial vehicle; selecting, for each graphical object representing an object within the field of view of the three-dimensional view, a graphical object display size for the object, wherein the display size of an object is determined based on a non-linear projection per distance between the object and the view point that is scaled based on a continuous, non-linear scaling function, which causes the object to be displayed smaller at certain distances and larger at other distances than it would if only linear scaling (e.g., inverse linear scaling performed by an inverse linear function) were applied, wherein the display size of the object is not too big when the object is nearby to the aerial vehicle and not too small when the object is far away; causing the display on the display screen of the one or more graphical objects with the selected display size.

In one embodiment of the method, the method further comprises detecting a threat level posed by the object and adjusting the position of the object on the display based on the determined threat level.

In one embodiment of the method, the adjusting comprises adjusting the position of the object further away from the center of the projected travel path of the aerial vehicle when the threat level is considered low.

In one embodiment of the method, the adjusting further comprises adjusting the position of the object closer to the center of the projected travel path of the aerial vehicle when the threat level is considered high.

In one embodiment of the method, the travel path of the aerial vehicle is along a runway and wherein the adjusting further comprises adjusting the position of the object further away from the center of the runway when the threat level is considered low.

In one embodiment of the method, the adjusting further comprises adjusting the position of the object closer to the center of the runway when the threat level is considered high.

In one embodiment of the method, the continuous, non-linear scaling function comprises one of a plurality of different compensation scaling functions.

In one embodiment of the method, the plurality of different compensation scaling functions comprises a first compensation scaling function for normal conditions and a second compensation scaling function for alert or bad visual conditions.

In one embodiment of the method, the first compensation scaling function used to calculate a scaling factor for a specific distance x comprises d<NUM>(a + (d*x-b)/(e*x+c)), wherein d<NUM>, a, b, c, d and e are constants and x is the distance from the object to the view point.

In one embodiment of the method, the second compensation scaling function used to calculate a scaling factor for a specific distance x comprises d<NUM>(a + (d*x-b)/(e*x+c)), wherein d<NUM>, a, b, c, d and e are constants, d<NUM> is not equal to d<NUM>, and x is the distance from the object to the view point.

In one embodiment of the method, d<NUM> > d<NUM>, thereby causing the display size of objects at a certain distance to be larger when the second compensation scaling function versus when the first compensation scaling function is used.

In one embodiment of the method, the display size of an object is determined by an inverse linear projection scaling function plus one of the plurality of different compensation scaling functions.

In one embodiment of the method, the inverse linear projection scaling function comprises <NUM>/x, where x is the distance from the object to the view point.

In another embodiment, a method for displaying objects in the travel path of an aerial vehicle is disclosed. The method comprises: obtaining object data regarding a calculated virtual object around the travel path of the aerial vehicle from one or more aerial vehicle sensors; determining if the calculated virtual object is within a field of view of a three-dimensional view in front of the aerial vehicle displayed on a display screen in the aerial vehicle; selecting, for a graphical object representing the calculated virtual object within the field of view of the three-dimensional view, a graphical object display size for the object, wherein the display size of an object is determined based on a non-linear projection per distance between the object and the view point that is scaled based on a continuous, non-linear scaling function, which causes the object to be displayed smaller at certain distances and larger at other distances than it would if only linear scaling (e.g., inverse linear scaling performed by an inverse linear function) were applied, wherein the display size of the object is not too big when the object is nearby the aerial vehicle and not too small when the object is far away; and causing the display on the display screen of the graphical object with the selected display size.

These aspects and other embodiments may include one or more of the following features. The method may further comprise detecting a threat level posed by the object and adjusting the position of the object on the display based on the determined threat level. The adjusting may further comprise adjusting the position of the object further away from the center of the projected travel path of the aerial vehicle when the threat level is considered low. The adjusting may further comprise adjusting the position of the object closer to the center of the projected travel path of the aerial vehicle when the threat level is considered high. The travel path of the aerial vehicle may be along a runway and the adjusting may further comprise adjusting the position of the object further away from the center of the runway when the threat level is considered low. The adjusting may further comprise adjusting the position of the object closer to the center of the runway when the threat level is considered high. The continuous, non-linear scaling function may comprise one of a plurality of different compensation scaling functions. The plurality of different compensation scaling functions may comprise a first compensation scaling function for normal conditions and a second compensation scaling function for alert or bad visual conditions. The first compensation scaling function used to calculate a scaling factor for a specific distance x may comprise d<NUM>(a + (d*x-b)/(e*x+c)), wherein d<NUM>, a, b, c, d and e are constants and x is the distance from the object to the view point. The second compensation scaling function used to calculate a scaling factor for a specific distance x may comprise d<NUM>(a + (d*x-b)/(e*x+c)), wherein d<NUM>, a, b, c, d and e are constants, d<NUM> is not equal to d<NUM>, and x is the distance from the object to the view point. d<NUM> may be greater d<NUM>, thereby causing the display size of objects at a certain distance to be larger when the second compensation scaling function versus when the first compensation scaling function is used. The display size of an object may be determined by an inverse linear projection scaling function plus one of the plurality of different compensation scaling functions. The inverse linear projection scaling function may comprise <NUM>/x, where x is the distance from the object to the view point.

In another embodiment, non-transitory computer readable media encoded with processor executable programming instructions is disclosed. When the processor executable programming instructions are executed by a processor, a method is performed. The method comprises: obtaining object data regarding sensed objects around the travel path of the aerial vehicle from one or more aerial vehicle sensors; determining if the sensed objects are within a field of view of a three-dimensional view in front of the aerial vehicle displayed on a display screen in the aerial vehicle; selecting, for each graphical object representing an object, a graphical object display size for the object, wherein the display size of an object is determined based on a non-linear projection per distance between the object and the view point that is scaled based on a continuous, non-linear scaling function, which causes the object to be displayed smaller at certain distances and larger at other distances than it would if only linear scaling (e.g., inverse linear scaling performed by an inverse linear function) were applied, wherein the display size of the object is not too big when the object is nearby to the aerial vehicle and not too small when the object is far away; and causing the display of the one or more graphical objects on the display screen.

Claim 1:
A display system for displaying a three-dimensional view in front of an aerial vehicle, the display system comprising:
a display screen for displaying the three-dimensional view;
one or more sensors for obtaining object data regarding objects within a travel path of the aerial vehicle,
a controller for receiving the object data from the one or more sensors, and for providing input to the display screen, the controller configured by programming instructions on non-transitory computer readable media to:
cause the display of one or more objects as graphical objects in a <NUM>-dimensional graphical representation of real space, wherein the display size of an object is determined based on a non-linear projection per distance between the object and a view point that is scaled based on a continuous, non-linear scaling function, which causes the object to be displayed smaller at certain distances and larger at other distances than it would if only linear scaling were applied, characterised in that the continuous, non-linear scaling function comprises one of a plurality of different compensation scaling functions comprising a first compensation scaling function (<NUM>, <NUM>) for normal conditions and a second compensation scaling function (<NUM>, <NUM>) for alert or bad visual conditions, wherein the display size of objects at a certain distance is larger when the second compensation scaling function is used than when the first compensation scaling function is used.