Patent Description:
Defensive Aids Systems of military vehicles generically consist of a missile warner, means for counter measures, and a display device, along with some controlling apparatus. In the past, aircraft in particular used to employ warners for flying objects based on radar or ultraviolet sensors, and counter measures included e.g. flares against infrared-guided and chaffs against radar-guided flying objects. Over the past decade, missile warners have increasingly turned towards sensors operating in the infrared part of the spectrum.

In this context, <CIT>,discloses a method for transitioning a target from a missile warning system to a fine tracking system. Registered features surrounding a threat identified in an image of the missile warning system are used to identify a location of the threat within an image of the fine tracking system.

Furthermore, <CIT> discloses an unmanned a weapon system providing automated tracking and prosecution of targets such as unmanned aerial vehicles at close ranges. Images of the target may be provided on a display for viewing by an observer, and an identification camera may provide a higher quality image of the target for a visual identification.

Infrared sensors are simultaneously employed for panoramic night vision, as the infrared frequencies predominantly used by these sensors also form the wave band of common night vision devices. As it is imperative for a missile warner to cover the complete sphere around the aircraft, night vision in every direction can therefore be provided. The number of sensors on an aircraft is usually limited (typically, missile warners comprise between three and six sensors), and each sensor will therefore need to cover a relatively large solid angle in order to achieve a covering of the full sphere. In consequence, the angular resolution provided by these sensors is low, which in turn limits the detection/recognition and identification ranges of these sensors. Therefore the panoramic night vision provided by sensors of the missile warner is used in particular for orientation (indicating e.g. the horizon, or large objects like mountains or houses), and provides assistance for take-off and landing, as e.g. in whiteout or brownout situations. However, due to their low resolution, these sensors prove inadequate for reconnaissance purposes. This problem exists in various degrees also on vehicles other than aircraft.

Therefore there exists a demand for a cost-effective and efficient way to provide better resolution to the image obtained from the missile warner sensors. Further constraints on systems intended to satisfy this demand can e.g. derive from the weight of additional equipment, or from the readiness with which the pilot or driver of the vehicle has access to the data.

At least some of the above-mentioned problems are solved by an apparatus according to claim <NUM>, a system according to claim <NUM>, a method according to claim <NUM>, and a computer program product according to claim <NUM>. The dependent claims refer to further advantageous realizations for the subject matters of the independent claims.

The present invention relates to an apparatus to improve a situational awareness of a pilot or driver controlling a vehicle by means of a control appliance, the control appliance comprising a display for depicting surroundings of the vehicle, and the vehicle comprising a missile warner and a sensor for fine tracking (FTS). The FTS is configured to provide high-resolution images for a tracking of approaching missiles detected by the missile warner. The apparatus comprises a control unit configured to couple a directable line of sight of the FTS with the display, and to employ the high-resolution images of the FTS to improve the depiction of the surroundings of the vehicle on the display. The control unit is furthermore configured to direct the line of sight of the FTS between times where the FTS is employed in a counter measure against an approaching missile.

Here, the term vehicle is understood to be broadly defined, and may stand for any land-based, airborne, sea-based or amphibious manned or unmanned locomotive machine. The control appliance can be for a pilot or driver who is on board of the vehicle, or remote from the vehicle as e.g. in a case where the vehicle is a drone. Embodiments provide an FTS focused on an angle of ten degrees or less, or e.g. of two or five degrees, around a central line of sight, with resolutions at least <NUM> times, but often <NUM> to <NUM> or more times higher than the resolution achieved by means of sensors of the missile warner. The line of sight can be pointed in any direction; it is understood that the possible directions of the line of sight of the FTS should merely encompass at least a part of the directions to the depicted surroundings. Furthermore, it is not required that the surroundings depicted on the display are derived from sensors of the missile warner; data from other sensors or cameras, both on board or remote from the vehicle, and operating both in the infrared or in other wave bands (such as e.g. in the visible domain), as well as maps or mere geographical data may equally be employed to depict the surroundings on the display.

Optionally, the control unit is configured to direct the line of sight of the FTS.

It is understood that the control unit is not necessarily the only device adapted to point the FTS.

Optionally, the control unit is furthermore configured to couple to the control appliance such that the driver or pilot can indicate a desired line of sight of the FTS.

This can be achieved e.g. by means of a pointing device, like a control column or a joystick, or by a computer mouse.

Optionally the control unit is configured to track a line of sight of the driver or pilot, and to align the line of sight of the FTS with the line of sight of the driver or pilot.

Such eye trackers are common appliances in controlling equipment of military vehicles, in particular of manned aircraft.

The present invention also relates to a system for imaging surroundings of a vehicle, comprising a missile warner and a FTS configured to provide high-resolution images for a tracking of approaching missiles detected by the missile warner, a control appliance comprising a display for depicting surroundings of the vehicle, and an apparatus as described above.

Optionally the missile warner comprises one or more infrared sensors, configured to produce sensor data for depicting a surround (panoramic, or full-sphere) view from the vehicle, and the system comprises a Directional Infrared Counter Measures (DIRCM) appliance which includes the FTS as one of its parts, wherein the line of sight of the FTS is directable and the FTS is a further infrared sensor, and wherein the DIRCM appliance is configured to initiate, using the high resolution images of the FTS, counter measures to protect the vehicle against threatening flying objects.

Besides missile warners operating in the infrared, recent years have seen an increased use of DIRCM as replacements or as supplements of counter measures such as e.g. flares against infrared-directed flying objects. Typically, DIRCM appliances direct a laser beam at the homing head of the approaching flying object, which is then confused by using a suitable modulation of the beam, to the effect that seeking out the aircraft is rendered impossible for the homing head, and that the flying object loses its orientation. Maximising the energy density of the laser beam at the homing head requires a small divergence of the laser beam. The beam must therefore be pointed very precisely, which in consequence requires a very precise sighting. The angular resolution of the missile warner sensors is usually insufficient for this task, such that the DIRCM appliance typically relies on its own FTS, which is highly focused but provides a high angular resolution. Present realisations of DIRCM comprise an FTS operating in the same infrared wave band as the sensors of the missile warner, as the goal of detection and tracking of an approaching flying object is the same for both missile warner and DIRCM sensors. The FTS can be coupled to the DIRCM laser beam, and can be pointed in any direction by means of an alignment facility involving e.g. a gimbal system and contained in every DIRCM appliance.

Optionally the DIRCM appliance is configured to perform a precise classification of the flying object based on the high-resolution images provided by the FTS.

Classification of flying objects in the infrared is based on the data of the sensors which detected the flying object, i.e. by data of the missile warner sensor or sensors. Once the FTS has been directed at the flying object, its data can be used for a more precise classification.

Optionally, the missile warner comprises one or more infrared sensors configured to produce sensor data for depicting a surround view of the vehicle, and the system further comprises a directional visible counter measures appliance, which includes the FTS as a part, and wherein the line of sight of the FTS is directable and the FTS includes a further sensor operating in the visible part of the electromagnetic spectrum. Similarly to the DIRCM appliance before, the directional visible counter measures appliance is configured to initiate, using the high resolution images of the FTS, counter measures to protect the vehicle against flying objects.

Optionally, the directional visible counter measures appliance is configured to perform a more precise classification of the flying object based on the high-resolution images provided by the FTS.

According to the invention, the control unit is configured to direct the line of sight of the FTS between times where the FTS is employed in the counter measure against an approaching missile or times where the DIRCM appliance operates the FTS.

The control unit should only operate the FTS at times where the DIRCM system does not direct the line of sight of the FTS. This can be achieved e.g. by subordinating requests to direct the FTS from the control unit to requests from the DIRCM.

Optionally, the display can be a helmet-mounted display, a multi-function display, or a head-up display or any other display providing the image to the pilot. These types of displays are common in controlling equipment of military vehicles.

Optionally, the DIRCM appliance and/or the directed visible counter measures appliance is/are configured, during a counter measure against a flying object, to identify launch coordinates where the flying object was launched, and wherein the FTS, after being employed for the counter measure operation by the DIRCM appliance and/or by the directed visible counter measures appliance, is configured to automatically direct its line of sight at the identified launch coordinates. This provides the advantage that the pilot or driver can immediately examine the launch site.

This invention also relates to a method for improving a situational awareness of a pilot or driver of a vehicle, the vehicle comprising a display for depicting surroundings of the vehicle, a missile warner, and a sensor for fine tracking (FTS), wherein the FTS is configured to provide high-resolution images for a tracking of approaching missiles detected by the missile warner, with the method comprising the steps.

The invention furthermore relates to a computer program product comprising stored software code adapted to perform the steps of this method when executed on a data processing system.

Embodiments provide in particular the advantage that a DIRCM, which hitherto is only active for a brief time during a counter measure action, can be employed for different tasks in other times. In particular, the high resolution image of the DIRCM appliance, or the FTS respectively, is made available to external systems for further operations. These comprise, in particular, the following:
In combination with a pointing device, like e.g. a joystick or an eye tracker of the pilot or driver, the line of sight of the FTS can be coupled to that device, such that an observer will always get a high-resolution image from the selected direction or the pilot or driver's line of sight embedded into and/or superimposed onto e.g. a multi-function display, or a helmet-mounted display. In this way, depending on the wave band the camera is operating in, a video in the visible domain (<NUM> - <NUM>), in the near infrared (<NUM> - <NUM>), short-wave infrared (<NUM> - <NUM>), mid-wave infrared (<NUM> - <NUM>) or far infrared (<NUM>-<NUM>) can be displayed.

For an FTS operating in the mid-wave or far infrared, the line of sight of the FTS can be coupled to a pointing device, such that pilot or driver always obtains a night-vision capable infrared image from his or her line of sight embedded into or superimposed onto a multi-function display or a helmet-mounted display.

In addition, the high-resolution image can be embedded into the panoramic view obtained from the missile warner sensors. In this way, a low-resolution panoramic view, suited e.g. for mere orientation, is combined with a highly resolved image, which admits a much more detailed view over longer distances in a limited area which for example can be used for a digital zoom. This embodiment admits an extension of the range of detection, recognition and identification as determined according to STANAG <NUM> by a factor of up to <NUM>.

Furthermore, the high-resolution sensor can be employed as a sensor for verification. This means that the (pre-)alarm generated by the missile warning system is investigated further by means of the high-resolution sensor, and therefore can be confirmed as an alarm or dismissed as a false alarm with greater precision due to the higher resolution. In order to achieve this goal, a classifier (distinguishing a threatening from a non-threatening flying object or other light sources) must be integrated in the image processing of the FTS into the DIRCM.

Various embodiments of the present invention will be described in the following by way of examples only, and with respect to the accompanying drawings, in which:.

Accordingly, while examples are capable of various modifications and alternative forms, the illustrative examples in the figures will herein be described in detail. It should be understood, however, that there is no intent to limit examples to the particular forms disclosed, but on the contrary, examples are to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure. Like reference numbers refer to like or similar elements throughout the description of the figures.

The terminology used herein is for the purpose of describing illustrative examples only and is not intended to be limiting. As used herein, the singular forms "a, " "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes" and/or "including," when used herein, specify the presence of stated features, steps, operations, elements and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which examples belong.

<FIG> shows an apparatus to improve a situational awareness of a pilot or driver controlling a vehicle by means of a control appliance <NUM>, the control appliance <NUM> comprising a display <NUM> for depicting surroundings of the vehicle, and the vehicle comprising a missile warner <NUM> and a sensor for fine tracking (FTS) <NUM>. The FTS <NUM> is configured to provide high-resolution images <NUM> for a tracking of approaching missiles detected by the missile warner <NUM>. The apparatus comprises a control unit <NUM> configured to couple a directable line of sight of the FTS <NUM> with the display <NUM> , and to employ the high-resolution images of the FTS <NUM> to improve the depiction of the surroundings of the vehicle on the display <NUM>.

In this embodiment, the missile warner <NUM> exemplarily comprises a wide-angle sensor <NUM>, which operates in the infrared and whose field of view is understood to cover a large solid angle of the sphere surrounding the vehicle, at a correspondingly low angular resolution. A typical missile warner <NUM> requires four to six such wide-angle sensors <NUM> in order to cover the full sphere around the vehicle. The missile warner <NUM> is furthermore depicted with a classifier <NUM>, which holds the control devices configured to detect, locate and identify flying objects in the data collected by the wide-angle sensor <NUM>, to a precision constrained by the low angular resolution of the wide-angle sensor <NUM>. The classifier <NUM> is adapted to detect a flying object and to issue an alarm signal, upon which counter measures are initiated, which may be performed automatically or under a control of the pilot or driver. The classifier <NUM> is furthermore configured to transmit data collected about the flying object to other systems, as e.g. to counter measure devices, or to the driver or pilot.

The data of the wide-angle sensor <NUM> is furthermore transmitted to a display <NUM> in the control appliance <NUM> of the vehicle, where it is processed and used as a source for panoramic night vision <NUM>. Due to the low resolution of the wide-angle sensors, this image is not sufficient for reconnaissance purposes. Larger objects - e.g. landmarks like mountains, or structures like buildings - are distinguishable, and the low-resolution image <NUM> of the surroundings can be beneficial for an orientation of the pilot or driver regarding the position of the vehicle in the landscape, or for cases of heavy weather or environmental conditions like e.g. whiteouts or brownouts.

One of the counter measures coupled to the missile warner consists of a Directional Infrared Counter Measure (DIRCM) appliance <NUM>, which comprises the FTS <NUM> together with an exemplary infrared laser device <NUM>. Alternatively or in addition to the DIRCM appliance <NUM>, the counter measures, and in particular the FTS <NUM>, could also operate in the visible range of the electromagnetic spectrum. The following description will focus on infrared counter measures, but this should not be construed to limit the counter measures to infrared wavelengths.

The DIRCM appliance <NUM> reacts to an alert of the missile warner <NUM>, and directs an infrared laser beam e.g. at a homing head of a heat-seeking missile, with the aim of confusing the orientation of the missile such that it misses its target. To this effect, the laser beam is itself highly collimated - e.g. to less than a degree in angular divergence - and therefore needs to be directed at the flying object in a very precise way. In order to achieve this precise alignment, the DIRCM appliance <NUM> operates the FTS <NUM> as one of its parts. The FTS <NUM> - which in this embodiment may e.g. collect data in the infrared part of the spectrum, as does the wide-angle sensor <NUM> - covers only a very small solid angle, but achieves a high resolution, and can therefore also distinguish objects more precisely and/or at a greater distance than the wide-angle sensor <NUM> of the missile warner <NUM>. A DIRCM control unit <NUM> is depicted which couples the FTS <NUM> and the laser <NUM>, and directs both of them at the flying object for example by using a gimbaled system.

Conventionally, the FTS <NUM> is not employed by systems other than the DIRCM appliance. The DIRCM appliance <NUM> operates the laser <NUM> and the FTS <NUM> only in reaction to a prior warning signal of the missile warner <NUM>. The FTS <NUM>, however, is typically configured to constantly collect data. The apparatus, comprising the control unit <NUM> and connections to other components of the vehicle, is configured to exploit this data in order to improve the low-resolution image <NUM> which is provided, in this embodiment, by the wide-angle sensors <NUM> of the missile warner <NUM>. In the present embodiment, this improvement consists of a high-resolution image <NUM> in a small area of the display corresponding to the direction of the FTS <NUM>. A pointing device - as here e.g. a joystick <NUM> - is part of the control appliance <NUM>, and adapted to direct the FTS <NUM> as desired by the pilot or driver. As counter measures (e.g. in the case of missiles) are highly time-sensitive operations, which can surpass the reaction capacity of the driver or pilot, the operation of the DIRCM appliance <NUM> triggered by the alarm of the missile warner <NUM> should have priority over the input of the driver on the direction of the FTS <NUM>. For this reason, the control unit <NUM> and/or the DIRCM appliance <NUM> can be configured to prioritise DIRCM commands to the FTS <NUM> if the DIRCM is operational.

<FIG> shows the steps of a method for improving a situational awareness of a pilot or driver of a vehicle according to the present invention. The vehicle comprises a display <NUM> for depicting a surround of the vehicle, a missile warner <NUM>, and an FTS <NUM>. The FTS <NUM> is configured to provide high-resolution images for tracking of missiles detected by the missile warner <NUM>. The method comprises a step S10 of coupling a directable line of sight of the FTS <NUM> with the display <NUM>, followed by a step S20 of employing the high-resolution images of the FTS <NUM> in order to improve the depiction of surroundings of the vehicle on the display <NUM>.

It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its scope.

Furthermore, while each embodiment may stand on its own as a separate example, it is to be noted that in other embodiments the defined features can be combined differently, i.e. a particular feature descripted in one embodiment may also be realized in other embodiments. Such combinations are covered by the disclosure herein unless it is stated that a specific combination is not intended.

Claim 1:
An apparatus to improve a situational awareness of a pilot or driver, wherein the pilot or driver controls a vehicle by means of a control appliance (<NUM>), the control appliance (<NUM>) comprising a display (<NUM>) for depicting surroundings of the vehicle, and the vehicle comprising a missile warner (<NUM>) and a sensor for fine tracking, FTS, (<NUM>), wherein the FTS (<NUM>) is configured to provide high-resolution images for a tracking of an approaching missile detected by the missile warner (<NUM>), the high-resolution images having a resolution higher than a resolution achieved by means of sensors of the missile warner (<NUM>) and encompassing a part of the depicted surroundings, the apparatus characterized by:
a control unit (<NUM>) configured to couple a directable line of sight of the FTS (<NUM>) with the display (<NUM>), to employ the high-resolution images of the FTS (<NUM>) to improve the depiction of the surroundings of the vehicle on the display, and to direct the line of sight of the FTS (<NUM>) between times where the FTS (<NUM>) is employed in a counter measure against an approaching missile.