Situational awareness components of an enhanced vision system

A virtual sphere provided by an enhanced vision system includes synthetic imagery filling said virtual sphere and a common view window mapped to a dedicated position within the synthetic imagery. Imagery of the line of sight of a user is displayed in the common view window. By providing the common view window, visual communication between all users may be possible. By connecting a virtual user to the enhanced vision system and by displaying the imagery for the line of sight of the virtual user in the common view window, the workload of a human operator may be reduced and the time line of actions may be shortened. The enhanced vision system of the present invention may be used, but is not limited to, in a military aircraft to enhance the situational awareness of the flight crew.

BACKGROUND OF THE INVENTION

The present invention generally relates to enhanced vision systems and, more particularly, to situational awareness components of an enhanced vision system and a method for enhancing flight crew situational awareness.

The maintenance of situational awareness has always been and is still today a pilot's uppermost concern. Loss of situational awareness is most often a main factor in airplane accidents, which is true for commercial aircraft as well as for military aircraft. One of the most dangerous challenges military aviators face is poor visibility, especially during operations in unprepared environments. In recent years, enhanced vision systems have been developed that improve the ability of pilots to see airport features and surrounding terrain at night and during periods of reduced visibility while flying close to the ground. Still, with constantly increasing complexity of the tasks of a flight crew, especially of a crew flying a military plane involved in war actions, there is a need to further develop existing enhanced vision systems in order to enhance flight crew situational awareness. For example, a flight crew of a military airplane carrying a weapon, such as the advanced tactical laser (ATL) or a gunship, needs to maintain common awareness for battle space while focused on individual tasks. A weapon operator may be narrowly focused on a single target while cockpit crewmembers may have broad situational awareness. If the cockpit crew becomes aware of an important new development, the cockpit crew needs to rapidly cue the weapon operator. Similarly, a battle manager may observe an emerging threat to the aircraft and need to notify the cockpit crew immediately, for example, by visually cueing them. Furthermore, it may happen that only one of the crewmembers becomes aware of a newly developing situation. In this case, this crewmember needs to be able to communicate with the other crewmembers. Presently, there is no effective solution on how to share visual information among flight crewmembers, such as a weapon operator or a battle manager, to rapidly cue other members to targets or situations observed by one crewmember.

Furthermore, a weapon operator, for example, an ATL weapon operator, needs to maintain situational awareness of multiple targets, while focused on a single target at a time. Currently, the weapon operator is required to search for each target separately. While focused on one target, other targets may move under cover, making it harder for the weapon operator to search for them. Therefore, if more than one target needs to be tracked at the same time, prior art requires one operator for each target to be tracked.

Another task that requires situational awareness of the flight crewmembers, for example, of a low flying helicopter, is obstacle detection, for example, of power lines, since striking a power line will be disastrous for any aircraft. Passive power line detection systems for aircraft have been developed, for example, U.S. Patent Application No. US2002/0153485 A1 published by Nixon et. al. This obstacle detection system determines the presence of small, curvilinear objects, such as power lines. While the detected objects will be displayed for the pilot such that evasive maneuvers can be performed by the pilot as necessary, the pilot cannot change his line of sight and look to the right or left. The pilot's line of sight needs to be where he suspects, for example, power lines.

Prior artificial vision systems typically use a single turreted sensor system that is slaved to a pilot's line of sight. As the pilot turns his head, the entire turret rotates to follow his line of sight. Consequently, all users can see imagery only in the pilot's line of sight. Thus, if the pilot is looking to one side, an operator aide, such as an obstacle detection system or a tracking system, can only view imagery in that direction. Using prior art vision systems it is not possible that the users can independently monitor views in different directions.

Currently a next generation of enhanced vision systems (EVS) is being developed, for example, the enhanced vision system described by Yelton, Bernier, and Sanders-Reed in Proc SPIE, 5424, April 2004, hereby incorporated by reference, combine imagery from multiple sensors, possibly running at different frame rates and pixel counts, onto a display. In the case of a helmet mounted display (HDM), the user line of sight is continuously changing with the result that the sensor pixels rendered on the display are changing in real time. In a prior art enhanced vision system, the various sensors provide overlapping fields of view, which requires stitching imagery together to provide a seamless mosaic to the user. Furthermore, different modality sensors may be present requiring the fusion of imagery from the sensors having a common field of view. Still further, it is possible to combine sensor imagery with synthetic imagery, such as 3D terrain from digital elevation maps, overhead satellite imagery, or flight path symbology. The output of an enhanced vision system may be presented on a head-down, head-up, or helmet mounted display. All of this takes place in a dynamic flight environment where the aircraft (with fixed mounted sensors) is changing position and orientation while the users are independently changing their lines of sight. Modern enhanced vision systems, for example, the enhanced vision system described by Yelton, Bernier, and Sanders-Reed, may provide new opportunities for visual sharing information and for using independent operator aides and intelligent agents, not available in systems pre-dating enhanced vision systems. However, current prior art enhanced vision systems are “dumb” systems in the sense that these systems provide integrated imagery to a human user who supplies all the intelligence for interpretation.

Prior art further includes, for example, an enhanced vision system called “Flying Infrared for Low-level Operations” (FLILO) disclosed by Guell in IEEE AES Systems Magazine, September 2000, pp. 31-35. FLILO enhances situational awareness for safe low level/night time and moderate weather flight operations, such as take-off, landing, taxiing, approaches, drop zone identification and short austere airfield operations. FLILO provides electronic/real time vision to the pilots through a series of imaging sensors, an image processor, and a wide field-of-view see-through helmet mounted display integrated with a head tracker. While enhancing the situational awareness of a flight crew, the FLILO enhanced vision system does not offer visual communication between flight crew members or allows the user to maintain situational awareness of multiple tasks.

As can be seen, there is a need for increasing not only the pilot's situational awareness but also the flight crew's situational awareness. Furthermore, there is a need to enable instant visual communication between a cockpit crew, a weapon operator, a battle manager, or other personnel of a military aircraft. Also, there is a need to share visual information among cockpit crewmembers. Moreover, a need exists to enable a weapon operator to maintain situational awareness of multiple targets. Still further, there is a need to add “intelligent” components to existing enhanced vision systems that may provide help to human users or that may replace human users.

There has, therefore, arisen a need to provide components that enable sharing of visual information among flight crew members and that may be added to existing enhanced vision systems. There has further arisen a need to provide components that reduce the workload of an operator, such as a weapon operator, and that may be added to prior art enhanced vision systems. There has still further arisen a need to provide “intelligent” components that may handle various flight and battle operations based on the broad area coverage provided by an enhanced vision system.

SUMMARY OF THE INVENTION

The present invention provides situational awareness components of enhanced vision systems and a method for enhancing flight crew situational awareness. The present invention further provides a platform for sharing visual information among flight crewmembers. The present invention still further provides components that combine tasks, such as automatic target tracking, obstacle detection, or blind spot monitoring, with the broad area coverage provided by an enhanced vision system. These components may be used as, but are not limited to, operator aides or intelligent agents to assist, for example, battle managers and payload managers, of military aircraft carrying an advanced tactical laser or gunship. The present invention still further provides a method for enhancing flight crew situational awareness.

In one aspect of the present invention, a virtual sphere provided by an enhanced vision system comprises synthetic imagery filling the virtual sphere and a common view window mapped to a dedicated position within the synthetic imagery. The imagery of the line of sight of a user is displayed in the common view window.

In another aspect of the present invention, a virtual sphere provided by an enhanced vision system comprises sensor synthetic imagery filling the virtual sphere, sensor imagery overlaying the synthetic imagery, a first common view window mapped to a dedicated position within the synthetic imagery, and a second common view window mapped to a dedicated position within the synthetic imagery. The imagery of the line of sight of a first user is displayed in the first common view window. The imagery of the line of sight of a second user is displayed in the second common view window.

In still another aspect of the present invention, an enhanced vision system comprises a plurality of physical sensors providing sensor imagery; synthetic imagery supplementing the sensor imagery and including a common view window mapped to a dedicated position within the synthetic imagery, a first user, and a second user. The first user is a human operator wearing a helmet mounted display. The common view window is visible in the helmet mounted display. The human operator selects the position of the common view window in the helmet mounted display. The second user is a virtual user.

In a further aspect of the present invention, an enhanced vision system comprises a plurality of physical sensors providing sensor imagery; synthetic imagery supplementing the sensor imagery, a video distribution module, a first enhanced vision system processor receiving imagery from the video distribution module, a second enhanced vision system processor receiving imagery from the video distribution module, a third enhanced vision system processor receiving imagery from the video distribution module, a first human operator connected with the first enhanced vision system processor, a second human operator connected with the second enhanced vision system processor, and a virtual user connected with the third enhanced vision system processor. The synthetic imagery includes a first common view window mapped to a dedicated position within the synthetic imagery and a second common view window mapped to a dedicated position within the synthetic imagery. The video distribution module combines the sensor imagery and the synthetic imagery. The first human operator wears a first helmet mounted display. The first common view window and the second common view window are visible in the first helmet mounted display. The second human operator wears a second helmet mounted display. The first common view window and the second common view window are visible in the second helmet mounted display. In still a further aspect of the present invention, an enhanced vision system comprises a plurality of physical sensors fixed mounted to an aircraft providing sensor imagery, synthetic imagery supplementing the sensor imagery, a video distribution module, a first enhanced vision system processor receiving imagery from the video distribution module, a second enhanced vision system processor receiving imagery from the video distribution module, a third enhanced vision system processor receiving imagery from the video distribution module, a first human operator connected with the first enhanced vision system processor, a second human operator connected with the second enhanced vision system processor, and a passive obstacle detection device connected with the third enhanced vision system processor. The synthetic imagery includes a first common view window mapped to a dedicated position within the synthetic imagery, a second common view window mapped to a dedicated position within the synthetic imagery, and a third common view window mapped to a dedicated position within the synthetic imagery. The video distribution module combines the sensor imagery and the synthetic imagery. The first human operator is on board of the aircraft. The second human operator is off board of the aircraft. The first human operator sends a line of sight pointing request to the first enhanced vision system processor. The first human operator receives imagery for the requested line of sight from the first enhanced vision system processor. The imagery is displayed in the first common view window. The second human operator sends a line of sight pointing request to the second enhanced vision system processor. The second human operator receives imagery for the requested line of sight from the second enhanced vision system processor. The imagery is displayed in the second common view window. The passive obstacle detection device sends a line of sight pointing request to the third enhanced vision system processor. The passive obstacle detection device receives imagery for the requested line of sight from the third enhanced vision system processor. The imagery is displayed in the third common view window.

In still another aspect of the present invention, a method for enhancing flight crew situational awareness comprises the steps of: equipping an aircraft with an enhanced vision system including a common view window; connecting a first human operator with the enhanced vision system; producing line of sight imagery for the first human operator with the enhanced vision system; connecting a virtual user with the enhanced vision system; producing line of sight imagery for the virtual user with the enhanced vision system; displaying the line of sight imagery of the virtual user in the common view window; alerting the first human operator; and viewing line of sight of the virtual user by the first human operator.

DETAILED DESCRIPTION OF THE INVENTION

Broadly, an embodiment of the present invention provides situational awareness components of an enhanced vision system. Contrary to the known prior art, the spherical coverage provided by an enhanced vision system as in one embodiment of the present invention includes at least one common view window that displays the line of sight of a designated flight crewmember. Furthermore, operator aides and intelligent agents as in another embodiment of the present invention can be added to existing enhanced vision systems as virtual crewmembers. Enhanced vision systems including situational awareness components as in one embodiment of the present invention may be used, for example, in military aircraft to enhance the situational awareness of the flight crew. The situational awareness components as in another embodiment of the present invention may be used as, but are not limited to, operator aides or intelligent agents to assist, for example, battle managers of a military helicopter carrying an advanced tactical laser or gunship, or payload managers of a transport aircraft. By using an enhanced vision system as in one embodiment of the present invention it may be possible that a human user, independent operator aides and intelligent agents, such as an obstacle detection system or a tracking system, can independently monitor views in different directions, which is not possible using prior art single turreted sensor systems. Furthermore, it may be possible, to use the enhanced vision system as in one embodiment of the present invention in ground vehicles, such as truck, ships, etc., both military and commercial.

In one embodiment, the present invention provides a common view window incorporated within a virtual sphere provided by a prior art enhanced vision system. The imagery displayed in this common view window may be designated to the line of sight of a designated flight crewmember. This enables all other flight crew members to follow the line of sight of the designated crewmember and adjust their own line of sight if needed. Prior art enhanced vision systems do not provide components that allow communication between flight crew members as in one embodiment of the present invention.

An embodiment of the present invention further provides a plurality of common view windows incorporated within a virtual sphere provided by a prior art enhanced vision system. Each of the common view windows could be assigned to the line of sight of one flight crewmember further enhancing the communication between flight crewmembers. Contrary to the prior art, where each flight crewmember receives the various sensor images independently and only sees images in his line of sight, it will be possible, by adding the common view windows as in one embodiment of the present invention, that each crew member also follows the line of sight of each of the other crew members. By providing a plurality of common view windows as in one embodiment of the present invention, it may further be possible that one flight crew member alerts the other flight crew members of his line of sight and that each of the other crew members may switch their line of sight directly to the alerted line of sight. Such communication between flight crewmembers is not possible using prior art enhanced vision systems.

An embodiment of the present invention further provides virtual users of an enhanced vision system, such as operator aides and intelligent agents, that could assist or even replace human operators, for example, flight crew members of a military aircraft or a transport aircraft. The imagery for the line of sight of operator aides as in one embodiment of the present invention, for example, automatic target trackers and passive obstacle detection systems, may be included as part of the synthetic vision overlay of the enhanced vision system to the human operator, such as a flight crew member. Contrary to the known prior art, operator aides, such as automatic target trackers, as in one embodiment of the present invention could enable, for example, a weapons operator to track multiple targets instead of tracking only one target. By combining, for example, automatic target trackers with wide coverage sensors available through prior art enhanced vision systems, the existing standard auto tracker technology can be extended and the flight crew situational awareness can be enhanced. By providing intelligent agents, integrated imagery provided by an enhanced vision system can be interpreted by an electronic processing unit instead of supplying all the intelligence for interpretation of the imagery by a human user as currently done.

An embodiment of the present invention further provides a method for enhancing flight crew situational awareness. By providing a platform that enables sharing of visual information obtained from a prior art enhanced vision system among flight crew members as in one embodiment of the present invention, the flight crew awareness can be improved compared to prior art methods. By adding virtual users, such as operator aides and intelligent agents, to an existing enhanced vision system, the situational awareness of each flight crewmember may be enhanced by reducing the workload of the human operator. Furthermore, the timeline of operations may be improved using the method for enhancing flight crew situational awareness as in one embodiment of the present invention compared to prior art methods that only use a prior art enhanced vision system.

Referring now toFIG. 1, a schematic view of a virtual sphere10provided by an enhanced vision system30to a user20is illustrated according to one embodiment of the present invention. The virtual sphere10may include synthetic imagery12and sensor imagery13. The synthetic imagery12may fill the entire virtual sphere10. The sensor imagery13may overlay the synthetic imagery12according to the position of fixed physical sensors31(shown inFIG. 3), for example, on an aircraft. Images provided by the physical sensors31(illustrated inFIG. 3), such as visible light cameras and infrared cameras, may be stitched together to a combined multi-spectral image viewable as sensor imagery13by the user20. The user20may be a human operator21, as shown inFIG. 1or a virtual user23, as shown inFIG. 3. The sensor imagery13may further include images from modality sensors, such as an image14from a rear view mirror sensor. The sensors31providing the sensor imagery13may be preferably fixed mounted sensors mounted to an aircraft. Consequently, the sensors31change position and orientation as the aircraft does. The synthetic imagery12may include a synthetic map15created from a digital terrain database and a common view window16. The synthetic imagery12may further include overhead satellite imagery and flight path symbology (not shown). The sensor imagery13and the synthetic imagery12may be combined by the enhanced vision system30(FIG. 3) to provide the spherical coverage10. It may also be possible to turn off the sensor imagery13, for example, in conditions of zero visibility, and to use the synthetic imagery12only. The output of the enhanced vision system30may be presented on a helmet mounted display17. The output of the enhanced vision system30may further be presented on a head-down or head-up display (not shown). The human operator21may wear a helmet mounted display17and may change his line of sight by turning his head211. More than one human operator21may be connected to the enhanced vision system30. Each human operator21may wear a helmet mounted display17and may change his line of sight independently from other human operators21. The common view window16may be visible in the helmet mounted display17. The human operator21may be a member of a flight crew, such as a pilot, a cockpit crewmember, a weapon operator, a battle manager, or a payload manager on board of an aircraft. The human operator21may further be a remote operator on the ground for an unmanned aircraft, a human operator (off board)22.

As shown inFIG. 1, the common view window16may be mapped to a dedicated position on the virtual sphere10within the area of synthetic imagery12. Alternatively, the human operator21may be able to select the position of the common view window16in his helmet mounted display17as desired. Furthermore, the common view window16may be programmed to follow the line of sight of the human operator21. Programming the common view window16to follow the line of sight of a human operator21may enable the human operator21to drag the common view window16around the virtual sphere10while he is trying to find the same imagery in his helmet mounted display17as displayed in the common view window16. The imagery displayed in the common view window16may be assigned to the line of sight of a human operator21, such as a flight crewmember, that takes control of the common view window16. Consequently, all human operators21connected to the enhanced vision system30, for example, all flight crewmembers may view the line of sight of the human operator21that took control of the common view window16. If needed, a different human operator21may take control of the common view window16to alert the other users20. The common view window16may then display the line of sight of the human operator21that took control of the common view window16visible to all users20of the enhanced vision system30. For example, the co-pilot of an aircraft may take control of the common view window16. Wherever the co-pilot may look, his line of sight will be displayed in the common view window16visible for all other flight crewmembers in their helmet mounted displays17. If one of the other flight crewmembers sees a imagery12or13in his helmet mounted display17that he wants to share, this flight crewmember takes control of the common view window16and instantly, this information becomes visible in the common view window16that may be visible in the helmet mounted display17of each crewmember. The user20that takes control of the common view window16may further be a remote operator22(as shown inFIG. 3). The user20that takes control of the common view window16may further be a virtual user23(as shown inFIG. 3). It may further be desirable to add some information to the imagery displayed in the common view window16, for example, information identifying the human operator that has control of the common view window16. Enabling the human operator21who has control of the common view window16to draw overlays on the imagery displayed in the common view window15may further enhance the graphic communication between human operators21.

Referring now toFIG. 2, a schematic view of a virtual sphere10provided by an enhanced vision system30to a user20is illustrated according to another embodiment of the present invention. The virtual sphere10may include sensor imagery13and synthetic imagery12. The sensor imagery13may contain a combined multi-spectral image. The synthetic imagery12may include a synthetic map15, a common view window161, a common view window162, a common view window163, and a common view window164. The common view window161, the common view window162, the common view window163, and the common view window164may be visible in the helmet mounted display17of a first human operator21(on board), in the helmet mounted display17of a second human operator21(on board), and in the helmet mounted display17of a third human operator22(off board). The common view window161may display the imagery of the line of sight of a first human operator21(on board). The common view window162may display the imagery of the line of sight of the second human operator21(on board). The common view window163may display the imagery of the line of sight of the third human operator22(off board). The common view window164may display the imagery of the line of sight of a user20. The first human operator and the second human operator may be on board, for example, of an aircraft and, therefore, at the same location as the enhanced vision system30. The third human operator22may be a remote operator that is off board, for example, of an aircraft and, therefore, at a different location than the enhanced vision system30. The user20may be an additional human operator21(on board), an additional human operator22(off board), or a virtual user23(FIG. 3). The virtual user23may be an operator aide or an intelligent agent. Additional common view windows16may be mapped on the virtual sphere10besides the common view windows161,162,163, and164(shown inFIG. 2). The imagery displayed in the common view windows161,162,163, and164may be assigned to a specific user20as described above. It may further be possible that any human operator21connected to the enhanced vision system30may take control of the common view window161as needed.

Referring now toFIG. 3, a block diagram of an enhanced vision system30is illustrated according to one embodiment of the present invention. The enhanced vision system30may include a plurality of physical sensors31, synthetic imagery12, a video distribution module32, a plurality of enhanced vision system processors33, and a plurality of users20. The sensors31may provide images that may be stitched together to create the sensor imagery13(FIG. 1). The synthetic imagery12may supplement the sensor imagery13. The video distribution module32may combine the sensor imagery13and the synthetic imagery12to form the virtual sphere10(FIG. 1). The video distribution module32may distribute the combined imagery data to the enhanced vision system processors33. Each enhanced vision system processor33may receive a line of sight pointing request35from a user20. Corresponding to the line of sight pointing request35from the user20, the enhanced vision system processor33may provide imagery36for the requested line of sight to the user20. The quantity of enhanced vision system processors33connected with the video distribution module32may depend on the quantity of users20and may further be equal to the quantity of users20. The user20may be a human operator21that is on board, a human operator22that is off board, or a virtual user23. The virtual user23, such as an operator aide or an intelligent agent, may include a processor34. The human operator may be a flight crewmember of an aircraft, such as a pilot, a co-pilot, a weapons operator, a battle manager, or a payload manager. In the case that the user20is a human operator21(on board) or a human operator22(off board), the human operator21or22may wear a helmet mounted display17and the human operator21or22may send a line of sight pointing request35to the enhanced vision system processor33by turning their head211(FIG. 1). The enhanced vision processor33may then send imagery36for the requested line of sight to the human operator21or22. The received imagery36may be displayed on the helmet mounted display17of the human operator21or22. The human operator21or22may then provide the analysis, processing, and interpretation of the received imagery36. In the case that the user20is a virtual user23, the processor34may send a line of sight pointing request35to the enhanced vision system processor33. The enhanced vision processor33may then send imagery36for the requested line of sight to the processor34of the electronic processing unit23. The processor34may provide the processing of the imagery36received from the enhanced vision system processor33. Further, the imagery36received by the processor34may be displayed in the helmet mounted display17of the human operator21or22, for example, visible in the common view window164(as shown inFIG. 2).

The processor34may be an operator aide processor. Operator aides may be, for example, a passive obstacle detection device. The processor34may have a forward facing line of sight, may send a pointing request35for this forward facing line of sight, and may always receive imagery36provided by the forward facing sensors regardless of the line of sight of the human operator21. The imagery36received by the processor34may be displayed in the helmet mounted display17of the human operator21, for example, in the common view window164(as shown inFIG. 2). The processor34may further sound an alarm if an obstacle is detected in the flight path, the human operator21may change his line of sight immediately to the line of sight of the processor34and take action as needed. Furthermore, a transport aircraft equipped with the enhanced vision system30may have rear looking sensors31that may, for example, monitor cargo deployment. The processor34of the virtual user23may receive imagery36provided by these rear looking sensors31. The imagery36received by the processor34may be displayed in the helmet mounted display17of the human operator21, for example, in the common view window164(as shown inFIG. 2). The processor34may visually alert the human operator21using, for example, the common view window164. The processor34may further sound an alarm if a problem is detected in the cargo area, the human operator21may change his line of sight immediately to the line of sight of the processor34and take action as needed.

The processor24, as shown inFIG. 3, may further be a processor of an intelligent agent. One user20of the enhanced vision system30may be a tracking system. The tracking system may be a virtual user23that may include a processor34for each tracker. The tracking system may be able to track an object as the object moves relative to the aircraft. The processor34of a tracker may send a pointing request35for its current line of sight to a target. This will generate output imagery36centered on the target, which the processor34of the tracker uses to update its line of sight. As the target moves, the tracker may follow the target within the sensor (31) boundaries. The imagery36of the line of sight received by the processor34of the tracker and, therefore, the position of the moving target may be displayed on the helmet mounted display17of the human operator21, for example, in the common view window164(FIG. 2). Such tracking system may be used, for example, by a weapon operator. The weapons operator may also be a human operator21as illustrated inFIGS. 1,2, and3. The weapon operator21may, for example, identify three targets. The weapon operator21may initiate an automatic tracker on each of the three targets. While the weapon operator21is dealing with one target, the trackers may keep track of the other two targets. When the weapon operator21is ready to deal with one of these targets, he may quickly reacquire the target even if the target is hidden from the weapon operator's21view. Consequently, the currently exiting auto tracker technology may be extended by using wide coverage sensors31of the enhanced vision system30. Virtual users23may further include adaptive cruise control, station keeping, and blind spot monitoring.

Referring now toFIG. 4, a flow chart of a method40for enhancing flight crew situational awareness is illustrated according to another embodiment of the present invention. The method40for enhancing flight drew situational awareness may include the steps of: providing an aircraft equipped with an enhanced vision system30as illustrated inFIG. 3(step41), connecting a first human operator21with the enhanced vision system30by wearing a first helmet mounted display17(step42) (as shown inFIG. 1), connecting a second human operator21with the enhanced vision system30by wearing a second helmet mounted display17(step43) (as shown inFIG. 1). The method40may further include the steps of: sending a line of sight pointing request35from first human operator21to the enhanced vision system processor33by turning head211and receiving imagery36displayed in the first helmet mounted display17(step44) (as shown inFIG. 3) and sending a line of sight pointing request35from second human operator21to the enhanced vision system processor33by turning head211and receiving imagery36displayed in the second helmet mounted display17(step45)(as shown inFIG. 3). The first human operator21may then take control of common view window16illustrated inFIG. 1alerting the second human operator21(step46). The imagery36for the line of sight of the line of sight of the first human operator21may be displayed in the common view window16visible for the second human operator21(step47). Seeing a new event occurring in his line of sight, the second human operator21may take control of common view window16illustrated inFIG. 1alerting the first human operator21(step48). The imagery36(as shown inFIG. 3) for the line of sight of the second human operator21will now be displayed in the common view window16visible for the first human operator21(step49).

The method40may further include the steps of: connecting a virtual user23to the enhanced vision system30(step51), sending a line of sight pointing request35from processor34of the virtual user23to the enhanced vision system processor33, and receiving imagery36(step52) as illustrated inFIG. 3. The virtual user23may take control of the common view window16alerting the first human operator and the second human operator21(step53). The imagery36for the line of sight of the line of sight of virtual user23may be displayed in the common view window16visible for the first human operator21and second human operator21(step54). By providing the common view window16in the synthetic imagery12on the virtual sphere10, for example, as in step41, visual communication between the first human operator21and the second human operator21may be possible. By connecting a virtual user23to the enhanced vision system30as in step51and by displaying the imagery36for the line of sight of the virtual user23in the common view window16(as in step53), the workload of the human operator21may be reduced and the time line of actions may be shortened. By providing more than one common view window16(as illustrated inFIG. 2), the flight crew situational awareness may be further enhanced. Even though the enhanced vision system30as in one embodiment of the present invention has been described to be used by pilots of an military aircraft, it may be possible to use the enhanced vision system30in commercial aircrafts and, furthermore, in ground vehicles, such as truck, ships, etc., both military and commercial.