Patent Description:
For example, this kind of image generating device is disclosed in Patent Document <NUM>. The image generating device of Patent Document <NUM> includes a camera, an imaging data receiving part, a position acquiring part, an image generating part, a target object display item generating part, and a display unit, and is configured to perform geo-referencing of the image stream from the camera.

In Patent Document <NUM>, the imaging data receiving part receives the stream image data captured by the camera of a ship. The position acquiring part acquires the position of a target object around the ship (another ship etc.). The image generating part generates an image to be displayed on the display unit based on the image of the stream image data. The image supports the position and the view of the camera. The target object display item generating part generates a target object display item indicative of the target object at a point on the image. The display is configured to display the image, and the target object display item placed at the point on the image.

The image generating part generates the image by projecting the image on a three-dimensional virtual space in which points correspond to the points on the image, and the environment around the water-surface movable body is indicated. Further, the target object display item generating part generates the target object display item based on positional information on the target object, and the points of the three-dimensional virtual space.

However, in the configuration of Patent Document <NUM>, a time interval at which the position acquiring part acquires the position of a target object, such as another ship which moves around the ship may became longer. In that case, the frequency at which the indication of the target object display item is updated decreases, and the motion of the target object display item in the image may be displayed unnaturally without smoothness.

From <CIT>, a wearable display device is known. In one embodiment, a display view of a maritime scenery for navigating on a river, bay or ocean with an approaching ship is shown.

The paper by <NPL>, deals with providing augmented reality in a maritime situation, for informing a person responsible for navigating a vessel about safety-relevant environmental features.

The doctoral thesis by <NPL>, teaches as well to use augmented reality for maritime navigation purposes.

The paper by <NPL> discloses to equip sailboats with two cameras and with sensors, displaying information obtained by the sensors in an image that is based on a camera image.

The paper (of which the author is not known) having the title; "Automatic identification system", <NUM> may <NUM>, that was retrievable under URL:https://en. org/w/index. php?title=Automatic identifcation system&old id=<NUM>, is related to a system for avoiding collision of ships.

The present disclosure is made in view of the above situation, and one purpose thereof is to realize a virtual reality image with a smooth motion.

The problem to be solved by the present invention is as described above, and means to solve the problem and the effect thereof is defined by the independent claims and furthermore described below.

According to one aspect of the present invention, an image generating device with the following configuration is provided. That is, the image generating device includes a captured image inputting module, a position acquiring module, a posture acquiring module, an additional display information acquiring module, a synthesized image generating module, a target object position estimating module, and an image updating module. The captured image inputting module acquires an image captured by an imaging device mounted on a water-surface movable body. The position acquiring module acquires positional information indicative of a position of the water-surface movable body. The posture acquiring module acquires posture information indicative of a posture of the water-surface movable body. The additional display information acquiring module non-continuously acquires additional display information including information indicative of a position of a target object. The synthesized image generating module generates a synthesized image every time additional display information is acquired, where a graphic rendering a three-dimensional virtual reality object indicative of the additional display information is synthesized with the captured image based on measured position and imaging direction of the imaging device relative to the water-surface movable body, the positional information, the posture information, and the additional display information. The target object position estimating module estimates a position of the target object at a timepoint between acquisition of two consecutive pieces of additional display information, based on a speed of the target object included in the additional display information. The image updating module causes the synthesized image generating module to update the rendering of the virtual reality object of the target object in the synthesized image such that the three-dimensional virtual reality object is rearranged in the synthesized image based on a result of the estimation by the target object position estimating module at said timepoint between acquisition of two consecutive pieces of additional display information.

According to this configuration, even when a time interval of acquisition of the additional display information indicative of the position of the target object is long, the graphic rendering the virtual reality object can be updated by estimating the position of the target object at the suitable timing. Therefore, it may become possible to realize the virtual reality image with a smooth motion.

The image generating device may have the following configuration. That is, the image generating device may include a target object direction estimating module. The target object direction estimating module may estimate a direction of the target object at the timing later than the timing at which the additional display information is acquired the latest, based on an angular velocity of the direction of the target object included in the additional display information or an angular velocity of the direction of the target object estimated from the additional display information. The image updating module may cause the synthesized image generating module to update the rendering of the virtual reality object of the target object in the synthesized image based on a result of the estimation by the target object direction estimating module.

According to this configuration, the graphic rendering the virtual reality object can be updated by estimating the direction of the target object at the suitable timing similarly to the position. Therefore, a more natural motion of the image can be realized.

In the image generating device, the synthesized image generating module may synthesize the graphic rendering the virtual reality object having the shape of a ship with the captured image.

According to this configuration, the movement of a surrounding ship can be expressed by the smooth motion of the three-dimensional computer graphic of a ship.

In the image generating device, the additional display information may include AIS information received by an AIS receiver.

According to this configuration, even when the target object is based on the additional display information received by the AIS receiver of which the updating frequency tends to be less in many cases, the virtual reality image with the smooth motion can be realized.

In the image generating device, the additional display information may include information on the position and the speed of the target object acquired based on successive radar images from a radar device.

According to this configuration, even when the target object (another ship) is based on the additional display information from the radar device which cannot update information such as the position of the target object, until a radar antenna revolves one time, the virtual reality display with the smooth motion can be realized.

Next, one embodiment of the present disclosure is described with reference to the drawings. <FIG> is a block diagram illustrating the entire configuration of an image generating device <NUM> according to one embodiment of the present disclosure. <FIG> is a side view illustrating various kinds of instruments mounted on a ship <NUM>.

The image generating device <NUM> illustrated in <FIG> may be a device which is mounted on the ship (water-surface movable body) <NUM> as illustrated, for example, in <FIG>, and may generate an image expressing the situation around the ship <NUM> in virtual reality based on the image captured by a camera (imaging device) <NUM>. The image generated by the image generating device <NUM> may be displayed on a display unit <NUM>.

The display unit <NUM> may be configured as, for example, a display unit for a navigation assisting device to which a ship operator who operates the ship <NUM> refers. However, the display unit <NUM> is not limited to the above configuration, and, for example, it may be a display unit for a portable computer which is carried by a ship operator's assistant who monitors the surrounding situation from the ship <NUM>, a display unit for a passenger to watch in the cabin of the ship <NUM>, or a display part for a head mounted display, such as a wearable glass, worn by a passenger.

The image generating device <NUM> may generate a synthesized image which is an output image to the display unit <NUM> by synthesizing the image around the ship <NUM> captured by the camera <NUM> installed in the ship <NUM>, and graphics which expresses additional display information on the perimeter of the ship <NUM> in virtual reality (as will be described in detail later).

Next, mainly referring to <FIG>, the camera <NUM> and various kinds of ship instruments which may be electrically connected to the image generating device <NUM> are described.

The camera <NUM> may be configured as a wide-angle video camera which images the perimeter of the ship <NUM>. This camera <NUM> may have a live output function, capable of generating video data (image data) as the imaged result on real time, and outputting it to the image generating device <NUM>. As illustrated in <FIG>, the camera <NUM> may be installed in the ship <NUM> so that an imaging direction becomes horizontally forward of the hull.

The camera <NUM> may be attached to the ship <NUM> through a rotating mechanism (not illustrated) and, therefore, the imaging direction can be changed in a given angle range on the basis of the hull of the ship <NUM> by inputting a signal for instructing pan/tilt from the image generating device <NUM>. Moreover, since the height and the posture of the ship <NUM> variously change due to waves etc., the height of the camera <NUM> may change in connection with the change, and the posture (imaging direction) may also change three-dimensionally.

The image generating device <NUM> of this embodiment may be electrically connected to, in addition to the camera <NUM>, a GNSS compass (a direction sensor, a posture sensor) <NUM>, an angular velocity sensor <NUM>, a GNSS receiver <NUM>, an acceleration sensor <NUM>, an AIS receiver <NUM>, an ECDIS <NUM>, a plotter <NUM>, a radar device <NUM>, and a sonar <NUM>, etc. as the ship instrument.

The GNSS compass <NUM> may be provided with a plurality of GNSS antennas (positioning antenna) fixed to the ship <NUM>. The GNSS compass <NUM> may calculate spatial relationships of the GNSS antennas based on radio waves received from positioning satellites. Particularly, the GNSS compass <NUM> of this embodiment may be configured to obtain the spatial relationships of the GNSS antennas based on phase differences of carrier phases of the radio waves received by the respective GNSS antennas (since this processing is known, detailed description thereof is omitted). Therefore, a bow direction of the ship <NUM> can be acquired with sufficient accuracy.

The GNSS compass <NUM> may acquire the posture of the ship <NUM> three-dimensionally. In other words, the GNSS compass <NUM> may detect a roll angle and a pitch angle of the ship <NUM>, as well as the bow direction (i.e., yaw angle of the ship <NUM>). The posture information on the ship <NUM> acquired by the GNSS compass <NUM> may be outputted to a posture acquiring module <NUM> of the image generating device <NUM>, and other ship instruments utilizing the posture information.

The angular velocity sensor <NUM> may be, for example, comprised of a known oscillating gyroscope sensor, which is capable of detecting a yaw angular velocity, a roll angular velocity, and a pitch angular velocity of the ship <NUM> at a cycle shorter than a posture detection interval of the GNSS compass <NUM> (e.g., <NUM> second). By using both an angle detected by the GNSS compass <NUM> and an integrated value of the angular velocity detected by the angular velocity sensor <NUM>, the posture of the ship <NUM> can be acquired at a time interval shorter than the case where only the GNSS compass <NUM> is used. Moreover, the angular velocity sensor <NUM> may function as an alternative for acquiring the posture information, when the radio wave from the positioning satellite is interrupted with an obstacle, such as a bridge, and the posture becomes undetectable by the GNSS compass <NUM>.

The GNSS receiver <NUM> may find the position of the ship <NUM> (in detail, a latitude, a longitude, and a height of the GNSS antenna) based on the radio waves which the GNSS antennas received from the positioning satellites. The GNSS receiver <NUM> may output the acquired positional information to a position acquiring module <NUM> of the image generating device <NUM>, and/or other ship instruments utilizing the positional information.

The acceleration sensor <NUM> may be configured, for example, as a known capacitance detection type sensor, which is capable of detecting accelerations about the yaw axis, the roll axis, and the pitch axis of the ship <NUM> at a cycle shorter than a position detection interval of the GNSS receiver <NUM> (e.g., <NUM> second). By using both the position detected by the GNSS receiver <NUM>, and a double integrated value of the acceleration detected by the acceleration sensor <NUM>, the position of the ship <NUM> can be acquired at a time interval shorter than the case where only the GNSS receiver <NUM> is used. Moreover, the acceleration sensor <NUM> may function as an alternative for acquiring the positional information, when the radio wave from the positioning satellite is interrupted, and the detection of the position is impossible by the GNSS receiver <NUM>.

In this embodiment, as illustrated in <FIG>, the angular velocity sensor <NUM>, the GNSS receiver <NUM>, and the acceleration sensor <NUM> may be built in the GNSS compass <NUM>. However, some or all of the angular velocity sensor <NUM>, the GNSS receiver <NUM>, and the acceleration sensor <NUM> may be provided independently from the GNSS compass <NUM>.

The AIS receiver <NUM> may receive AIS information transmitted from another ship, a land station, etc. The AIS information may include various information, such as a position (latitude and longitude) of another ship which travels around the ship <NUM>, a length and a width of another ship, and a type and identifying information of another ship, a ship speed, a course, a destination of another ship, and a position and identifying information of a landmark.

The ECDIS <NUM> may acquire the positional information on the ship <NUM> from the GNSS receiver <NUM> and output the information on the perimeter of the ship <NUM> to the image generating device <NUM> based on electronic nautical chart information stored beforehand.

The plotter <NUM> may generate information on a traveling trail of the ship <NUM> by continuously acquiring the position of the ship <NUM> from the GNSS receiver <NUM>. Moreover, by allowing a user to set a plurality of waypoints (points through which the ship <NUM> is scheduled to pass), the plotter <NUM> can generate a scheduled route by sequentially connecting these waypoints.

The radar device <NUM> may detect a target object, such as another ship, which exists around the ship <NUM>. Moreover, the radar device <NUM> may have a known target tracking function (TT) which can acquire and track the target object, and may find a position and a velocity vector (TT information) of the target object.

The sonar <NUM> may detect a school of fish etc. by transmitting an ultrasonic wave underwater and receiving a reflective wave which is the ultrasonic wave reflected on the school of fish etc..

The image generating device <NUM> may be connected to a keyboard <NUM> and a mouse <NUM> which the user operates. The user can perform various kinds of instructions about generation of an image by operating the keyboard <NUM> and the mouse <NUM>. The instructions may include the pan/tilt operation of the camera <NUM>, setting of displaying or not-displaying of various information, and a setup of a viewpoint of a synthesized image.

Next, a configuration of the images generating device <NUM> is described in detail mainly referring to <FIG>.

As illustrated in <FIG>, the image generating device <NUM> may include a captured image inputting module <NUM>, the position acquiring module <NUM>, the posture acquiring module <NUM>, an additional display information acquiring module <NUM>, a memory <NUM>, an imaging position setting module <NUM>, an antenna position setting module <NUM>, a viewpoint setting module <NUM>, a display setting module <NUM>, a synthesized image generating module <NUM>, a target object position estimating module <NUM>, a target object direction estimating module <NUM>, and an image updating module <NUM>.

In detail, although the image generating device <NUM> is configured as a known computer and is not illustrated, it may include a CPU, a ROM, a RAM, and a HDD. Further, the image generating device <NUM> may be provided with a GPU for performing three-dimensional image processing (described later) at high speed. The HDD stores, for example, software for performing image synthesizing method of the present disclosure. By collaboration of the hardware and the software, the image generating device <NUM> can be functioned as the captured image inputting module <NUM>, the position acquiring module <NUM>, the posture acquiring module <NUM>, the additional display information acquiring module <NUM>, the memory <NUM>, the imaging position setting module <NUM>, the antenna position setting module <NUM>, the viewpoint setting module <NUM>, the display setting module <NUM>, the synthesized image generating module <NUM>, the target object position estimating module <NUM>, the target object direction estimating module <NUM>, the image updating module <NUM> etc..

The captured image inputting module <NUM> may accept an input of image data outputted from the camera <NUM>, for example, at <NUM> frames per second. The captured image inputting module <NUM> may output the inputted image data to the synthesized image generating module <NUM> (a data synthesizing module <NUM> described later).

The position acquiring module <NUM> may acquire the current position of the ship <NUM> on real time based on the detection results of the GNSS receiver <NUM> and the acceleration sensor <NUM>.

The posture acquiring module <NUM> may acquire the current posture of the ship <NUM> on real time based on the detection results of the GNSS compass <NUM> and the angular velocity sensor <NUM>.

The additional display information acquiring module <NUM> may acquire information to be displayed in addition to the image captured by the camera <NUM> (additional display information) based on information outputted to the image generating device <NUM> from the AIS receiver <NUM>, the ECDIS <NUM>, the plotter <NUM>, the radar device <NUM>, the sonar <NUM>, etc. Although various information can be considered as the additional display information, the information may be, for example, about another ship <NUM> received by the AIS receiver <NUM> as illustrated in <FIG>. Note that, the details of the additional display information will be described later.

The memory <NUM> of <FIG> may be configured as a memory which stores various kinds of information. The memory <NUM> may store three-dimensional shapes of virtual reality objects expressing various kinds of the additional display information as templates. Although the templates of the three-dimensional shapes stored in the memory <NUM> may be, for example, a small ship or boat, a large-sized ship or vessel, a buoy, a lighthouse, etc., they are not limited to these examples.

The imaging position setting module <NUM> may set the position of the camera <NUM> in the ship <NUM> (imaging position), specifically, the position of the camera <NUM> in the longitudinal direction and the width direction of the ship, and the position of the camera in the vertical direction (the height of the camera <NUM>). Although the height of the camera <NUM> may be a height from a waterline normally assumed in the ship <NUM>, it is not limited to this height and it may also be, for example, a height from the ship's bottom. This setup of the imaging position may be performed, for example, by the user operating the keyboard <NUM>, the mouse <NUM>, etc. to input the result of an actual measurement of the position of the camera <NUM>.

The antenna position setting module <NUM> may set the position of the GNSS antenna in the ship <NUM> (antenna position). This antenna position may be, for example, positions in the longitudinal direction, the width direction, and the vertical direction of the ship with respect to a reference point 4a set in the ship <NUM> as a reference of control as illustrated in <FIG>. Although this reference point 4a may be defined variously, it may be defined at a position at the center of the hull of the ship <NUM> and at the same height as the waterline normally assumed, in this embodiment. The setup of the antenna position may be performed, for example, by inputting an actual measurement value, similar to the imaging position described above.

The viewpoint setting module <NUM> of <FIG> may set the viewpoint of the image generated by the synthesized image generating module <NUM> (described later), for example, by the user operating the keyboard <NUM> and the mouse <NUM>.

The display setting module <NUM> may set, for example, displaying or not-displaying of the additional display information in the image generated by the synthesized image generating module <NUM> (described later). The settings to the display setting module <NUM> may be performed by the user operating the keyboard <NUM> and the mouse <NUM>.

The position acquiring module <NUM>, the posture acquiring module <NUM>, the additional display information acquiring module <NUM>, the memory <NUM>, the imaging position setting module <NUM>, the antenna position setting module <NUM>, the viewpoint setting module <NUM>, and the display setting module <NUM> may acquire, store or output the setting information to the synthesized image generating module <NUM>.

The synthesized image generating module <NUM> may generate an image expressing extended reality by synthesizing three-dimensional computer graphics with the captured image of the camera <NUM> inputted into the captured image inputting module <NUM>. This synthesized image generating module <NUM> may include a three-dimensional scene generating module <NUM> and the data synthesizing module <NUM>.

As illustrated in <FIG>, the three-dimensional scene generating module <NUM> may establish a three-dimensional scene of virtual reality by placing virtual reality objects 41v, 42v,. corresponding to the additional display information in a three-dimensional virtual space <NUM>. Therefore, a three-dimensional scene data (three-dimensional display data) <NUM> which is data of the three-dimensional scene may be generated. Note that the details of the three-dimensional scene will be described later.

The data synthesizing module <NUM> of <FIG> may generate a graphic which three-dimensionally expresses the additional display information by rendering the three-dimensional scene data <NUM> generated by the three-dimensional scene generating module <NUM>. The data synthesizing module <NUM> may also output a synthesized image as illustrated in <FIG>, i.e., the image synthesized the graphics 41f, 42f,. with the captured image of the camera <NUM>. As illustrated in <FIG>, in this synthesized image, the graphics 41f, 42f,. indicative of the additional display information may be placed on a sea surface image captured by the camera <NUM> (illustrated in this figure by broken lines for convenience of description) in a superimposed fashion. The data synthesizing module <NUM> may output the generated synthesized image to the display unit <NUM>. Note that the details of generation of the graphics and the data synthesizing will be described later.

The target object position estimating module <NUM> of <FIG> may estimate the current position of a moving target object (e.g., another ship <NUM> of <FIG>) among suitable target objects for displaying the additional display information, based on the latest-acquired additional display information. Moreover, the target object direction estimating module <NUM> may estimate the present direction of this moving target object based on the latest-acquired additional display information. The estimated results by the target object position estimating module <NUM> and the target object direction estimating module <NUM> may be used as the position and the direction of the virtual reality object 44v being placed in the three-dimensional virtual space <NUM> by the three-dimensional scene generating module <NUM> as illustrated in <FIG>. Note that, the details of the estimations will be described later.

The image updating module <NUM> may input information for updating the position and the direction of the target object in the three-dimensional scene (hereinafter, may be referred to as "updating information") into the synthesized image generating module <NUM> based on the estimated results by the target object position estimating module <NUM> and the target object direction estimating module <NUM>. The three-dimensional scene generating module <NUM> of the synthesized image generating module <NUM> may update the three-dimensional scene based on the inputted updating information. The data synthesizing module <NUM> of the synthesized image generating module <NUM> may update the synthesized image based on the updated three-dimensional scene.

Next, the additional display information described above acquired by the additional display information acquiring module <NUM> is described in detail. <FIG> is a conceptual diagram illustrating one example of the additional display information to be displayed in the image generating device <NUM>.

The additional display information may be information displayed in addition to the image captured by the camera <NUM>, and may be various items according to the purpose and the function of the ship instruments connected to the image generating device <NUM>. For example, as for the AIS receiver <NUM>, the received AIS information (e.g., the position and the direction of another ship, the position of a buoy, and the position of a virtual buoy) may be the additional display information. As for the ECDIS <NUM>, the additional display information may be a position of a dangerous water area, a traveling prohibited area, a lighthouse, or a buoy contained in the electronic nautical chart. As for the plotter <NUM>, the additional display information may be a position of a recorded trace of the ship <NUM>, a scheduled route setting, a waypoint, an arrival area, and a stopping area. As for the radar device <NUM>, the additional display information may be a position, a speed, etc. of a detected target object. As for the sonar <NUM>, a position of a detected school of fish may be the additional display information. These information may be inputted on real time into the image generating device <NUM> from the ship instruments. The additional display information acquiring module <NUM> may assign identifying information (e.g., an identification number) for uniquely identifying and managing each of the inputted additional display information.

In <FIG>, one example of the additional display information located around the ship <NUM> is illustrated. In <FIG>, on the sea surface (on the water surface), waypoints <NUM> indicative of the destination, and a route line <NUM> in the shape of a polygonal line indicative of the scheduled route to the destination may be defined. Moreover, near the route line <NUM>, a polygon-shaped (rectangular shape) stopping area <NUM> may be defined. The waypoint <NUM>, the route line <NUM>, and the stopping area <NUM> may be set by the user suitably operating the plotter <NUM> beforehand to specify the position of each location.

Moreover, in the example of <FIG>, it may be detected using the AIS information acquired by the AIS receiver <NUM> that another ship <NUM> is traveling to the right of the ship <NUM> at a location a little distant forward from the ship <NUM>, and a virtual buoy <NUM> is located near, forward and left side of the ship <NUM>. Note that, although the virtual buoy is not actually provided on the sea due to situations, such as a difficulty of installation, it may mean an imaginary buoy (intangible) displayed as a label or marker in the screen of a navigation device.

Each additional display information may include information at least indicative of the position (latitude and longitude) of one or more locations on the sea surface (water surface) at which it is placed. For example, the additional display information indicative of the route line <NUM> includes information on positions of two locations used as bends of the polygonal line (the positions of the locations of the bends correspond to the positions of the waypoint <NUM>). The additional display information on the stopping area <NUM> may include information at the position of each location used as a vertex of the polygon. Moreover, the additional display information indicative of another ship <NUM> may include information indicative of the position, the bow direction, the speed, the turning angular velocity, the length and the width of another ship <NUM>.

Next, the establishment of the three-dimensional scene by the three-dimensional scene generating module <NUM>, and the synthesizing of the images by the data synthesizing module <NUM> are described in detail referring to <FIG> is a conceptual diagram illustrating the three-dimensional scene data <NUM> which is generated by placing the virtual reality objects 41v, 42v,. on the three-dimensional virtual space <NUM>, and a projection screen <NUM> placed on the three-dimensional virtual space <NUM>.

As illustrated in <FIG>, the three-dimensional virtual space <NUM> where virtual reality objects 41v, 42v,. are placed by the three-dimensional scene generating module <NUM> may be configured in a rectangular coordinate system which uses a suitable reference position of the ship <NUM> (e.g., the reference point 4a described above) as the origin, where an xz plane which is a horizontal plane imitates the sea surface (water surface). In the example of <FIG>, the axis of coordinates may be set such that the +z direction is always in agreement with the bow direction, the +x direction is in agreement with the rightward, and the +y direction is in agreement with upward. Each location (coordinates) in the three-dimensional virtual space <NUM> may be set so as to correspond to the actual position around the ship <NUM>.

In <FIG>, one example where the virtual reality objects 41v, 42v, 43v, 44v, and 45v are placed in the three-dimensional virtual space <NUM>, in order to express the situation around the ship <NUM> illustrated in <FIG>, is illustrated. Each of the virtual reality objects 41v, 42v, 43v, 44v, and 45v may be placed so that it contacts the xz plane to reflect a relative position of the additional display information indicated by the virtual reality object with respect to the ship <NUM> on the basis of the bow direction of the ship <NUM>. Upon determining the positions where these virtual reality objects 41v, 42v,. are placed, a calculation using the positions of the GNSS antennas set by the antenna position setting module <NUM> illustrated in <FIG> may be performed.

The virtual reality object 44v indicative of another ship <NUM> may have the shape of a ship, and may be expressed using a template of the model of a large-sized ship stored beforehand in the memory <NUM>. Moreover, the direction of the model may be oriented so as to indicate the direction of another ship <NUM> acquired from the AIS information.

The virtual reality object 45v indicative of the virtual buoy <NUM> may be expressed using a template of the model of a buoy stored beforehand in the memory <NUM>, similar to the virtual reality object 44v of another ship <NUM>.

The virtual reality object 41v of the waypoint <NUM> may be expressed three-dimensionally in the shape of a thin disk. The virtual reality object 42v of the route line <NUM> may be expressed three-dimensionally in the shape of a polygonal line comprised of strip plates having a certain thickness and a certain width. The virtual reality object 43v of the stopping area <NUM> may be expressed three-dimensionally in the shape of a plate having a certain thickness and having a contour of the stopping area <NUM>. For these virtual reality objects 41v, 42v, and 43v, the three-dimensional shapes may be created each time, without using the templates of the models.

The three-dimensional scene generating module <NUM> may generate the three-dimensional scene data <NUM> as described above. In the example of <FIG>, since the virtual reality objects 41v, 42v,. are placed on the basis of azimuth from the position of the ship <NUM> as the origin, when the position of the ship <NUM> (positions in the east-and-west direction and the north-and-south direction) changes from the state of <FIG>, or when the bow direction changes due to a turn etc., a new three-dimensional scene where the virtual reality objects 41v, 42v,. are rearranged may be established, and the three-dimensional scene data <NUM> may then be updated. Moreover, for example, when the contents of the additional display information are changed due to traveling of another ship <NUM> from the state of <FIG>, the three-dimensional scene data <NUM> may be updated so that the latest additional display information is reflected.

Then, the data synthesizing module <NUM> may place in the three-dimensional virtual space <NUM> the projection screen <NUM> which defines the position and the area where the captured image of the camera <NUM> is projected. The synthesizing of the images can be realized by setting the position and the direction of a viewpoint camera <NUM> described below so that both the projection screen <NUM> and the virtual reality objects 41v, 42v,. are contained in the view of the camera <NUM>.

The data synthesizing module <NUM> may simulate in the three-dimensional virtual space <NUM> the position and the direction of the camera <NUM> mounted on the ship <NUM>, and place the projection screen <NUM> so as to directly oppose to the camera <NUM>. For the simulation of the position of the camera <NUM>, the position of the camera <NUM> on the basis of the hull can be obtained based on the setting value of the imaging position setting module <NUM> illustrated in <FIG>.

Upon the simulation of the position and the direction of the camera <NUM>, the change in the direction of the camera <NUM> by the pan/tilt operation described above may be taken into consideration. Further, the simulation may be performed so that the change in the position and the direction of the camera <NUM> due to the change in the posture and the height of the ship <NUM> are reflected based on the positional information and the posture information acquired by the position acquiring module <NUM> and the posture acquiring module <NUM>. The data synthesizing module <NUM> may interlock with the change in the position and the direction of the camera <NUM> to change the position and the direction of the projection screen <NUM> placed in the three-dimensional virtual space <NUM>.

Then, the data synthesizing module <NUM> may generate a two-dimensional image by performing known rendering to the three-dimensional scene data <NUM> and the projection screen <NUM>. In more detail, the data synthesizing module <NUM> may place as the virtual camera, the viewpoint camera <NUM> in the three-dimensional virtual space <NUM>, and define a frustum (visual cone) <NUM> which defines an area to be rendered so that the viewpoint camera <NUM> is used as a vertex and its line-of-sight direction becomes the center axis. Then, the data synthesizing module <NUM> may convert coordinates of a vertex of the polygon located inside the frustum <NUM> among the polygons which constitute each object (one of the virtual reality objects 41v, 42v,. , and the projection screen <NUM>) into coordinates of the two-dimensional virtual screen corresponding to the display area of the synthesized image in the display unit <NUM> by a perspective projection. Then, based on the vertex placed on the virtual screen, a two-dimensional image may be generated by generating and processing pixels at a given resolution.

The two-dimensional image generated in this way may include graphics obtained by rendering the three-dimensional scene data <NUM> (in other words, the graphics as results of rendering the virtual reality objects 41v, 42v,. Moreover, during the generation of the two-dimensional image, the captured image of the camera <NUM> may be placed so as to be pasted at the position corresponding to the projection screen <NUM>. Therefore, the synthesizing of the images by the data synthesizing module <NUM> may be realized. Since the projection screen <NUM> has a shape which is curved so as to meet the spherical shell centering on the camera <NUM>, it can prevent distortion of the captured image by a perspective projection.

The viewpoint camera <NUM> may define the viewpoint of the synthesized image, and its position and direction may be defined by settings of the viewpoint setting module <NUM>. However, by making a special setting in the viewpoint setting module <NUM>, the data synthesizing module <NUM> may set a mode when generating the synthesized image as a mode in which the position and the direction of the viewpoint camera <NUM> change automatically so that they are always in agreement with the position and the direction of the camera <NUM> (viewpoint tracking mode). In this viewpoint tracking mode, since the entire view of the viewpoint camera <NUM> is always covered by the projection screen <NUM> (i.e., the captured image of the camera <NUM>), the realistic synthesized image with presence can be realized.

On the other hand, the data synthesizing module <NUM> may also set the mode into a mode in which the position and the direction of the viewpoint camera <NUM> follows the viewpoint set in the viewpoint setting module <NUM> by a proper operation of an input device, regardless of the position and the direction of the camera <NUM> (independent viewpoint mode). The input device may include, for example, a keyboard <NUM>, a mouse <NUM>, a touch panel (not illustrated), a joy stick, etc. In this independent viewpoint mode, the user can move the viewpoint freely to confirm the additional display information at a position out of the imaging coverage of the camera <NUM>.

Next, a relation between the image captured by the camera <NUM> and the synthesized image is described referring to one example. <FIG> is a view illustrating one example of the image captured by the camera <NUM>. <FIG> is a view illustrating the synthesized image outputted from the data synthesizing module <NUM>.

In <FIG>, one example of the image captured by the camera <NUM> of the ship <NUM> in the situation indicated in <FIG> is illustrated. Another ship 44r floating on the sea surface is in the captured image. Moreover, a bow part of the ship <NUM> is located at the lower center of the image.

Since the virtual buoy <NUM> is virtual as described above, it will not be caught by the camera <NUM>, as illustrated in <FIG>. Since the waypoint <NUM>, the route line <NUM>, and the stopping area <NUM> are also created by the setup of the plotter <NUM>, they will not appear in the image captured by the camera <NUM>.

<FIG> illustrates a result of synthesizing the two-dimensional image obtained by rendering the three-dimensional scene data <NUM> in <FIG> with the captured image illustrated in <FIG>. However, in <FIG>, the parts which appear in the image captured by the camera <NUM> are illustrated by broken lines for convenience in order to facilitate the distinction from other parts (similar processing is also applied to other views indicating the synthesized image). In the synthesized image of <FIG>, the graphics 41f, 42f, 43f, 44f, and 45f expressing the additional display information are placed so as to overlap with the captured image. The graphic 44f indicative of another ship is placed so as to substantially overlap with the position of another ship 44r in the captured image.

The graphics 41f, 42f,. may be generated as results of rendering the three-dimensional shapes of the virtual reality objects 41v, 42v,. which constitute the three-dimensional scene data <NUM> illustrated in <FIG>, from the viewpoint at the same position and the direction as those of the camera <NUM>. Therefore, even when the graphics 41f, 42f,. are superimposed with the realistic image captured by the camera <NUM>, a visual disagreement may not substantially occur.

As illustrated in <FIG>, the graphics 41f, 42f,. which express the additional display information in virtual reality may be placed on the synthesized image as if they are placed on the sea surface in the captured image. In other words, the graphics 41f, 42f,. which express the additional display information in virtual reality may be arranged along the water surface in the captured image. This may be realized by placing the virtual reality objects 41v, 42v,. illustrated in <FIG> so as to touch the xz plane located below the camera <NUM> by a distance calculated based on the height set by the imaging position setting module <NUM> (see <FIG>), and correctly placing the position of the projection screen <NUM> in consideration of the position and the direction of the camera <NUM>.

Next, a change in the synthesized image accompanying the shake of the ship <NUM> is described. <FIG> is a conceptual diagram illustrating a case where the ship <NUM> shakes in the pitch direction and the roll direction from the state of <FIG>. <FIG> is a view illustrating the synthesized image in the case of <FIG>.

As described above, since the camera <NUM> is attached to the ship <NUM>, its position and direction may change in connection with the posture of the ship <NUM> inclining by a wave etc., or the ship <NUM> running over a wave. In this embodiment, when a shake (pitching, rolling and heaving) occurs on the ship <NUM>, the data synthesizing module <NUM> may change the position and the direction of the camera <NUM> in the three-dimensional virtual space <NUM> so as to simulate a change in the posture of the ship <NUM> acquired by the posture acquiring module <NUM>, and a change in the position of the ship <NUM> in the vertical direction acquired by the position acquiring module <NUM>, and change the position of the projection screen <NUM> accordingly.

In <FIG>, a situation when the posture of the ship <NUM> changes in the pitch direction and the roll direction from the state of <FIG> is illustrated. In the example of <FIG>, the ship <NUM> inclines the bow downward and the port downward, and the position and the direction of the camera <NUM> change so as to reflect the inclination. The projection screen <NUM> may interlockedly move to directly oppose to the camera <NUM> of which the position and the direction changed.

In the example of <FIG>, by the viewpoint tracking mode, the position and the direction of the viewpoint camera <NUM> also change so as to follow the camera <NUM> of which the position and the direction changed as described above. One example of the synthesized image corresponding to <FIG> is illustrated in <FIG>, and as illustrated in this figure, even if the position and the direction of the camera <NUM> change in connection with the shake of the ship <NUM>, since the position and the direction of the projection screen <NUM> interlockedly change, and the position and the direction of the viewpoint camera <NUM> which renders the three-dimensional scene change, a comfortable synthesized image can be continuously obtained.

In the viewpoint tracking mode, each time the pitch angle or the roll angle changes more than a given value with the shakes of the ship <NUM>, the rendering of the three-dimensional scene data <NUM> may be updated by the data synthesizing module <NUM>, and, therefore, the graphics 41f, 42f,. based on the latest viewpoint may be generated. Accordingly, the display of the graphics 41f, 42f,. can be changed appropriately to maintain a state where they are placed on the sea surface, while the direction in which the sea surface appears change in the captured image of the camera <NUM> with the shakes of the ship <NUM>.

Therefore, the extended-reality image in which the imaginary objects can be seen as if they are floating on the sea surface, and which is natural and high in the actual feeling can be obtained. Moreover, by the user looking at the sea surface projected on the display unit <NUM>, since the graphics 41f, 42f,. indicating the virtual reality come into the user's field of view comprehensively, he/she can acquire every piece of necessary information.

Next, processing executed by the target object position estimating module <NUM> and the target object direction estimating module <NUM> for placing the virtual reality object related to the moving target object (e.g., the virtual reality object 44v of another ship) in the three-dimensional virtual space <NUM> is described.

In the above example, the information on another ship <NUM> indicated by the virtual reality object 44v may be received by the AIS receiver <NUM>. That is, another ship <NUM> may be an AIS target object. Each time the AIS receiver <NUM> receives the AIS information, it may output this information to the image generating device <NUM> on real time. However, since the interval at which another ship <NUM> transmits its position etc. as the AIS information changes according to the situation, it may be about <NUM> time per <NUM> seconds, and, therefore, the updating of the information may not be frequent.

Thus, if the position and the direction of the virtual reality object 44v which constitute the three-dimensional scene data <NUM> is updated only at the timing of the image generating device <NUM> acquiring the AIS information, the motion of the graphic 44f indicative of another ship in the synthesized image may become discontinuously and coarse, thereby causing an unnatural indication.

In this regard, the image generating device <NUM> of this embodiment may be provided with the target object position estimating module <NUM>, the target object direction estimating module <NUM>, and the image updating module <NUM>. The target object position estimating module <NUM> and the target object direction estimating module <NUM> may estimate the position and the direction of another ship <NUM> at a timing later than the acquisition of the additional display information, based on the latest-acquired additional display information (AIS information) for another ship <NUM>. Then, the image updating module <NUM> may cause the synthesized image generating module <NUM> to update the synthesized image so that the results of the estimation are reflect to the position and the direction of the virtual reality object 44v.

Below, this update is concretely described with reference to <FIG> is a conceptual diagram illustrating a situation where the position and the direction of the virtual reality object 44v of another ship being placed in the three-dimensional virtual space <NUM> are updated for the additional display information on another ship <NUM> based on the AIS information.

In <FIG>, the three-dimensional virtual space <NUM> where the three-dimensional scene generating module <NUM> establishes the three-dimensional scene is illustrated. It is assumed that the AIS receiver <NUM> receives the AIS information from another ship <NUM> at a certain time (this time is set as t=<NUM>), and the additional display information acquiring module <NUM> of the image generating device <NUM> acquires the position, the speed, the bow direction, and the turning angular velocity of another ship <NUM> based on the AIS information. Based on this result, the three-dimensional scene generating modules <NUM> may place the virtual reality object 44v at a position and in a direction corresponding to the obtained position and bow direction.

Also after that, the image generating device <NUM> may stand by for the additional display information acquiring module <NUM> acquiring new AIS information. However, when the AIS information is not received for a certain period of time (e.g., for <NUM> seconds), the target object position estimating module <NUM> and the target object direction estimating module <NUM> may estimate the position and the bow direction of another ship <NUM> at this timing, and the image updating module <NUM> may cause the synthesized image generating module <NUM> to update the synthesized image based on the estimated results. That is, the image updating module <NUM> may cause the three-dimensional scene generating module <NUM> to update the three-dimensional scene so that the virtual reality object 44v is rearranged based on the estimated results for the position and the bow direction of another ship <NUM>, and may cause the data synthesizing module <NUM> to synthesize a new three-dimensional scene with the captured image. This estimation can be obtained by calculating by using a known formula based on the speed and the turning angular velocity of another ship <NUM> obtained at t=<NUM>, when the position and the bow direction of another ship <NUM> obtained at the same timing are used as references. Therefore, also at t=<NUM>, the position and the direction of the virtual reality object 44v can be updated.

In the example of <FIG>, the estimation of the position and the direction by the target object position estimating module <NUM> and the target object direction estimating module <NUM> is performed every <NUM> seconds. Therefore, also at t=<NUM> and t=<NUM>, the position and the direction of the virtual reality object 44v are updated based on the estimation.

It is assumed that, at a timing of t=<NUM> which is <NUM> seconds after the reception of the last AIS information, the AIS receiver <NUM> receives the new AIS information from another ship <NUM>. In this case, the three-dimensional scene generating module <NUM> may immediately update the position and the direction of the virtual reality object 44v so that they become the obtained position and bow direction.

Also after that, the estimation of the position and the direction by the target object position estimating module <NUM> and the target object direction estimating module <NUM> may be performed every <NUM> seconds. Therefore, at a timing of t=<NUM>, the position and the direction of the virtual reality object 44v may be updated based on the estimation.

By configured in this way, even when the acquisition time interval of the additional display information indicative of the position of another ship <NUM> is long, the indication of the graphic 44f in the synthesized image can be updated at a comparatively short time interval by estimating the position and the bow direction of another ship <NUM> and updating the position and the direction of the virtual reality object 44v. Therefore, the extended virtual reality expression in the synthesized image can be realized with the smooth motion.

Although in the example of <FIG> the position and the bow direction of another ship <NUM> are estimated and the position and the direction of the virtual reality object 44v are updated every <NUM> seconds, the interval of updating the position etc. by the estimation can be arbitrarily defined as <NUM> second, <NUM> seconds, <NUM> seconds, <NUM> seconds, etc., other than <NUM> seconds. Moreover, the time interval for updating position etc. may be carried out at an unequal time interval, instead of the equal time interval as described above.

Note that the processing for updating the position and the direction of the virtual reality object 44v by the estimation of the position and the direction may also be effective, for example, for another ship (another ship as a TT target object) detected by the radar device <NUM>, as well as another ship <NUM> based on the AIS information. That is, since the radar image in the radar device <NUM> is not updated until the radar antenna revolves one time, the updating of the TT information may not be frequent. Therefore, by estimating the position and the direction and finely updating the position and the direction of the virtual reality object as described above, the coarseness of the motion in the synthesized image can be reduced, thereby improving the reality.

As described above, the image generating device <NUM> of this embodiment may include the captured image inputting module <NUM>, the position acquiring module <NUM>, the posture acquiring module <NUM>, the additional display information acquiring module <NUM>, the synthesized image generating module <NUM>, the target object position estimating module <NUM>, and the image updating module <NUM>. The captured image inputting module <NUM> may acquire the image captured by the camera <NUM> installed in the ship <NUM>. The position acquiring module <NUM> may acquire the positional information indicative of the position of the ship <NUM>. The posture acquiring module <NUM> may acquire the posture information indicative of the posture of the ship <NUM>. The additional display information acquiring module <NUM> may acquire the additional display information including the information indicative of the position of another ship <NUM> which is a target object. The synthesized image generating module <NUM> may generate the synthesized image in which the graphic 44f rendering the three-dimensional virtual reality object 44v indicative of the additional display information is synthesized with the captured image of the camera <NUM> based on the positional information, the posture information, and the additional display information. The target object position estimating module <NUM> may estimate the position of another ship <NUM> which is the target object at a timing later than the timing at which the latest additional display information is acquired for another ship <NUM> based on the speed of another ship <NUM> contained in the additional display information. The image updating module <NUM> may cause the synthesized image generating module <NUM> to update the graphic of the virtual reality object 44v of another ship <NUM> (i.e., the graphic 44f) in the synthesized image based on the result of the estimation by the target object position estimating module <NUM>.

Therefore, even when the acquisition time interval of the additional display information indicative of the position of the target object (in this embodiment, another ship <NUM>) is long, the graphic 44f rendering the virtual reality object 44v can be updated by estimating the position of another ship <NUM> at the suitable timing. Therefore, it may become possible to realize the virtual reality image with a smooth motion.

Moreover, the image generating device <NUM> of this embodiment may be provided with the target object direction estimating module <NUM>. This target object direction estimating module <NUM> may estimate the bow direction of another ship <NUM> after the AIS information is acquired for another ship <NUM> until the subsequent AIS information is acquired based on the turning angular velocity of another ship <NUM> contained in the additional display information. The image updating module <NUM> may cause the synthesized image generating module <NUM> to update the graphic of the virtual reality object 44v of another ship <NUM> (i.e., the graphic 44f) in the synthesized image based on the result of the estimation by the target object direction estimating module <NUM>.

Therefore, the graphic 44f rendering the virtual reality object 44v can be updated by estimating the direction of the target object (in this embodiment, another ship <NUM>) similarly to the position. Therefore, a more natural motion of the image can be realized.

Moreover, in the image generating device <NUM> of this embodiment, the synthesized image generating module <NUM> can synthesize the graphic 44f rendering the virtual reality object 44v of the shape of another ship with the image captured by the camera <NUM>.

Therefore, the movement of a surrounding ship can be expressed by the smooth motion of the three-dimensional computer graphic of another ship.

Moreover, in the image generating device <NUM> of this embodiment, the additional display information may include the AIS information received by the AIS receiver <NUM>.

Therefore, even when another ship <NUM> is based on the AIS information of which the updating frequency tends to be less in many cases, the virtual reality image with the smooth motion can be realized.

Moreover, in the image generating device <NUM> of this embodiment, the additional display information may include the information on the position and the speed of another ship which is acquired based on the transition of the radar image acquired by the radar device <NUM>.

Therefore, even when another ship is based on the information from the radar device <NUM> which cannot be updated, such as the position of the target object, until the radar antenna revolves one time, the virtual reality display with the smooth motion can be realized.

Although the suitable embodiment of the present disclosure is described above, the above configuration may be changed, for example, as follows.

The three-dimensional scene generating module <NUM> may be configured to estimate only the position among the position and the direction of the moving target object (the direction is not estimated).

In an example that does not form part of the invention, since the additional display information is various as described above, the additional display information may not include the information on the speed of the target object. In this case, the target object position estimating module <NUM> may estimate the speed, for example, based on the transition of the position of the target object contained in the additional display information, and estimate the position of the target object based on the estimated speed. Thus, the target object position estimating module <NUM> may estimate the position of the target object based on the speed of the target object estimated from the additional display information, instead of the speed of the target object contained in the additional display information.

Similarly to the above, the additional display information may not include the information on the angular velocity of the direction of the target object. In this case, the target object direction estimating module <NUM> may estimate the angular velocity, for example, based on the transition of the direction of the target object contained in the additional display information, and estimate the direction of the target object based on the estimated angular velocity. Thus, the target object direction estimating module <NUM> may estimate the direction of the target object based on the angular velocity of the direction of the target object estimated from the additional display information, instead of the angular velocity of the direction of the target object contained in the additional display information.

The data synthesizing module <NUM> may be configured so that the three-dimensional scene data <NUM> and the projection screen <NUM> may not be rendered simultaneously. That is, the data synthesizing module <NUM> may be configured to create separately the two-dimensional images which are rendering results of only the three-dimensional scene data <NUM> (the images of the graphics 41f, 42f,. ), and the two-dimensional image which is a rendering result of only the projection screen <NUM> (the image in which the captured image is pasted on the projection screen <NUM>), and then synthesize the two-dimensional images. In this case, while performing the rendering of the three-dimensional scene data <NUM> as needed according to the movement etc. of the ship <NUM>, the rendering of the projection screen <NUM> can be performed at a short time interval according to the frame rate of the image captured by the camera <NUM>.

In the camera <NUM>, the pan/tilt function described above may be omitted, and the imaging direction may be fixed, for example, to forward. Moreover, the camera <NUM> may be installed so as to image a direction other than the forward (e.g., rearward).

The camera <NUM> may be configured so that it can simultaneously image the all directions over <NUM>° around the ship <NUM>.

When the user changes the direction of the viewpoint camera <NUM>, the pan/tilt operation of the camera <NUM> may be automatically performed so as to follow the change in the direction.

Upon generating the three-dimensional scene data <NUM> by the three-dimensional scene generating module <NUM>, the virtual reality objects 41v, 42v,. may be placed on the basis of the bow direction using the position of the ship <NUM> as the origin in the above embodiment, as illustrated in <FIG>. However, the virtual reality objects 41v, 42v,. may be placed on the basis of north in which the +z direction always points north, without the bow direction. In this case, when the bow direction of the ship <NUM> changes due to a turn etc., the direction of the ship <NUM> in the three-dimensional virtual space <NUM> may be changed to the yaw direction, instead of rearranging the virtual reality objects 41v, 42v,. Then, the changes in the position and the direction of the camera <NUM> at this time may be simulated in the three-dimensional virtual space <NUM>, and a rendering may be performed in the interlocking manner while changing the position and the direction of the viewpoint camera <NUM>, thereby obtaining the completely same rendering result as the case of the bow direction basis.

Alternatively, the coordinate system of the three-dimensional virtual space <NUM> may be defined so that a fixed point suitably defined on the earth is used as the origin, and, for example, the +z direction serves as north and the +x direction serves as east, instead of using the position of the ship <NUM> as the origin. In this case, in the three-dimensional virtual space <NUM> of the coordinate system fixed to the earth, the position and the direction of the ship <NUM> may change based on the positional information and the posture information, and the change in the position and the direction of the camera <NUM> accompanying the change may be simulated in the three-dimensional virtual space <NUM>.

In the image generating device <NUM>, a shake of the synthesized image accompanying the shake of the ship <NUM> may be reduced. For example, the three-dimensional scene generating module <NUM> may reduce the change in the position and the direction of the viewpoint camera <NUM>, even when the ship <NUM> shakes.

The display mode of the graphic 44f in the synthesized image may be changed between the case where the virtual reality object 44v is placed based on the additional display information and the case where the virtual reality object 44v is placed based on the results of the estimations by the target object position estimating module <NUM> and the target object direction estimating module <NUM>. For example, it is possible to display near the graphic 44f a mark indicating that the position and the direction are based on the estimations.

The ship instruments (information source of the additional display information) connected to the image generating device <NUM> are not limited to what is described in <FIG>, and other ship instruments may be included.

The present disclosure is applicable not only to the ship which travels on the sea, but may also be applicable to arbitrary water-surface movable bodies which can travel, for example, the sea, a lake, or a river.

When displaying the graphics etc. superimposedly on the image captured by the imaging device as described above, the limited display area can be effectively utilized by simultaneously displaying additional information 91a and 91b, such as the scale images etc. indicative of the direction, as illustrated in <FIG> and <FIG>. At this time, the positions of the additional information may be automatically changed and moved so that the graphics are not hidden by the additional information as much as possible. Moreover, the additional information 91a and 91b may be displayed so as to incline according to the inclination of the hull. By displaying in such a way, exact additional information can always be visually observed even when the hull inclines.

Claim 1:
An image generating device (<NUM>), comprising:
a captured image inputting module (<NUM>) configured to input an image captured by an imaging device (<NUM>) mounted on a water-surface movable body (<NUM>);
a position acquiring module (<NUM>) configured to acquire positional information indicative of a position of the water-surface movable body (<NUM>);
a posture acquiring module (<NUM>) configured to acquire posture information indicative of a posture of the water-surface movable body (<NUM>);
an additional display information acquiring module (<NUM>) configured to non-continuously acquire additional display information including information indicative of a position of a target object (<NUM>) at consecutive points in time;
a synthesized image generating module (<NUM>) configured to generate, every time additional display information is acquired, a synthesized image where a graphic rendering a three-dimensional virtual reality object indicative of the additional display information is synthesized with the captured image based on measured position and imaging direction of the imaging device relative to the water-surface movable body, the positional information, the posture information, and the additional display information;
a target object position estimating module (<NUM>) configured to estimate a position of the target object (<NUM>) at a timepoint between acquisition of two consecutive pieces of additional display information, based on a speed of the target object (<NUM>) included in the last acquired additional display information; and
an image updating module (<NUM>) configured to cause the synthesized image generating module (<NUM>) to update the rendering of the three-dimensional virtual reality object of the target object (<NUM>) in the synthesized image such that the three-dimensional virtual reality object is rearranged in the synthesized image based on a result of the estimation by the target object position estimating module (<NUM>) at said timepoint between acquisition of two consecutive pieces of additional display information.