Patent Description:
Conventionally, a remote operation system allowing to remotely operate a work machine at a separate place while viewing a remote image displayed on a head-mounted type image display device is known. For example, Patent Document <NUM> discloses a remote operation system for remotely operating an excavator as the work machine.

The system disclosed in Patent Document <NUM> is configured to switch the contents of an image displayed on a head-mounted type image display device in accordance with a movement of the head of the operator detected by a wearable sensor. Specifically, in the system, when the head of the operator pitches down, the displayed image is switched from a front image (remote image) captured by an image capturing device installed to the excavator to an instrument screen image, and when the head of the operator pitches up from this state, the displayed image is switched back to the remote image. <CIT> teaches a remote operation system according to the preamble of claim <NUM>, in order to operate the forklift. The forklift is equipped with a vehicle communication unit, an operation sensor and a vehicle operation unit. The remote operation system may perform remote operation stop control for stopping the remote operation and movement stop control for stopping the movement of the forklift. <CIT> teaches a remote control system for remotely controlling a work machine. The remote control system includes a stereo camera and a head-mounted display. The remote control system extracts from video picked up by the stereo camera, a partial video in a desired direction according to a direction of a head of an operator detected by the angle sensor, and displays correction video in the head-mounted display. <CIT> teaches a remote operation terminal to perform remote operation of a work device. The remote operation terminal being provided with a terminal-side control device that is configured to calculate the actuation direction of the crane device in relation to the operation of the suspended load movement operation tool. <CIT> teaches a system and method to retrofit manual lift trucks with a supplemental control system (retrofit kit) that includes sensors, communication devices, computers, electrical circuits and mechanical actuators such that a lift truck can carry out material handling tasks autonomously without the presence of a human operator. The retrofit also allows the lift truck to be controlled remotely by a human tele-operator and can transmit and receive data from a remote computer. <CIT> teaches a remote driving system of a forklift, for reducing staffs working strength, improving handling efficiency, realizing the safety in production.

Optional features of the invention are provided by the dependent claims. In the system disclosed in Patent Document <NUM>, the remote image or the instrument screen image is selectively displayed on the head-mounted type image display device. That is, in the system, no instrument information of the excavator is provided to the operator confirming the movement of an arm by using the remote image. Therefore, the operator needs to operate a remote operation with information less than the information obtained when performing a boarding operation.

The invention has been made in view of the above, and an objective is to provide a remote operation system capable of providing beneficial and abundant information to an operator performing a remote operation while viewing a remote image.

In order to solve the above issue, a remote operation system according to the invention includes: an image capturing device, provided at a forklift at a place separate from an operator; a head-mounted type image display device, mounted to a head of the operator; a command reception device, within reach of a hand of the operator and receiving a remote operation command by the operator; and a synthesized remote image generation device, obtaining a content of the remote operation command received by the command reception device, an action state of the forklift acting in accordance with the remote operation command, and a remote image captured by the image capturing device, and generating a synthesized remote image by synthesizing a command display image indicating the content of the remote operation command and an action display image indicating the action state of the forklift into the remote image. The head-mounted type image display device displays the synthesized remote image that is generated by the synthesized remote image generation device before eyes of the operator.

The head-mounted type image display device of the remote operation system also includes a posture detection part detecting a posture of a head of the operator. In such case, the synthesized remote image generation device of the remote operation system obtains the posture detected by the posture detection part, and generates the synthesized remote image by synthesizing the command display image and the action display image into a portion of the remote image that is cut out in accordance with the posture.

The synthesized remote image generation device of the remote operation system: (<NUM>) may be disposed on a side of the operator, and obtain the action state of the forklift and the remote image from the forklift via an Internet line; (<NUM>) may be disposed at the forklift, and obtain the content of the remote operation command from the command reception device via an Internet line; and (<NUM>) may be disposed at a place separate from the operator and the forklift, obtains the action state of the forklift and the remote image from the forklift via an Internet line, and obtain the content of the remote operation command from the command reception device via the Internet line.

The command reception device of the remote operation system may include four buttons arranged in a cross shape and corresponding to lifting and lowering of a fork provided at the forklift and forward and backward leaning of a mast provided at the forklift, In such case, the command display image may include four cargo symbols arranged in a cross shape and corresponding to the four buttons, and when any one of the four buttons is operated, an appearance of the cargo symbol corresponding to the button may change.

The command reception device of the remote operation system may include an analog stick for providing a command on a traveling direction and a traveling speed of the forklift. The command display image may include a vehicle symbol with the forklift set as a motif and a traveling symbol whose relative position from the vehicle symbol changes in response to a tilting direction and a tilting amount of the analog stick.

According to the invention, a remote operation system capable of providing beneficial and abundant information to an operator performing a remote operation while viewing a remote image can be provided.

In the following, the embodiments of the remote operation system according to the invention will be described with reference to the drawings.

<FIG> is a view illustrating a remote operation system 1A according to a first embodiment of the invention. The remote operation system 1A is a system for an operator O to remotely operate a forklift F at a separate place. As shown in <FIG>, the remote operation system 1A includes a head-mounted type image display device <NUM>, a command reception device <NUM>, a first computation device 12A, and a first communication device <NUM> on the side of the operator O, as well as an image capturing device <NUM>, a second computation device 21A, and a second communication device <NUM> on the side of the forklift F.

The head-mounted type image display device <NUM> is known as a head-mounted display (HMD). The head-mounted type image display device <NUM> is mounted to the head of the operator O, and displays a synthetic remote image, which will be described afterwards, before the eyes of the operator O.

The head-mounted type image display device <NUM> has a posture detection part not shown herein. The posture detection part detects a posture of the head of the operator O, and outputs a signal (referred to as "posture signal" in the following) corresponding to the posture. The posture signal includes a signal indicating to which extent the head of the operator O tilts in the left-right direction and a signal indicating to which extent the head of the operator O tilts in the upper-lower direction.

The command reception device <NUM> is similar to a controller of a game console, and, as shown in <FIG>, includes four buttons <NUM>, <NUM>, <NUM>, and <NUM> arranged in a cross shape and an analog stick <NUM>. The command reception device <NUM> outputs a signal indicating whether each of the buttons <NUM>, <NUM>, <NUM>, and <NUM> is pressed (referred to as "button operation signal" in the following) and a signal indicating a tilting direction and a tilting amount of the analog stick <NUM> (referred to as "stick operation signal" in the following). The buttons <NUM>, <NUM>, <NUM>, and <NUM> are operated by using the left thumb of the operator O. Also, the analog stick <NUM> is operated by using the right thumb of the operator O.

The first computation device 12A is similar to a personal computer, and is wiredly connected to the head-mounted type image display device <NUM> and the command reception device <NUM> via an interface for external machine connection, such as USB. In the embodiment, the first computation device 12A is equivalent to a "synthesized remote image generation device".

The first communication device <NUM> is similar to a router, and is wiredly connected to the first computation device 12A via a communication interface such as LAN. The first communication device <NUM> is also connected to an Internet line NT.

The image capturing device <NUM> is a <NUM>° camera installed to the forklift F. Instead of a seat for an operator to be seated, the forklift F includes a pole <NUM> having a length less than <NUM>, and the image capturing device <NUM> is installed to an upper end of the pole <NUM>. That is, the image capturing device <NUM> is installed to a position where the head of the operator is located in the case where the operator is seated at the seat. The image capturing device <NUM> captures a <NUM>° remote image and outputs a signal (referred to as "remote image signal" in the following) related to the remote image.

The second computation device 21A is a vehicle control unit ECU capable of controlling a cargo device <NUM> (which includes a mast, a fork, and a hydraulic system driving the mast and the fork) and a traveling device <NUM> (which includes a steering device that changes the direction of a steering wheel and a motor that drives a driving wheel). The second computation device 21A is also provided with a function of grasping the state of each part of the forklift F and a function of determining whether an abnormality occurs in each part of the forklift F. The second computation device 21A generates a signal (referred to as "action state signal") relating the state of each part of the forklift F that is grasped and whether an abnormality occurs.

In the embodiment, the second computation device 21A is capable of constantly grasping at least the weight of the load on the fork, a tilt angle of the mast, a lift height of the fork, the rotation number of the motor (the traveling speed of the forklift F) and the residual amount of the battery.

The second communication device <NUM> is similar to a router disposed in the facility in which the forklift F works, and is wirelessly connected to the second computation device 21A via a communication interface. The second communication device <NUM> is also connected to the Internet line NT.

When the operator O presses one of the buttons <NUM>, <NUM>, <NUM>, and <NUM> of the command reception device <NUM>, the button operation signal corresponding to the button (e.g., the first button <NUM>) is transmitted to the second computation device 21A via a path as follows: "command reception device <NUM> -> first computation device 12A -> first communication device <NUM> -> Internet line NT -> second communication device <NUM> -> second computation device 21A". The button operation signal may be converted into an appropriate form on the path.

The second computation device 21A operates the cargo device <NUM> in accordance with the button operation signal transmitted from the command reception device <NUM>. Specifically, the second computation device 21A lifts the fork in accordance with the button operation signal corresponding to the first button <NUM>, lowers the fork in accordance with the button operation signal corresponding to the second button <NUM>, makes the mast lean rearward in accordance with the button operation signal corresponding to the third button <NUM>, and makes the mast lean forward in accordance with the button operation signal corresponding to the fourth button <NUM>.

When the operator O tilts the analog stick <NUM> of the command reception device <NUM> to any direction, the stick operation signal corresponding to the tilting direction and the tilting amount of the analog stick <NUM> is transmitted to the second computation device 21A via the same path as the path of the button operation signal, that is "command reception device <NUM> -> first computation device 12A -> first communication device <NUM> -> Internet line NT -> second communication device <NUM> -> second computation device 21A". The stick operation signal may be converted into an appropriate form on the path.

The second computation device 21A operates the traveling device <NUM> in accordance with the stick operation signal transmitted from the command reception device <NUM>. Specifically, the second computation device 21A operates the steering device, so that the traveling direction of the forklilft F is consistent with the direction corresponding to the tilting direction indicated in the stick operation signal, and operates the motor so that the traveling speed of the forklift F is consistent with the speed corresponding to the tilting amount indicated by the stick operation signal.

The remote image signal relating to the <NUM>° remote image output by the image capturing device <NUM> is transmitted to the first computation device 12A via a path as follows "image capturing device <NUM> -> second computation device 21A -> second communication device <NUM> -> Internet line NT -> first communication device <NUM> -> first computation device 12A". The remote image signal may be converted into an appropriate form on the path.

The action state signal relating to the state of each part of the forklift F and whether an abnormality occurs as grasped by the second computation device 21A is transmitted to the first computation device 12A via a path as follows: "second computation device 21A -> second communication device <NUM> -> Internet line NT -> first communication device <NUM> -> first computation device 12A".

The posture signal output by the head-mounted type image display device <NUM> is transmitted to the first computation device 12A via a path as follows "head-mounted type image display device <NUM> -> first computation device 12A".

Accordingly, the first computation device 12A, as the synthesized remote image generation device, obtains the button operation signal and the stick operation signal output by the command reception device <NUM>, the posture signal output by the head-mounted type image display device <NUM>, the remote image signal output by the image capturing device <NUM>, and the action state signal output by the second computation device 21A.

The first computation device 12A extracts a portion of the remote image based on the obtained posture signal. In other words, the first computation device 12A, as shown in <FIG>, cuts out an image A in a size equivalent to the field of view of the operator O from the <NUM>° remote image in accordance with the posture of the operator O. If the posture of the head of the operator O changes, the position of the image A that is cut out also changes. Meanwhile, even if the posture of the head of the operator O changes, the size of the image A that is cut out does not change.

The first computation device 12A generates a command display image based on the button operation signal and the stick operation signal that are obtained.

As shown in <FIG>, the command display image includes four cargo symbols <NUM>, <NUM>, <NUM>, and <NUM> disposed in a cross shape and corresponding to the buttons <NUM>, <NUM>, <NUM>, and <NUM> of the command reception device <NUM>. The first cargo symbol <NUM> corresponds to the first button <NUM> for lifting the fork, and the appearance (e.g., colors, the same applies to the following) changes depending on whether the first button <NUM> is pressed. The second cargo symbol <NUM> corresponds to the second button <NUM> for lowering the fork, and the appearance changes depending on whether the second button <NUM> is pressed. The third cargo symbol <NUM> corresponds to the third button <NUM> for tilting the mast rearward, and the appearance changes depending on whether the third button <NUM> is pressed. The fourth cargo symbol <NUM> corresponds to the fourth button <NUM> for leaning the mast forward, and the appearance changes depending on whether the fourth button <NUM> is pressed.

As shown in <FIG>, the command display image further includes a vehicle symbol <NUM> setting the forklift F as a motif and a traveling symbol <NUM> located in the vicinity of the vehicle symbol <NUM>. The relative position of the traveling symbol <NUM> with reference to the vehicle symbol <NUM> changes in accordance with the tilting direction and the tilting amount of the analog stick <NUM>.

The arrangement of the cargo symbols <NUM>, <NUM>, <NUM>, <NUM>, the vehicle symbol <NUM>, and the traveling symbol <NUM> in the command display image substantially correspond to the arrangement of the buttons <NUM>, <NUM>, <NUM>, and <NUM> and the analog stick <NUM> in the command reception device <NUM>. That is, the symbols <NUM>, <NUM>, <NUM>, and <NUM> corresponding to the buttons <NUM>, <NUM>, <NUM>, and <NUM> operated by the left thumb are arranged on the left side of the command display image, and the symbols <NUM> and <NUM> corresponding to the analog stick <NUM> operated by the right thumb are disposed on the right side of the command display image.

The command display image is an image indicating the content of a remote operation command received by the command reception device <NUM>, that is, an image indicating the content of a remote operation command input by the operator O.

The first computation device 12A generates an action display image based on the action state signal that is obtained.

As shown in <FIG>, the action display image includes a load meter <NUM> indicating the weight of the load on the fork, a tilt meter <NUM> indicating the tilt angle of the mast, a lift meter <NUM> indicating the lift height of the fork, a speed meter <NUM> indicating the traveling speed of the forklift F, and a battery meter <NUM> indicating the residual power amount in the battery, and an hour meter <NUM> indicating the current time or the accumulated operation time. Further to the above, the action display image may also include multiple warning symbols <NUM> whose appearance changes, such as being lit up, when an abnormality occurs.

As shown in <FIG>, the first computation device 12A generates a synthesized remote image by synthesizing the command display image and the action display image into the portion (the image A) of the remote image. Then, the first computation device 12A transmits a signal (referred to as "synthesized remote image signal" in the following) related to the synthesized remote image to the head-mounted type image display device <NUM>.

The head-mounted type image display device <NUM> displays the synthesized remote image before the eyes of the operator O based on the synthesized remote image signal that is transmitted.

In this way, in the remote operation system 1A according to the first embodiment, the command display image and the action display image are presented to the operator O performing the remote operation while viewing the remote image. Therefore, the operator O can confirm that the operation command is correctly received by using the command display image and grasp the state of the forklift F that cannot be grasped from the remote image by using the action display image without interrupting the remote operation.

<FIG> is a view illustrating a remote operation system 1B according to a second embodiment of the invention. The remote operation system 1B includes the head-mounted type image display device <NUM>, the command reception device <NUM>, a first computation device 12B, and the first communication device <NUM> on the side of the operator O, as well as the image capturing device <NUM>, a second computation device 21B, and the second communication device <NUM> on the side of the forklift F. The remote operation system 1B differs from the remote operation system 1A in the point that the remote operation system 1B includes the first computation device 12B instead of the first computation device 12A and includes the second computation device 21B instead of the second computation device 21A, and the rest of the remote operation system 1B is common to the remote operation system 1A.

The first computation device 12B, like the first computation device 12A, is wiredly connected to the head-mounted type image display device <NUM> and the command reception device <NUM> via an interface for external machine connection, such as USB.

The second computation device 21B, like the second computation device 21A, is a vehicle control unit ECU capable of controlling the cargo device <NUM> and the traveling device <NUM>, and is provided with the function of grasping the state of each part of the forklift F and the function of determining whether an abnormality occurs in each part of the forklift F. In addition, like the second computation device 21A, the second computation device 21B generates the action state signal relating to the state of each part of the forklift that is grasped and whether an abnormality occurs. In the embodiment, the second computation device 21B is equivalent to a "synthesized remote image generation device".

When the operator O presses one of the buttons <NUM>, <NUM>, <NUM>, and <NUM> of the command reception device <NUM>, the button operation signal corresponding to the button (e.g., the first button <NUM>) is transmitted to the second computation device 21B via a path as follows: "command reception device <NUM> -> first computation device 12B -> first communication device <NUM> -> Internet line NT -> second communication device <NUM> -> second computation device 21B". The button operation signal may be converted into an appropriate form on the path.

The second computation device 21B, like the second computation device 21A, operates the cargo device <NUM> in accordance with the button operation signal transmitted from the command reception device <NUM>.

When the operator O tilts the analog stick <NUM> of the command reception device <NUM> to any direction, the stick operation signal corresponding to the tilting direction and the tilting amount of the analog stick <NUM> is transmitted to the second computation device 21B via the same path as the path of the button operation signal, that is "command reception device <NUM> -> first computation device 12B -> first communication device <NUM> -> Internet line NT -> second communication device <NUM> -> second computation device 21B". The stick operation signal may be converted into an appropriate form on the path.

The second computation device 21B, like the second computation device 21A, operates the traveling device <NUM> in accordance with the stick operation signal transmitted from the command reception device <NUM>.

The remote image signal relating to the <NUM>° remote image output by the image capturing device <NUM> is transmitted to the second computation device 21B via a path as follows "image capturing device <NUM> -> second computation device 21B".

The posture signal output by the head-mounted type image display device <NUM> is transmitted to the second computation device 21B via a path as follows "head-mounted type image display device <NUM> -> first computation device 12B -> first communication device <NUM> -> Internet line NT -> second communication device <NUM> -> second computation device 21B". The posture signal may be converted into an appropriate form on the path.

Accordingly, the second computation device 21B, as the synthesized remote image generation device, obtains the button operation signal and the stick operation signal output by the command reception device <NUM>, the posture signal output by the head-mounted type image display device <NUM>, the remote image signal output by the image capturing device <NUM>, and the action state signal generated by the second computation device 21B itself.

The second computation device 21B, like the first computation device 12A, extracts a portion of the remote image based on the obtained posture signal (see <FIG>).

The second computation device 21B, like the first computation device 12A, generates the command display image and the action display image based on the button operation signal, the stick operation signal, and the action state signal that are obtained.

The second computation device 21B, like the first computation device 12A, generates the synthesized remote image (see <FIG>) by synthesizing the command display image and the action display image into the portion (the image A) of the remote image. Then, the second computation device 21B transmits the synthesized remote image signal relating to the synthesized remote image that is generated to the head-mounted type image display device <NUM> via a path as follows: "second computation device 21B -> second communication device <NUM> -> Internet line NT -> first communication device <NUM> -> first computation device 12B -> head-mounted type image display device <NUM>".

In this way, in the remote operation system 1B according to the second embodiment as well, the command display image and the action display image are presented to the operator O performing the remote operation while viewing the remote image. Therefore, the operator O can confirm that the operation command is correctly received by using the command display image and grasp the state of the forklift that cannot be grasped from the remote image by using the action display image without interrupting the remote operation.

<FIG> is a view illustrating a remote operation system 1C according to a third embodiment of the invention. The remote operation system 1C includes the head-mounted type image display device <NUM>, the command reception device <NUM>, the first computation device 12B, and the first communication device <NUM> on the side of the operator O, the image capturing device <NUM>, the second computation device 21A and the second communication device <NUM> on the side of the forklift F, as well as a third computation device <NUM>. The remote operation system 1C differs from the remote operation system 1A in the point that the remote operation system 1C includes the first computation device 12B instead of the first computation device 12A and further includes the third computation device <NUM>, and the rest of the remote operation system 1C is common to the remote operation system 1A. The remote operation system 1C differs from the remote operation system 1B in the point that the remote operation system 1C includes the second computation device 21A instead of the second computation device 21B and further includes the third computation device <NUM>, and the rest of the remote operation system 1C is common to the remote operation system 1B.

The third computation device <NUM> is a computer providing cloud service and set at a place separate from the operator O as well as the forklift F, and is connected to the Internet line NT. In the embodiment, the third computation device <NUM> is equivalent to a "synthesized remote image generation device".

When the operator O presses one of the buttons <NUM>, <NUM>, <NUM>, and <NUM> of the command reception device <NUM>, the button operation signal corresponding to the button (e.g., the first button <NUM>) is transmitted to the second computation device 21A via a path as follows: "command reception device <NUM> -> first computation device 12B -> first communication device <NUM> -> Internet line NT -> second communication device <NUM> -> second computation device 21A", and is also transmitted to the third computation device <NUM> via a path as follows: "command reception device <NUM> -> first computation device 12B -> first communication device <NUM> -> Internet line NT -> third computation device <NUM>". The button operation signal may be converted into an appropriate form on the path.

When the operator O tilts the analog stick <NUM> of the command reception device <NUM> to any direction, the stick operation signal corresponding to the tilting direction and the tilting amount of the analog stick <NUM> is transmitted to the second computation device 21A and the third computation device <NUM> via the same paths as the paths of the button operation signal. That is, the stick operation signal is transmitted to the second computation device 21A via a path as follows: "command reception device <NUM> -> first computation device 12B -> first communication device <NUM> -> Internet line NT -> second communication device <NUM> -> second computation device 21A", and is also transmitted to the third computation device <NUM> via a path as follows: "command reception device <NUM> -> first computation device 12B -> first communication device <NUM> -> Internet line NT -> third computation device <NUM>". The stick operation signal may be converted into an appropriate form on the path.

The remote image signal relating to the <NUM>° remote image output by the image capturing device <NUM> is transmitted to the third computation device <NUM> via a path as follows "image capturing device <NUM> -> second computation device 21A -> second communication device <NUM> -> Internet line NT -> third computation device <NUM>". The remote image signal may be converted into an appropriate form on the path.

The action state signal relating to the state of each part of the forklift F that is grasped by the second computation device 21A and whether an abnormality occurs is transmitted to the third computation device <NUM> via a path as follows: "second computation device 21A -> second communication device <NUM> -> Internet line NT -> third computation device <NUM>".

The posture signal output by the head-mounted type image display device <NUM> is transmitted to the third computation device <NUM> via a path as follows "head-mounted type image display device <NUM> -> first computation device 12B -> first communication device <NUM> -> Internet line NT -> third computation device <NUM>". The posture signal may be converted into an appropriate form on the path.

Accordingly, the third computation device <NUM>, as the synthesized remote image generation device, obtains the button operation signal and the stick operation signal output by the command reception device <NUM>, the posture signal output by the head-mounted type image display device <NUM>, the remote image signal output by the image capturing device <NUM>, and the action state signal output by the second computation device 21A.

The third computation device <NUM>, like the first computation device 12A and the second computation device 21B, extracts a portion of the remote image based on the obtained posture signal (see <FIG>).

The third computation device <NUM>, like the first computation device 12A and the second computation device 21B, generates the command display image and the action display image based on the button operation signal, the stick operation signal, and the action state signal that are obtained.

The third computation device <NUM>, like the first computation device 12A and the second computation device 21B, generates the synthesized remote image (see <FIG>) by synthesizing the command display image and the action display image into the portion (the image A) of the remote image. Then, the third computation device <NUM> transmits the synthesized remote image signal relating to the synthesized remote image that is generated to the head-mounted type image display device <NUM> via a path as follows: "third computation device <NUM> -> Internet line NT -> first communication device <NUM> -> first computation device 12B -> head-mounted type image display device <NUM>".

In this way, in the remote operation system 1C according to the third embodiment as well, the command display image and the action display image are presented to the operator O performing the remote operation while viewing the remote image. Therefore, the operator O can confirm that the operation command is correctly received by using the command display image and grasp the state of the forklift that cannot be grasped from the remote image by using the action display image without interrupting the remote operation.

Although the first embodiment, the second embodiment, and the third embodiment of the remote operation system according to the invention are described as the above, the configuration of the invention is not limited thereto.

For example, the head-mounted type image display device <NUM> may also be wirelessly connected to the first computation device 12A, 12B or the first communication device <NUM>. In the case where the head-mounted type image display device <NUM> is wirelessly connected to the first communication device <NUM>, the posture signal is transmitted to the first computation device 12A via a path as follows "head-mounted type image display device <NUM> -> first communication device <NUM> -> first computation device 12A", transmitted to the second computation device 21B via a path as follows "head-mounted type image display device <NUM> -> first communication device <NUM> -> Internet line NT -> second communication device <NUM> -> second computation device 21B", or transmitted to the third computation device <NUM> via a path as follows "head-mounted type image display device <NUM> -> first communication device <NUM> -> Internet line NT -> third computation device <NUM>". In addition, in such case, the synthesized remote image signal is transmitted to the head-mounted type image display device <NUM> via a path as follows: "first computation device 12A -> first communication device <NUM> -> head-mounted type image display device <NUM>", via a path as follows "second computation device 21B -> second communication device <NUM> -> Internet line NT -> first communication device <NUM> -> head-mounted type image display device <NUM>", or a path as follows "third computation device <NUM> -> Internet line NT -> first communication device <NUM> -> head-mounted type image display device <NUM>".

In addition, the command reception device <NUM> may also be wirelessly connected to the first computation device 12A, 12B, or the first communication device <NUM>. In the case where the command reception device <NUM> is wirelessly connected to the first communication device <NUM>, the button operation signal and the stick operation signal are transmitted to the first computation device 12A via a path as follows "command reception device <NUM> -> first communication device <NUM> -> first computation device 12A", transmitted to the second computation device 21B via a path as follows "command reception device <NUM> -> first communication device <NUM> -> Internet line NT -> second communication device <NUM> -> second computation device 21B", or transmitted to the third computation device <NUM> via a path as follows "command reception device <NUM> -> first communication device <NUM> -> Internet line NT -> third communication device <NUM>".

In addition, the first computation device 12A, 12B may also be wirelessly connected to the first communication device <NUM>.

In addition, the command reception device <NUM> is not limited to a device similar to a controller of a game console held by both hands. In the invention, any device that is within reach of a hand the operator O and capable of receiving the remote operation command from the operator O can be used as the command reception device <NUM>.

In addition, the image capturing device <NUM> is not limited to a <NUM>° camera. In place of a <NUM>° camera, a combination of multiple cameras facing different directions may be used, for example. In such case, it suffices as long as the synthesized remote image generation device 12A, 21B, <NUM> selects the remote image captured by one or more of the cameras to be used in accordance with the posture signal, and synthesizes the command display image and the action display image into the selected remote image.

In addition, the image capturing device <NUM> may also be installed to the forklift F by a means other than the pole <NUM>. For example, the image capturing device <NUM> may also be suspended from a head guard.

In addition, the second computation device 21A, 21B may also be a unit separate from the vehicle control unit ECU handling the control of the cargo device <NUM> and the traveling device <NUM>. However, in such case, the two units need to be able to exchange signals with each other.

In addition, the command display image and the action display image shown in <FIG> merely serve as an example. The layouts of the images and the information included in the images may also be modified as appropriate. The layouts of the images and the information included in the images may also be customizable by the operator O.

Claim 1:
A remote operation system (1A, 1B, 1C), comprising:
an image capturing device (<NUM>), configured to be disposed at a forklift (F) at a place separate from an operator (O);
a head-mounted type image display device (<NUM>), adapted to be mounted to a head of the operator (O);
a command reception device (<NUM>), adapted to be disposed within reach of a hand of the operator (O) and receiving a remote operation command by the operator (O); and the remote operation system (1A, 1B, 1C) characterized in further comprising:
a synthesized remote image generation device (12A, 21B, <NUM>), obtaining a content of the remote operation command received by the command reception device (<NUM>), an action state of the forklift (F) acting in accordance with the remote operation command, and a remote image captured by the image capturing device (<NUM>), and generating a synthesized remote image by synthesizing a command display image indicating the content of the remote operation command and an action display image indicating the action state of the forklift (F) into the remote image,
wherein the head-mounted type image display device (<NUM>) displays the synthesized remote image that is generated by the synthesized remote image generation device (12A, 21B, <NUM>) before eyes of the operator (O),
wherein the head-mounted type image display device (<NUM>) comprises a posture detection part detecting a posture of the head of the operator (O),
the synthesized remote image generation device (12A, 21B, <NUM>) obtains the posture detected by the posture detection part, and generates the synthesized remote image by synthesizing the command display image and the action display image into a portion of the remote image that is cut out in accordance with the posture, wherein a size of the portion of the remote image is unchanged in response to change of the posture.