Patent Description:
A technology for remotely operating a work machine has been proposed (see Patent Literature <NUM>, for example). <CIT> discloses a machine automation system for a machine that includes an implement, comprising a plurality of vehicle electronic control modules, a local transceiver, an Ethernet LAN, a first controller area network, an autonomy electronic control module, an environment monitoring system and a remote shutdown module, wherein the autonomy electronic control module is configured to generate output control signals, and to transmit the output control signals to at least one of the vehicle electronic control modules, wherein the output control signals control an operation of the machine, wherein the remote shutdown module is disposed on the machine and configured to receive a safety control signal from off-board the machine, and wherein the machine automation system is configured to operate advance machine control features concurrently with execution of the input control signals received from off-board the machine.

When a work machine (actual machine) is remotely operated by a remote operator and is likely to be further operated by an actual machine operator on the work machine, however, the remote operation of the work machine performed by the remote operator in a situation in which the actual machine operator is about to get on the work machine may cause unexpected situations, such as contact between the work machine and the actual machine operator.

An object of the present invention is to provide a system and the like capable of assisting remote operation of the work machine performed by the remote operator in such a way that contact between the work machine and the actual machine operator and other unfavorable situations can be avoided.

A remote operation assistance server (<NUM>) according to the present invention comprises an evaluation result recognition element (<NUM>) that communicates with a work machine (<NUM>) comprising an actual machine operation mechanism and a mode switching mechanism that switches an actual machine operation mode in which the work machine (<NUM>) is operated in accordance with an operation aspect of the actual machine operation mechanism to a remote operation mode in which the work machine (<NUM>) is operated in accordance with an operation aspect of a remote operation mechanism that forms a remote operation apparatus (<NUM>) and vice versa, and recognizes as a first evaluation result a result of evaluation of whether or not the mode switching mechanism has switched the operation mode to the remote operation mode, based on the communication, wherein the remote operation assistance server (<NUM>) is characterized in that the evaluation result recognition element (<NUM>) further recognizes as a second evaluation result a result of evaluation of whether or not short-range wireless communication between the work machine (<NUM>) and a portable terminal (<NUM>) has been established, based on the communication, and in that the remote operation assistance server (<NUM>) further comprises a first assistance processing element (<NUM>) that carries out a first assistance process of permitting remote operation of the work machine (<NUM>) using the remote operation apparatus (<NUM>) based on communication with each of the remote operation apparatus (<NUM>) and the work machine (<NUM>) when the first evaluation result recognized by the evaluation result recognition element (<NUM>) is affirmative and the second evaluation result recognized by the evaluation result recognition element (<NUM>) is negative, and a second assistance processing element (<NUM>) that carries out a second assistance process of prohibiting remote operation of the work machine (<NUM>) using the remote operation apparatus (<NUM>) when the first and second evaluation results recognized by the evaluation result recognition element (<NUM>) are affirmative.

A remote operation assistance system according to the present invention comprises the remote operation assistance server according to the present invention, the work machine, and the portable terminal.

The remote operation assistance server and the remote operation assistance system (hereinafter referred to as "remote operation assistance server and the like") having the configuration described above recognize a first evaluation result and a second evaluation result. The phrase "recognizing evaluation results" is a concept that encompasses not only carrying out an evaluation process as an evaluation processing element to acquire a result of the evaluation, but also acquiring the result of the evaluation performed by the evaluation processing element directly from the evaluation processing element or acquiring the result indirectly via a storage device such as a database.

In a situation in which the first evaluation result is affirmative and the work machine can be remotely operated via the remote operation apparatus, and further in a situation in which the second evaluation result is negative and it is highly probable that a worker carrying the portable terminal, such as the actual machine operator, is so far away from the work machine that the short-range wireless communication between the work machine and the portable terminal is not established, remote operation of the work machine is permitted. On the other hand, even in the situation in which the first evaluation result is affirmative and the work machine can be remotely operated via the remote operation apparatus, but in a situation in which the second evaluation result is affirmative and it is highly probable that a worker carrying the portable terminal, such as the actual machine operator, is so close to the work machine or on the work machine that the short-range wireless communication between the work machine and the portable terminal is established, remote operation of the work machine is inhibited. Remote operation of the work machine performed by the remote operator can thus be assisted while unexpected situations, such as contact between the work machine and the actual machine operator, are avoided.

A remote operation assistance system as an embodiment of the present invention shown in <FIG> is formed of a remote operation assistance server <NUM>, a remote operation apparatus <NUM>, which is used to remotely operate a work machine <NUM>, and a portable terminal <NUM>.

The remote operation assistance server <NUM> comprises an evaluation result recognition element <NUM>, a first assistance processing element <NUM>, and a second assistance processing element <NUM>. Each of the elements is formed of an arithmetic processing apparatus (single-core processor, multi-core processor, or processor core that form one of processors), reads necessary data and software from a memory or any other storage, and performs arithmetic processing described later and according to the software on the data.

The remote operation apparatus <NUM> comprises a remote control apparatus <NUM>, a remote input interface <NUM>, and a remote output interface <NUM>. The remote control apparatus <NUM> is formed of an arithmetic processing apparatus (single-core processor, multi-core processor, or processor core that form one of processors), reads necessary data and software from a memory or any other storage, and performs arithmetic processing according to the software on the data. The remote input interface <NUM> comprises a remote operation mechanism <NUM>. The remote output interface <NUM> comprises an image output apparatus <NUM> and a remote wireless communication instrument <NUM>.

The remote operation mechanism <NUM> comprises a travel operation apparatus, a turning operation apparatus, a boom operation apparatus, an arm operation apparatus, and a bucket operation apparatus. The operation apparatuses each comprise operation levers that receive pivotal operation. The operation levers (travel levers) of the travel operation apparatus are operated to move a lower traveler <NUM> of the work machine <NUM>. The travel levers may also serve as travel pedals. For example, travel pedals fixed to a base or a lower end portion of the travel levers may be provided. An operation lever (turning lever) of the turning operation apparatus is operated to move a hydraulic turning motor that forms a turning mechanism <NUM> of the work machine <NUM>. An operation lever (boom lever) of the boom operation apparatus is operated to move a boom cylinder <NUM> of the work machine <NUM>. An operation lever (arm lever) of the arm operation apparatus is operated to move an arm cylinder <NUM> of the work machine <NUM>. An operation lever (bucket lever) of the bucket operation apparatus is operated to move a bucket cylinder <NUM> of the work machine <NUM>.

The operation levers that form the remote operation mechanism <NUM> are disposed, for example, around a seat St, where an operator is seated, as shown in <FIG>. The seat St has the form of a high-back chair with armrests, and may instead have any form that allows a remote operator OP1 to be seated, such as the form of a low-back chair without a headrest, or a chair without a backrest.

A pair of right and left travel levers <NUM> according to right and left crawlers are disposed sideways on the right and left in front of the seat St. A single operation lever may serve as the plurality of operation levers. For example, a right operation lever <NUM> provided in front of a right frame of the seat St shown in <FIG> may function as the boom lever when operated in the forward-rearward direction and as the bucket lever when operated in the rightward-leftward direction. Similarly, a left operation lever <NUM> provided in front of a left frame of the seat St shown in <FIG> may function as the arm lever when operated in the forward-rearward direction and as the turning lever when operated in the rightward-leftward direction. The lever pattern may be arbitrarily changed by the operator's operation instructions.

The image output apparatus <NUM> for example, is configured with a diagonally right forward image output apparatus <NUM>, a front image output apparatus <NUM> and a diagonally left forward image output apparatus <NUM> arranged diagonally right forward of, in front of and diagonally left forward of the seat St, respectively as shown in <FIG>. The image output apparatuses <NUM> to <NUM> may each further comprise a loudspeaker (voice output apparatus).

The work machine <NUM> comprises an actual machine control apparatus <NUM>, an actual machine input interface <NUM>, an actual machine output interface <NUM>, and an actuation mechanism <NUM>. The work machine control apparatus <NUM> is formed of an arithmetic processing apparatus (single-core processor, multi-core processor, or processor core that form one of processors), reads necessary data and software from a memory or any other storage, and performs arithmetic processing according to the software on the data.

The work machine <NUM> is, for example, a crawler excavator (construction machine), and comprises a crawler-type lower traveler <NUM> and an upper turner <NUM>, which is turnably mounted on the lower traveler <NUM> via the turning mechanism <NUM>, as shown in <FIG>. A cab (driver's chamber) <NUM> is provided at a front left portion of the upper turner <NUM>. A work attachment <NUM> is provided at a front center portion of the upper turner <NUM>.

The actual machine input interface <NUM> comprises an actual machine operation mechanism <NUM>, a mode switching mechanism <NUM>, and an imaging apparatus <NUM>. The actual machine operation mechanism <NUM> comprises a plurality of operation levers disposed around a seat disposed in the cab <NUM> in the same manner in which those of the remote operation mechanism <NUM> are disposed. The cab <NUM> is provided with a drive mechanism or a robot that receives signals according to the operation aspect of the remote operation levers and moves the actual machine operation levers based on the received signals. The mode switching mechanism <NUM> is formed of buttons or levers, and a remote operation mode and an actual machine operation mode are switched from one to the other in accordance, for example, with the operation of the buttons. The imaging apparatus <NUM> is installed, for example, in the cab <NUM> and captures an image of the environment containing at least part of the actuation mechanism <NUM> through a windshield of the cab <NUM>.

The actual machine output interface <NUM> comprises a notification output apparatus <NUM> and an actual machine wireless communication instrument <NUM>. The notification output apparatus <NUM> is formed of at least one of a light emitting apparatus, an image output apparatus, and a voice output apparatus, and outputs second notification requesting mode switching operation to the exterior of the work machine <NUM> or the interior of the cab <NUM> in a visually and/or aurally recognizable form. The actual machine wireless communication instrument <NUM> has not only a long-range wireless communication function for communication with a remote base station that communicates with the remote operation assistance server <NUM> but a short-range wireless communication function for communication with the portable terminal <NUM>. The term "short-range wireless communication" means wireless communication over a relatively short contact making distance (shorter than or equal to <NUM> to <NUM>, for example), such as NFC used for RFID and Bluetooth ("Bluetooth" is a registered trademark). The term "long-range wireless communication" means wireless communication for communication over a relatively long contact making distance, such as Wi-Fi ("Wi-Fi" is a registered trademark) and mobile phone communication lines.

The work attachment <NUM> as the actuation mechanism comprises a boom <NUM>, which is attached to the upper turner <NUM> so as to be movable up and down, an arm <NUM>, which is pivotably linked to the front end of the boom <NUM>, and a bucket <NUM>, which is pivotably linked to the front end of the arm <NUM>. The following components are attached to the work attachment <NUM>: the boom cylinder <NUM>, the arm cylinder <NUM>, and the bucket cylinder <NUM>, which are each formed of an extendable and retractable hydraulic cylinder.

The boom cylinder <NUM> is interposed between the boom <NUM> and the upper turner <NUM> so as to extend or retract when receiving supplied hydraulic fluid to cause the boom <NUM> to pivot in the up-and-down direction. The arm cylinder <NUM> is interposed between the arm <NUM> and the boom <NUM> so as to extend or retract when receiving supplied hydraulic fluid to cause the arm <NUM> to pivot around a horizontal axis relative to the boom <NUM>. The bucket cylinder <NUM> is interposed between the bucket <NUM> and the arm <NUM> so as to extend or retract when receiving supplied hydraulic fluid to cause the bucket <NUM> to pivot around a horizontal axis relative to the arm <NUM>.

The portable terminal <NUM> is a terminal apparatus that can be carried by a worker, such as a smartphone or a tablet terminal, and comprises a control apparatus <NUM>, a portable input interface <NUM>, and a portable output interface <NUM>. The control apparatus <NUM> is formed of an arithmetic processing apparatus (single-core processor, multi-core processor, or processor core that form one of processors), reads necessary data and software from a memory or any other storage, and performs the arithmetic processing according to the software on the data.

The portable input interface <NUM> is formed, for example, of touch-panel-type buttons and switches. The portable output interface <NUM> comprises an image output apparatus <NUM> and a wireless communication instrument <NUM>. The image output apparatus <NUM> comprises a voice output apparatus as required and outputs first notification requesting the mode switching operation and third notification requesting separation from the work machine <NUM> in a visually and/or aurally recognizable form. The wireless communication instrument <NUM> has not only the long-range wireless communication function for communication with a remote base station that communicates with the remote operation assistance server <NUM> but a short-range wireless communication function for communication with the work machine <NUM>.

The functions of the remote operation assistance system having the configuration described above will be described with reference to the flowcharts shown in <FIG> and <FIG>. In the flowcharts, the block "C•" is used to simplify the illustration, means transmission and/or receipt of data, specifically, means a conditional branch where a branch direction process is carried out conditional on the transmission and/or receipt of the data.

A state in which each component in the present invention (arithmetic processing resource or hardware resource) "recognizes" information is a concept that encompasses processing for preparing the information in any form usable in subsequent processing, such as receiving the information, reading or retrieving the information from a storage or any other component, writing (storing and holding) or registering the information in the storage or any other component, and estimating, evaluating, identifying, measuring, predicting, or otherwise processing the information by performing arithmetic processing on a signal outputted from a sensor and/or received, retrieved or otherwise acquired basic information in accordance with a predetermined algorithm.

First, the remote operation apparatus <NUM> evaluates whether or not the remote operator OP1 has performed first specifying operation via the remote input interface <NUM> (STEP <NUM> in <FIG>). The "first specifying operation" is, for example, operation performed on a button or an operation lever that forms the remote input interface <NUM> or the remote operation mechanism <NUM>. When the result of the evaluation is negative (NO in STEP <NUM> in <FIG>), the evaluation of whether or not the first specifying operation has been performed and the following processes are repeated. On the other hand, when the result of the evaluation is affirmative (YES in STEP <NUM> in <FIG>), a status checking request is transmitted to the remote operation assistance server <NUM> via the remote wireless communication instrument <NUM> (STEP <NUM> in <FIG>).

In response to the request, when the status checking request is received by the remote operation assistance server <NUM>, the status checking request (or first evaluation result request) is transmitted to the work machine <NUM> (C01 in <FIG>).

When the status checking request is received by the work machine <NUM> via the actual machine wireless communication instrument <NUM> (C411 in <FIG>), a first evaluation process is carried out by the actual machine control apparatus <NUM> (STEP <NUM> in <FIG>). The first evaluation process is the process of evaluating whether or not the operation mode has been switched to the remote operation mode via the mode switching mechanism <NUM>. The first evaluation result is transmitted to the remote operation assistance server <NUM> via the actual machine wireless communication instrument <NUM> (STEP <NUM> in <FIG>).

When the first evaluation result is received by the remote operation assistance server <NUM> (C11 in <FIG>), the first evaluation result is transmitted to the remote operation apparatus <NUM> (STEP <NUM> <FIG>). The evaluation result recognition element <NUM> evaluates whether the first evaluation result is affirmative (remote operation mode) or negative (actual machine operation mode) (STEP <NUM> in <FIG>).

When the first evaluation result is affirmative (YES in STEP <NUM> in <FIG>), a first flag f1 is set at "<NUM>" (STEP <NUM> in <FIG>). On the other hand, when the first evaluation result is negative (NO in STEP <NUM> in <FIG>), the first flag f1 is set at "<NUM>" (STEP <NUM> in <FIG>).

When the first evaluation result is received by the remote operation apparatus <NUM> via the remote wireless communication instrument <NUM> (C21 in <FIG>), the first evaluation result is outputted to the image output apparatus <NUM>, which forms the remote output interface <NUM> (STEP <NUM> in <FIG>). The remote operator OP <NUM> of the remote operation apparatus <NUM> can therefore recognize whether the work machine <NUM> has been set to the remote operation mode or the actual machine operation mode.

The remote operation apparatus <NUM> evaluates whether or not the remote operator OP1 has performed second specifying operation via the remote input interface <NUM> (STEP <NUM> in <FIG>). The "second specifying operation" may, for example, be operation performed on a button or a control lever that forms the remote input interface <NUM> or the remote operation mechanism <NUM>, and may be operation in the same form of the first specifying operation or different therefrom. When the result of the evaluation is negative (NO in STEP <NUM> in <FIG>), the evaluation of whether or not the first specifying operation has been performed and the following processes are repeated. On the other hand, when the result of the evaluation is affirmative (YES in STEP <NUM> in <FIG>), a remote operation request is transmitted to the remote operation assistance server <NUM> via the remote wireless communication instrument <NUM> (STEP <NUM> in <FIG>).

In response to the request, when the remote operation request is received by the remote operation assistance server <NUM>, the remote operation request (or second evaluation result request) is transmitted to the work machine <NUM> (C02 in <FIG>).

When the remote operation request is received by the work machine <NUM> via the actual machine wireless communication instrument <NUM> (C412 in <FIG>), a second evaluation process is carried out by the actual machine control apparatus <NUM> (STEP <NUM> in <FIG>). The second evaluation process is the process of evaluating whether or not the short-range wireless communication between the work machine <NUM> and the portable terminal <NUM> has been established. The second evaluation result is transmitted to the remote operation assistance server <NUM> via the actual machine wireless communication instrument <NUM> (STEP <NUM> in <FIG>).

When the second evaluation result is received by the remote operation assistance server <NUM> (C12 in <FIG>), the second evaluation result is transmitted to the remote operation apparatus <NUM> (STEP <NUM> in <FIG>). The evaluation result recognition element <NUM> evaluates whether the second evaluation result is affirmative (whether short-range wireless communication between work machine <NUM> and portable terminal <NUM> has been established), or whether the second evaluation result is negative (short-range wireless communication between work machine <NUM> and portable terminal <NUM> has not been established) (STEP <NUM> in <FIG>).

The state in which the short-range wireless communication between the work machine <NUM> and the portable terminal <NUM> has been established indicates that it is highly probable that the actual machine operator OP2 (worker) carrying the portable terminal <NUM> having been powered on and capable of wireless communication is present in an area S, where the short-range wireless communication with respect to the work machine <NUM> can be performed, as shown in <FIG>. The state in which the short-range wireless communication between the work machine <NUM> and the portable terminal <NUM> has not been established indicates that it is highly probable that the actual machine operator OP2 (worker) carrying the portable terminal <NUM> having been powered on and capable of wireless communication is located in a position out of the area S, where the short-range wireless communication with respect to the work machine <NUM> can be performed, as shown in <FIG>.

When the second evaluation result is affirmative (YES in STEP <NUM> in <FIG>), a second flag f2 is set at "<NUM>" (STEP <NUM> in <FIG>). On the other hand, when the second evaluation result is negative (NO in STEP <NUM> in <FIG>), the second flag f2 is set at "<NUM>" (STEP <NUM> in <FIG>).

When the second evaluation result is received by the remote operation apparatus <NUM> via the remote wireless communication instrument <NUM> (C22 in <FIG>), the second evaluation result is outputted to the image output apparatus <NUM>, which forms the remote output interface <NUM> (STEP <NUM> in <FIG>). The remote operator OP1 of the remote operation apparatus <NUM> can thus recognize whether or not the short-range wireless communication between the work machine <NUM> and the portable terminal <NUM> has been established, and in turn whether or not a worker carrying the portable terminal <NUM>, such as the actual machine operator OP2, is close enough to the work machine <NUM> to allow establishment of the short-range wireless communication.

In the work machine <NUM>, the actual machine control apparatus <NUM> acquires a captured image via the imaging apparatus <NUM> (STEP <NUM> in <FIG>). The actual machine control apparatus <NUM> transmits captured image data representing the captured image to the remote operation assistance server <NUM> via the actual machine wireless communication instrument <NUM> (STEP <NUM> in <FIG>).

When the captured image data is received by the remote operation assistance server <NUM> (C13 in <FIG>), environment image data according to the captured image data (data representing all or part of captured image itself or simulated environment image generated based on captured image) is transmitted to the remote operation apparatus <NUM> (STEP <NUM> in <FIG>).

When the environment image data is received by the remote operation apparatus <NUM> via the remote wireless communication instrument <NUM> (C23 in <FIG>), an environmental image according to the environment image data is outputted to the image output apparatus <NUM> (STEP <NUM> in <FIG>). The environment image, which contains the boom <NUM>, the arm <NUM>, the bucket <NUM>, and the arm cylinder <NUM>, which are part of the work attachment <NUM> as the actuation mechanism, is thus displayed, for example, on each of the image output apparatus <NUM> and the second image output apparatus <NUM>.

The remote operation assistance server <NUM> evaluates the combination of the first flag f1 and the second flag f2 (f1, f2) according to the first and second evaluation results recognized by the evaluation result recognition element <NUM> (STEP <NUM> in <FIG>).

When (f1, f2) = (<NUM>, <NUM>) (Q4 in STEP <NUM> in <FIG>), a "first assistance process" is carried out by the first assistance processing element <NUM> (C14 in <FIG>). The case described above corresponds to a case where the first evaluation result and the second evaluation result recognized by the evaluation result recognition element <NUM> are affirmative and negative, respectively. The "first assistance process" is a control process of permitting remote operation of the work machine <NUM> using the remote operation apparatus <NUM> based on the communication with each of the remote operation apparatus <NUM> and the work machine <NUM>.

Specifically, in the remote operation apparatus <NUM>, the remote control apparatus <NUM> recognizes the operation aspect of the remote operation mechanism <NUM> (STEP <NUM> in <FIG>), and a remote operation command according to the operation aspect is transmitted to the remote operation assistance server <NUM> via the remote wireless communication instrument <NUM> (STEP <NUM> in <FIG>).

When the remote operation command is received by the remote operation assistance server <NUM>, the remote operation command is transmitted to the work machine <NUM> by the first assistance processing element <NUM> (C14 in <FIG>).

In the work machine <NUM>, when an operation command is received by the actual machine control apparatus <NUM> via the actual machine wireless communication instrument <NUM> (C44 in <FIG>), the action of the work attachment <NUM> and other components is controlled (STEP <NUM> in <FIG>). For example, the bucket <NUM> is used to scoop soil in front of the work machine <NUM>, the upper turner <NUM> is turned, and then the bucket <NUM> is used to drop the soil.

When (f1, f2) = (<NUM>, <NUM>) (Q3 in STEP <NUM> in <FIG>), a "second assistance process" is carried out by the second assistance processing element <NUM> (STEP <NUM> in <FIG>). In this case, both the first and second evaluation results are affirmative. The "second assistance process" is a control process of prohibiting remote operation of the work machine <NUM> using the remote operation apparatus <NUM>. For example, based on the communication with the remote operation apparatus <NUM>, the second assistance process prohibits the recognition of the operation aspect of the remote operation mechanism <NUM> (see STEP <NUM> in <FIG>) or the transmission of the operation command (see STEP <NUM> in <FIG>). Instead, the second assistance process prohibits the remote operation assistance server <NUM> from receiving the operation command or the transmission of the operation command to the work machine <NUM> (see C14 in <FIG>).

In this case, a third notification command is transmitted by the first assistance processing element <NUM> to the portable terminal <NUM> having established the short-range wireless communication with the work machine <NUM> (STEP <NUM> in <FIG>). The transmission of the third notification command corresponds to execution of a "preliminary assistance process". When the third notification command is received by the portable terminal <NUM> via the wireless communication instrument <NUM> (C62 in <FIG>), the third notification requesting separation from the work machine <NUM> is outputted in a visually and/or aurally recognizable form from the image output apparatus <NUM>, which forms the portable output interface <NUM>, and/or a voice output apparatus.

When (f1, f2) =(<NUM>, <NUM>) (Q1 in STEP <NUM> in <FIG>), a first notification command is transmitted by the first assistance processing element <NUM> to the portable terminal <NUM> having established the short-range wireless communication with the work machine <NUM> (STEP <NUM> in <FIG>). When the first notification command is received by the portable terminal <NUM> via the wireless communication instrument <NUM> (C61 in <FIG>), the first notification requesting switching the operation mode to the remote operation mode is outputted in a visually and/or aurally recognizable form for the actual machine operator OP2 close to the portable terminal <NUM> from the image output apparatus <NUM>, which forms the portable output interface <NUM>, and/or the voice output apparatus (STEP <NUM> in <FIG>).

When (f1, f2) =(<NUM>, <NUM>) (Q2 in STEP <NUM> in <FIG>), a second notification command is transmitted by the first assistance processing element <NUM> to the work machine <NUM> (STEP <NUM> in <FIG>). When the second notification command is received by the work machine <NUM> via the actual machine wireless communication instrument <NUM> (C422 in <FIG>), the second notification requesting switching the operation mode to the remote operation mode is outputted in a visually and/or aurally recognizable form for the actual machine operator OP2 outside the work machine <NUM> or inside the cab <NUM> from the notification output apparatus <NUM>, which forms the actual machine output interface <NUM> (STEP <NUM> in <FIG>).

According to the remote operation assistance system having the configuration described above and the remote operation assistance server <NUM>, which forms the remote operation assistance system, in a situation in which the first evaluation result is affirmative and the work machine <NUM> can be remotely operated via the remote operation apparatus <NUM>, and further in a situation in which the second evaluation result is negative and it is highly probable that a worker carrying the portable terminal <NUM>, such as the actual machine operator OP2, is so far away from the work machine <NUM> that the short-range wireless communication between the work machine <NUM> and the portable terminal <NUM> is not established (see <FIG>), remote operation of the work machine is permitted (see Q4 in STEP <NUM> -> C14 in <FIG>).

On the other hand, even in the situation in which the first evaluation result is affirmative and the work machine <NUM> can be remotely operated via the remote operation apparatus <NUM>, but in a situation in which the second evaluation result is affirmative and it is highly probable that a worker carrying the portable terminal <NUM>, such as the actual machine operator OP2, is so close to the work machine <NUM> or on the work machine <NUM> that the short-range wireless communication between the work machine <NUM> and the portable terminal <NUM> is established, remote operation of the work machine <NUM> is inhibited (see Q3 in STEP <NUM> -> C143 in <FIG>, and <FIG>).

Remote operation of the work machine <NUM> performed by the remote operator OP1 can thus be assisted while unexpected situations, such as contact between the work machine <NUM> and workers outside the work machine <NUM>, are avoided.

In a situation in which the remote operator OP1 has the intention to remotely operate the work machine <NUM>, but is prohibited from doing so because a worker such as the actual machine operator OP2 is close to the work machine <NUM>, remote operation of the work machine <NUM> may be permitted. Specifically, the "third notification (notification requesting separation from work machine)" is outputted via the portable output interface <NUM> of the portable terminal <NUM> in accordance with to the situation (see Q3 in STEP <NUM> → STEP <NUM> -> C62 -> STEP <NUM> in <FIG>).

A worker carrying the portable terminal <NUM>, such as the actual machine operator OP2, and close enough to the work machine <NUM> to allow the short-range wireless communication can thus be requested to move away from the work machine <NUM> (see <FIG>). Thereafter, when the worker moves away from the work machine <NUM>, the following state is achieved: the remote operator OP1 can remotely operate the work machine <NUM> operating in the remote operation mode (see <FIG>).

In a situation in which the remote operator OP1 has the intention to remotely operate the work machine <NUM>, but is inhibited from remotely operating the work machine <NUM> because the work machine <NUM> operates in the actual machine operation mode, remote operation of the work machine <NUM> may be permitted. Specifically, the "first notification (= notification requesting switching operation mode to remote operation mode)" is outputted via the portable output interface <NUM> of the portable terminal <NUM> in accordance with the situation (Q1 in STEP <NUM> -> STEP <NUM> -> C61 -> STEP <NUM> in <FIG>).

A worker carrying the portable terminal <NUM>, such as the actual machine operator OP2, and close enough to the work machine <NUM> to allow the short-range wireless communication can thus be requested to switch the operation mode to the remote operation mode (see <FIG>). Thereafter, when the worker switches the operation mode from the actual machine operation mode to the remote operation mode via the mode switching mechanism <NUM>, which forms the work machine <NUM>, the following state is achieved: the remote operator OP1 can remotely operate the work machine <NUM>.

In the situation in which the remote operator OP1 has the intention to remotely operate the work machine <NUM>, but is inhibited from remotely operating the work machine <NUM> because the work machine <NUM> operates in the actual machine operation mode, remote operation of the work machine <NUM> may be permitted. Specifically, the "second notification (= notification requesting switching operation mode to remote operation mode)" is outputted via the actual machine output interface <NUM> in accordance with the situation (Q2 in STEP <NUM> -> STEP <NUM> -> C422 -> STEP <NUM> in <FIG>).

A worker close enough to the work machine <NUM> to be capable of visual or aural recognition of the notification outputted via the actual machine output interface <NUM>, such as the actual machine operator OP2, can thus be requested to switch the operation mode to the remote operation mode (see <FIG>). It does not matter whether or not the worker carries the portable terminal <NUM>. Thereafter, when the worker switches the operation mode from the actual machine operation mode to the remote operation mode via the mode switching mechanism <NUM>, which forms the work machine <NUM>, the following state is achieved: the remote operator OP1 can remotely operate the work machine <NUM>.

In the embodiment described above, the remote operation assistance server <NUM> is formed of one or more servers separately from the remote operation apparatus <NUM>, the work machine <NUM>, and the portable terminal <NUM> (see <FIG>), and as another embodiment, the remote operation assistance server <NUM> may be a component of the remote operation apparatus <NUM>, the work machine <NUM>, or the portable terminal <NUM>. The elements <NUM>, <NUM>, and <NUM> of the remote operation assistance server <NUM> may each be a component of inter-communicable two or more of the remote operation apparatus <NUM>, the work machine <NUM>, and the portable terminal <NUM>.

The second assistance processing element <NUM> may cause, based on communication with the remote operation apparatus <NUM>, the remote output interface <NUM> of the remote operation apparatus <NUM> to output the state in which remote operation of the work machine <NUM> through operation of the remote operation mechanism <NUM> is prohibited.

The remote operation assistance server <NUM> having the configuration described above and other components allow the remote operator OP1 to recognize, via the remote output interface <NUM> of the remote operation apparatus <NUM>, that remote operation of the work machine <NUM> via the remote operation apparatus <NUM> is prohibited.

The first assistance processing element <NUM> may cause, based on communication with the remote operation apparatus <NUM>, the remote output interface <NUM> of the remote operation apparatus <NUM> to output the state in which remote operation of the work machine <NUM> through operation of the remote operation mechanism <NUM> is permitted and may carry out the first assistance process on the precondition that specifying operation has been issued via the remote input interface <NUM> of the remote operation apparatus <NUM>.

The remote operation assistance server <NUM> having the configuration described above and other components allow the remote operator to recognize, via the remote output interface <NUM> of the remote operation apparatus <NUM>, that remote operation of the work machine <NUM> via the remote operation apparatus <NUM> is permitted. Furthermore, depending on whether or not specifying operation has been issued via the remote input interface <NUM> of the remote operation apparatus <NUM>, and provided that the remote operator OP1's intention to remotely operate the work machine <NUM> is confirmed, remote operation of the work machine <NUM> is permitted.

Claim 1:
A remote operation assistance server (<NUM>) comprising:
an evaluation result recognition element (<NUM>) that communicates with a work machine (<NUM>) comprising an actual machine operation mechanism and a mode switching mechanism that switches an actual machine operation mode in which the work machine (<NUM>) is operated in accordance with an operation aspect of the actual machine operation mechanism to a remote operation mode in which the work machine (<NUM>) is operated in accordance with an operation aspect of a remote operation mechanism that forms a remote operation apparatus (<NUM>) and vice versa, and recognizes as a first evaluation result a result of evaluation of whether or not the mode switching mechanism has switched the operation mode to the remote operation mode, based on the communication, characterized in that the evaluation result recognition element (<NUM>) further recognizes as a second evaluation result a result of evaluation of whether or not short-range wireless communication between the work machine (<NUM>) and a portable terminal (<NUM>) has been established, based on the communication; and in that
the remote operation assistance server (<NUM>) further comprises:
a first assistance processing element (<NUM>) that carries out a first assistance process of permitting remote operation of the work machine (<NUM>) using the remote operation apparatus (<NUM>) based on communication with each of the remote operation apparatus (<NUM>) and the work machine (<NUM>) when the first evaluation result recognized by the evaluation result recognition element (<NUM>) is affirmative and the second evaluation result recognized by the evaluation result recognition element (<NUM>) is negative; and
a second assistance processing element (<NUM>) that carries out a second assistance process of prohibiting remote operation of the work machine (<NUM>) using the remote operation apparatus (<NUM>) when the first and second evaluation results recognized by the evaluation result recognition element (<NUM>) are affirmative.