Patent Description:
In a case where a work machine is remotely operated with a remote operation apparatus, it is necessary to match a setting of the remote operation apparatus with the work machine of a target of the remote operation.

In <CIT>, a correction value is registered to match operating characteristics of an actual machine operation mechanism such as an actual machine operation lever included in a work machine with operating characteristics of a remote operation mechanism such as a remote operation lever included in a remote operation apparatus configured to remotely operate the work machine, and correction control to operate the work machine according to an operation amount inputted into the remote operation mechanism is executed by correcting the operation amount inputted into the remote operation mechanism based on the correction value and transmitting the operation amount to the actual machine operation mechanism. The correction value is associated with a combination of type information of the work machine and type information of the remote operation mechanism.

Here, an actual machine operation mechanism that is a target of a remote operation by a remote operation apparatus may be changed. For example, actual machine operation mechanisms remotely operably mounted on a work machine that is the target of the remote operation by the remote operation apparatus are increased or decreased by remodeling. Also, in a case where the remote operation apparatus switches the work machine that is the target of the remote operation from a first work machine to a second work machine, an actual machine operation mechanism provided in the first work machine might not correspond to an actual machine operation mechanism provided in the second work machine. When the actual machine operation mechanism is changed, the registered correction value might not be applied, and it is necessary to acquire and register a new correction value.

Furthermore, a remote operation mechanism such as a remote operation lever included in the remote operation apparatus may be changed. For example, an operation pattern of the remote operation mechanism is changed. In this case, the remote operation mechanism corresponding to the actual machine operation mechanism is changed. Therefore, the registered correction value might not be applied, and it is necessary to acquire and register a new correction value.

Therefore, it is assumed that processing of securely matching the setting of the remote operation apparatus with the work machine of the target of the remote operation is executed by confirming and registering the correction value at start of a remote operation.

However, if the processing is executed every time communication between the remote operation apparatus and the work machine is established, the processing is executed even in a case where the actual machine operation mechanism or the remote operation mechanism is not changed. Consequently, the remote operation of the work machine cannot be started until the processing ends.

To solve the problem, an object of the present invention is to provide a server or the like which can achieve improvement in remote operation efficiency of a work machine by improving an efficiency of processing of matching a setting of a remote operation apparatus with the work machine of a target of the remote operation.

A remote operation support system of the present invention is a remote operation support system for supporting a remote operation of a work machine by a remote operation apparatus, the remote operation support system comprising a first support processing element which recognizes a determination result of whether a remote operation factor defining an operation setting of a remote operation mechanism included in the remote operation apparatus corresponds to an actual machine operation factor defining an operation setting of an actual machine operation mechanism included in the work machine, and a second support processing element which executes processing of making the actual machine operation factor correspond to the remote operation factor, based on communication with at least one of the remote operation apparatus and the work machine, on a requirement that the determination result recognized by the first support processing element is negative.

According to the remote operation support system with the above configuration, in a case where the remote operation factor does not correspond to the actual machine operation factor, processing of making an operation method of the remote operation mechanism included in the remote operation apparatus correspond to an operation method of the actual machine operation mechanism included in the work machine is executed. Consequently, the remote operation of the work machine may be started by the remote operation apparatus, without executing the processing of making both operation factors correspond in a case where both the factors are matched. Therefore, improvement in remote operation efficiency of the work machine can be achieved by improving an efficiency of the processing.

A remote operation supporting complex system shown in <FIG> includes a remote operation support server <NUM>, a remote operation apparatus <NUM> and/or a work machine <NUM>. The remote operation support server <NUM>, the remote operation apparatus <NUM> and the work machine <NUM> are configured to communicate with each other via a network. A mutual communication network between the remote operation support server <NUM> and the remote operation apparatus <NUM> may be the same as or different from a mutual communication network between the remote operation support server <NUM> and the work machine <NUM>.

To "recognize" information by constitutional elements of the present invention is a concept including any arithmetic processing for making the information available in subsequent arithmetic processing, such as receiving the information, reading the information from a storage device, searching the information from a database, measuring the information, determining, judging, presuming or predicting the information based on basic information received or obtained otherwise, and storing the information in the storage device.

The remote operation support server <NUM> included in a remote operation support system comprises a database <NUM>, a first support processing element <NUM>, and a second support processing element <NUM>. The database <NUM> stores and holds captured image data and the like. The database <NUM> may be constituted of a database server separate from the remote operation support server <NUM>. Each support processing element is constituted of an arithmetic processing device (a single core processor or a multicore processor or a processor core included in the multicore processor), and reads required data and software from a storage device such as a memory, and executes after-mentioned arithmetic processing of the data as a target in accordance with the software.

The remote operation apparatus <NUM> comprises a remote control device <NUM>, a remote input interface <NUM>, and a remote output interface <NUM>. The remote control device <NUM> is constituted of an arithmetic processing device (a single core processor or a multicore processor or a processor core included in the multicore processor), and reads required data and software from a storage device such as a memory, and executes arithmetic processing of the data as a target in accordance with the software.

The remote input interface <NUM> comprises a remote operation mechanism <NUM>. The remote output interface <NUM> comprises an image output device <NUM>, an acoustic output device <NUM>, and remote wireless communication equipment <NUM>.

The remote operation mechanism <NUM> includes a running operation device, a pivoting operation device, a boom operation device, an arm operation device, and a bucket operation device. Each operation device includes an operation lever receiving a rotating operation. The operation lever (running lever) of the running operation device is operated to move a lower running body <NUM> of the work machine <NUM>. The running lever may also serve as a running pedal. For example, a running pedal fixed to a base part or lower end of the running lever may be provided. The operation lever (pivot lever) of the pivot operation device is operated to move a hydraulic pivot motor included in a pivot mechanism <NUM> of the work machine <NUM>. The operation lever (boom lever) of the boom operation device is operated to move a boom cylinder <NUM> of the work machine <NUM>. The operation lever (arm lever) of the arm operation device is operated to move an arm cylinder <NUM> of the work machine <NUM>. The operation lever (bucket lever) of the bucket operation device is operated to move a bucket cylinder <NUM> of the work machine <NUM>.

The respective operation levers included in the remote operation mechanism <NUM> are arranged around a seat St on which an operator sits, for example, as shown in <FIG>. The seat St has a form of a high back chair with arm rests, but may be a sitting part of any form on which the operator can sit, such as a form of low back chair with no head rest, or a form of a chair with no back rest.

In front of the seat St, a pair of left and right running levers <NUM> corresponding to left and right crawlers are arranged on left and right side by side. One operation lever may serve as a plurality of operation levers. For example, a left operation lever <NUM> provided in front of a left frame of the seat St shown in <FIG> may function as the arm lever when operated in a front-rear direction, and function as the pivot lever when operated in a left-right direction. Similarly, a right operation lever <NUM> provided in front of a right frame of the seat St shown in <FIG> may function as the boom lever when operated in the front-rear direction, and function as the bucket lever when operated in the left-right direction. The lever pattern may be arbitrarily changed in response to an operation instruction of the operator.

For example, as shown in <FIG>, the image output device <NUM> is constituted of a central image output device <NUM>, a left image output device <NUM> and a right image output device <NUM> arranged forward, diagonally forward left and diagonally forward right of the seat St, each device having a substantially rectangular screen. The screen (image display region) of each of the central image output device <NUM>, the left image output device <NUM> and the right image output device <NUM> may have the same shape and size or different shapes and sizes.

As shown in <FIG>, a right edge of the left image output device <NUM> is adjacent to a left edge of the central image output device <NUM> in such a manner that the screen of the central image output device <NUM> and the screen of the left image output device <NUM> form an inclination angle θ1 (e.g., <NUM>° ≤ θ1 ≤ <NUM>°). As shown in <FIG>, a left edge of the right image output device <NUM> is adjacent to a right edge of the central image output device <NUM> in such a manner that the screen of the central image output device <NUM> and the screen of the right image output device <NUM> form an inclination angle θ2 (e.g., <NUM>° ≤ θ2 ≤ <NUM>°). The inclination angles θ1 and θ2 may be the same or different.

The respective screens of the central image output device <NUM>, the left image output device <NUM> and the right image output device <NUM> may be parallel to a vertical direction, or inclined to the vertical direction. At least one image output device of the central image output device <NUM>, the left image output device <NUM> and the right image output device <NUM> may be constituted of a plurality of divided image output devices. For example, the central image output device <NUM> may be constituted of a pair of image output devices each including a substantially rectangular screen, the devices being adjacent to each other in an up-down direction.

The acoustic output device <NUM> is constituted of one or more speakers, and constituted of a central acoustic output device <NUM>, a left acoustic output device <NUM> and a right acoustic output device <NUM> arranged behind the seat St, respectively in a left arm rest rear part and a right arm rest rear part, for example, as shown in <FIG>. The central acoustic output device <NUM>, the left acoustic output device <NUM> and the right acoustic output device <NUM> may have the same specification or different specifications.

The work machine <NUM> comprises an actual machine control device <NUM>, an actual machine input interface <NUM>, an actual machine output interface <NUM>, and a work mechanism <NUM>. The actual machine control device <NUM> is constituted of an arithmetic processing device (a single core processor or a multicore processor or a processor core included in the multicore processor), and reads required data and software from a storage device such as a memory, and executes arithmetic processing of the data as a target in accordance with the software.

The work machine <NUM> is, for example, a crawler excavator (construction machine), and comprises a crawler type lower running body <NUM>, and an upper pivot body <NUM> pivotally mounted on the lower running body <NUM> via the pivot mechanism <NUM>, as shown in <FIG>. The upper pivot body <NUM> has a front left part provided with a cab <NUM> (driver cab). The upper pivot body <NUM> has a front central part provided with the work mechanism <NUM>.

The actual machine input interface <NUM> includes an actual machine operation mechanism <NUM>, an actual machine image capturing device <NUM>, and a positioning device <NUM>. The actual machine operation mechanism <NUM> includes a plurality of operation levers arranged around a seat disposed in the cab <NUM> in the same manner as in the remote operation mechanism <NUM>. A drive mechanism or a robot which receives a signal corresponding to an operation mode of a remote operation lever and moves an actual machine operation lever based on the received signal is provided in the cab <NUM>. The actual machine image capturing device <NUM> is installed, for example, in the cab <NUM>, and captures an image of an environment including at least a part of the work mechanism <NUM> through a front window and a pair of left and right side windows. The front window and side windows may be partially or entirely omitted. The positioning device <NUM> is constituted of a GPS, a gyro sensor and the like as required.

The actual machine output interface <NUM> comprises actual machine wireless communication equipment <NUM>.

As shown in <FIG>, the work mechanism <NUM> as a work mechanism comprises a boom <NUM> risably and lowerably mounted on the upper pivot body <NUM>, an arm <NUM> rotatably coupled to a tip of the boom <NUM>, and a bucket <NUM> rotatably coupled to a tip of the arm <NUM>. In the work mechanism <NUM>, the boom cylinder <NUM>, the arm cylinder <NUM> and the bucket cylinder <NUM> are mounted, each of which is constituted of an expandable and contractible hydraulic cylinder.

The boom cylinder <NUM> is interposed between the boom <NUM> and the upper pivot body <NUM> in such a manner that the boom cylinder receives supply of hydraulic oil and expands and contracts to rotate the boom <NUM> in a rising and lowering direction. The arm cylinder <NUM> is interposed between the arm <NUM> and the boom <NUM> in such a manner that the arm cylinder receives supply of hydraulic oil and expands and contracts to rotate the arm <NUM> relative to the boom <NUM> around a horizontal axis. The bucket cylinder <NUM> is interposed between the bucket <NUM> and the arm <NUM> in such a manner that the bucket cylinder receives supply of hydraulic oil and expands and contracts to rotate the bucket <NUM> relative to the arm <NUM> around the horizontal axis.

<FIG> is a flowchart explaining a first function as a basic function of the remote operation support system with the above configuration. In the flowchart, each block denoted with a reference sign starting with "C" is used for simplicity of description, and the block means transmission and/or reception of data, and means conditional branch to execute processing in a branch direction on a condition of the transmission and/or reception of the data. The received data is stored in the storage device constituted of the database <NUM> and/or a nonvolatile or volatile memory. This also applies to an after-mentioned flowchart.

In the remote operation apparatus <NUM>, it is determined whether a designating operation through the remote input interface <NUM> by the operator is present (<FIG>/STEP210). "The designating operation" is, for example, a tapping operation in the remote input interface <NUM> to designate the work machine <NUM> intended to be remotely operated by the operator. If the determination result is negative (<FIG>/NO in STEP210), a series of processing ends. On the other hand, if the determination result is positive (<FIG>/YES in STEP210), an environment confirmation request is transmitted to the remote operation support server <NUM> through the remote wireless communication equipment <NUM> (<FIG>/STEP212).

In the remote operation support server <NUM>, in a case where the environment confirmation request is received, the first support processing element <NUM> transmits the environment confirmation request to the corresponding work machine <NUM> (<FIG>/C110).

In the work machine <NUM>, in a case where the environment confirmation request is received through the actual machine wireless communication equipment <NUM> (<FIG>/C410), the actual machine control device <NUM> acquires a captured image through the actual machine image capturing device <NUM> (<FIG>/STEP410). Here, image processing may be executed by the actual machine control device <NUM> or an image processing device included in this actual machine control device. Captured image data subjected to the image processing is transmitted to the remote operation support server <NUM> through the actual machine wireless communication equipment <NUM> by the actual machine control device <NUM> (<FIG>/STEP412).

In the remote operation support server <NUM>, in a case where the captured image data is received by the first support processing element <NUM> (<FIG>/C112), environment image data corresponding to the captured image is transmitted to the remote operation apparatus <NUM> by the second support processing element <NUM> (<FIG>/STEP110). The environment image data is captured image data itself, and is additionally image data representing a simulated environment image generated based on the captured image. In a case where an image processing device <NUM> is constituted of the remote operation support server <NUM>, the environment image data may be generated by subjecting the captured image data to image processing by the image processing device <NUM>.

In the remote operation apparatus <NUM>, in a case where the environment image data is received through the remote wireless communication equipment <NUM> (<FIG>/C210), an environment image corresponding to the environment image data is outputted to the image output device <NUM> by the remote control device <NUM> (<FIG>/STEP214).

Consequently, for example, as shown in <FIG>, in the environment image outputted by the image output device <NUM>, the boom <NUM> and the arm <NUM> which are parts of the work mechanism <NUM> and a pile of rubble or earth and sand (object of a work by the bucket <NUM>) are reflected in front of the cab <NUM> through a window frame constituted of a right window frame Q1, an upper window frame Q2, a left window frame Q3 and a lower window frame Q4 which define the cab <NUM>. The environment image may be generated in such a manner that at least parts of the window frames Q1 to Q4 are not reflected, by image processing of the captured image or by view angle adjustment of the actual machine image capturing device <NUM>. In a case where the actual machine image capturing device <NUM> is not provided in the cab <NUM> but is provided outside, a captured image in which constituent components of the cab <NUM>, such as the window frames Q1 to Q4, are not reflected, eventually the environment image can be acquired.

In the remote operation apparatus <NUM>, the remote control device <NUM> recognizes an operation mode of the remote operation mechanism <NUM> (<FIG>/STEP216), and a remote operation command corresponding to the operation mode is transmitted to the remote operation support server <NUM> through the remote wireless communication equipment <NUM> (<FIG>/STEP218).

In the remote operation support server <NUM>, in a case where the remote operation command is received by the second support processing element <NUM>, the remote operation command is transmitted to the work machine <NUM> by the first support processing element <NUM> (<FIG>/C114).

In the work machine <NUM>, in a case where the operation command is received through the actual machine wireless communication equipment <NUM> by the actual machine control device <NUM> (<FIG>/C412), an operation of the work mechanism <NUM> or the like is controlled (<FIG>/STEP414). For example, a work of scooping soil in front of the work machine <NUM> with the bucket <NUM>, pivoting the upper pivot body <NUM> and dropping the soil from the bucket <NUM> is executed.

<FIG> is a flowchart explaining a first embodiment of a second function as a calibration function of the remote operation support system with the above configuration.

In the remote operation apparatus <NUM>, the remote control device <NUM> transmits a remote operation factor together with an actual machine identifier to the remote operation support server <NUM> through the remote wireless communication equipment <NUM> (<FIG>/STEP220). The remote operation factor and the actual machine identifier may be transmitted from the remote operation apparatus <NUM> depending on determination that a trigger operation (input of the actual machine identifier or designation of the remote operation mode) in the remote input interface <NUM> is present. "The remote operation factor" is a factor which defines a remote operation setting of the remote operation mechanism <NUM> included in the remote operation apparatus <NUM>. "The remote operation factor" corresponds to the actual machine operation mechanism <NUM> included in the work machine <NUM> intended to be remotely operated by the remote operation mechanism <NUM>. Table <NUM> illustrates such correspondence.

As shown in Table <NUM>, in a case where the actual machine operation mechanism <NUM> included in the work machine <NUM> intended to be remotely operated by the remote operation mechanism <NUM> includes the boom operation lever, the arm operation lever, the bucket operation lever and the pivot operation lever, a remote operation setting A1 is set. In the remote operation setting A1, a remote operation generated by tilting the left and right operation levers is set, but a remote operation generated by tilting the running lever and an optional pedal is not set. In a case of intending the running of the work machine <NUM> including the operation of the lower running body <NUM> through the operation of the remote operation mechanism <NUM>, a remote operation setting A2 is set, and the remote operation generated by tilting the running lever is set. In a case where an optional device such as a grappler is intended to be operated by the remote operation mechanism <NUM>, a remote operation setting A3 is set, and a remote operation generated by tilting the OPT pedal is set. A remote operation setting other than the remote operation settings A1 to A3 may be set, and corresponds to, for example, changing of a setting of an operation pattern. "The actual machine identifier" is an identifier or data for identifying or specifying the work machine <NUM> that is a target of the remote operation of the remote operation apparatus <NUM>.

In the remote operation support server <NUM>, the first support processing element <NUM> receives the remote operation factor and actual machine identifier (<FIG>/C120), and an actual machine operation factor acquisition request is transmitted to the work machine <NUM> identified by the actual machine identifier (<FIG>/STEP220).

In the corresponding work machine <NUM>, upon receiving the actual machine operation factor acquisition request through the actual machine wireless communication equipment <NUM> by the actual machine control device <NUM> (<FIG>/C420), an actual machine operation factor is recognized by being read or searched from the storage device or the database (<FIG>/STEP420). Then, the actual machine control device <NUM> transmits the actual machine operation factor and actual machine identifier to the remote operation support server <NUM> through the actual machine wireless communication equipment <NUM> (<FIG>/STEP422). "The actual machine operation factor" is a factor which defines a setting of an actual machine operation of the actual machine operation mechanism <NUM> included in the work machine <NUM>, and corresponds to the actual machine operation mechanism <NUM> capable of receiving the remote operation. Table <NUM> illustrates such correspondence.

As shown in Table <NUM>, in a case where the actual machine operation mechanism <NUM> included in the work machine <NUM> intended to be remotely operated by the remote operation mechanism <NUM> includes the boom operation lever, the arm operation lever, the bucket operation lever and the pivot operation lever, an actual machine operation setting B1 is set. In the actual machine operation setting B1, an actual machine operation mechanism marked with a circle (O) is operable when receiving the remote operation, and an actual machine operation mechanism marked with no circle(-) is not operable even when receiving the remote operation. In a case where the actual machine operation mechanism <NUM> can receive the remote operation of running, an actual machine operation setting B2 is set, and in a case where the actual machine operation mechanism <NUM> can receive the remote operation of an optional device such as the grappler, an actual machine operation setting B3 is set. The actual machine operation factor is set when the actual machine control device <NUM> of the work machine <NUM> recognizes a configuration of the actual machine operation mechanism <NUM> of the work machine <NUM>, or when a service person who maintains the work machine <NUM> inputs the factor.

In the remote operation support server <NUM>, the actual machine operation factor and actual machine identifier are received by the first support processing element <NUM> (<FIG>/C121), and further received by the remote operation apparatus <NUM> (<FIG>/C220).

In the remote operation apparatus <NUM>, upon receiving the actual machine operation factor and actual machine identifier (<FIG>/C220), it is determined whether the actual machine operation factor corresponds to the remote operation factor (see <FIG>/C120) associated with the actual machine identifier (<FIG>/STEP221).

For example, in a case where the actual machine operation factor is the operation setting B1, the actual machine operation mechanism <NUM> can receive the remote operations of the boom, arm, bucket and pivot, and in a case where the operation setting A1 is set as the remote operation factor to the remote operation mechanism <NUM>, both the factors correspond. Similarly, in a case where the operation setting A2 is set to the operation setting B2 and the operation setting A3 is set to the operation setting B3, both the factors correspond.

Even when both the factors correspond, both the factors do not correspond in a case where actual machine operation mechanisms remotely operably mounted in the work machine <NUM> are increased or decreased by remodeling. Also, even when both the factors correspond, both the factors do not correspond in a case where the remote operation apparatus <NUM> switches the work machine that is the target of the remote operation. Furthermore, in a case where the setting of the operation pattern is changed as to the remote operation mechanism <NUM>, both the factors do not correspond.

In a case where the determination result is positive (<FIG>/YES in STEP221), a first flag f1 is set to "<NUM>" (<FIG>/STEP222). On the other hand, in a case where the determination result is negative (<FIG>/NO in STEP221), the remote operation factor is changed to correspond to the actual machine identifier (<FIG>/STEP223), and the first flag f1 is set to "<NUM>" (<FIG>/STEP224). Specifically, in a case where the first flag f1 has a value of "<NUM>", it is meant that the remote operation factor and the actual machine operation factor correspond, and in a case where the first flag f1 has a value of "<NUM>", it is meant that the remote operation factor and the actual machine operation factor do not correspond.

Next, in the remote operation apparatus <NUM>, it is determined whether the operation pattern is changed by the remote operation mechanism <NUM> (<FIG>/STEP225). For example, when the operator who operates the remote operation apparatus is changed, the operation pattern may be changed. In a case where the operation pattern is changed (<FIG>/YES in STEP225), the operation to be inputted into the remote operation mechanism <NUM> changes, and hence an amount of an output of the actual machine operation mechanism which is to be corrected relative to an operation amount inputted into the remote operation mechanism <NUM> changes to operate the work machine <NUM> according to the operation amount inputted into the remote operation mechanism <NUM>.

In a case where the operation pattern is not changed (<FIG>/NO in STEP225), it is determined whether the value of the first flag f1 is "<NUM>" (<FIG>/STEP226). In a case where the value of the first flag f1 is "<NUM>" (<FIG>/YES in STEP226), the remote operation factor is changed and the operation to be inputted into the remote operation mechanism <NUM> is changed, and hence the amount of the output of the actual machine operation mechanism which is to be corrected relative to the operation amount inputted into the remote operation mechanism <NUM> changes to operate the work machine <NUM> according to the operation amount inputted into the remote operation mechanism <NUM>.

In a case where the value of the first flag f1 is not "<NUM>" (<FIG>/NO in STEP226), it is determined whether the actual machine operation factor is changed (<FIG>/STEP227). For example, the remotely operable actual machine operation mechanisms <NUM> mounted in the work machine <NUM> that is the target of the remote operation of the remote operation apparatus <NUM> are increased or decreased by remodeling, and hence the actual machine operation factor at the previous remote operation may not correspond to the actual machine operation factor at the current remote operation. Also, in a case where the remote operation apparatus <NUM> switches the work machine that is the target of the remote operation from a first work machine to a second work machine, the actual machine operation mechanism provided in the first work machine may be different from the actual machine operation mechanism provided in the second work machine. In this case, the actual machine operation factor does not correspond. If the actual machine operation factor does not correspond, the amount of the output of the actual machine operation mechanism <NUM> which is to be corrected relative to the operation amount inputted into the remote operation mechanism <NUM> changes to operate the work machine <NUM> according to the operation amount inputted into the remote operation mechanism <NUM>. In a case where it is determined that the actual machine operation factor is not changed (<FIG>/NO in STEP227), a series of processing ends.

In cases where the operation pattern is changed (<FIG>/YES in STEP225), where the value of the first flag f1 is "<NUM>" (<FIG>/YES in STEP226), and where the actual machine operation factor is changed (<FIG>/YES in STEP227), a calibration request to adjust the amount of the output of the actual machine operation mechanism which is to be corrected relative to the operation amount inputted into the remote operation mechanism <NUM> in the remote operation apparatus <NUM> is transmitted together with the actual machine identifier to the remote operation support server <NUM> (<FIG>/STEP228).

In a case where the remote operation support server <NUM> receives the calibration request and actual machine identifier (<FIG>/C122), a calibration command is transmitted to the work machine <NUM> that is the target of the remote operation of the remote operation apparatus <NUM> and to the remote operation apparatus <NUM> (<FIG>/STEP122).

In a case where the work machine <NUM> receives the calibration command (<FIG>/C421), calibration processing of the actual machine operation mechanism <NUM> is executed (<FIG>/STEP424), and in a case where the remote operation apparatus <NUM> receives the calibration command (<FIG>/C221), calibration processing of the remote operation mechanism <NUM> is executed (<FIG>/STEP229).

For example, on executing calibration processing of calibrating the operation amount of the actual machine operation lever included in the actual machine operation mechanism <NUM>, an operation of a lever drive actuator is controlled to position the actual machine operation lever to each of a neutral position and an operation position with a maximum operation amount, a control command value or position of the lever drive actuator at each operation position is stored, and data defining a correspondence between a drive command of the operation lever and the control command value of the lever drive actuator is generated by using the stored control command value by a lever drive controller. Consequently, the data is determined in such a manner that an operation state or the operation position of the actual machine operation lever included in the actual machine operation mechanism <NUM> in response to the drive command can be prevented from causing variations depending on the work machine <NUM> that is the target of the remote operation.

When executing the above processing of generating the data and then activating the lever drive actuator in response to the drive command received from a lever operation command unit, the lever drive controller executes activation control of the lever drive actuator by use of the control command value determined from the received drive command and the stored and held data. This can appropriately prevent occurrence of the variations of the operation state of the actual machine operation lever included in the actual machine operation mechanism <NUM> of the work machine <NUM> which is achieved depending on the operation of the remote operation mechanism <NUM> included in the remote operation apparatus <NUM>. For example, an operation of a first operation lever of a hydraulic work machine at a neutral position and/or an operation of the lever with a maximum operation amount can be appropriately performed by the operation of the operation apparatus irrespective of differences in type and various elements of the work machine <NUM> that is the target of the remote operation.

The operation lever is not limited to an operation unit for the operator to execute an operation with hands, and may be an operation unit (e.g., a pedal type of operation unit) for the operator to execute an operation with feet.

As shown by a solid line in <FIG>, data concerned with the calibration processing concerning the operation of the actual machine operation lever of the work machine <NUM> is generated as relational data indicating a relation between the drive command and the control command value in such a manner that the operation amount of the actual machine operation lever corresponding to the control command value linearly changes relative to the drive command in a range of the drive command from <NUM>% to +<NUM>% and a range of the drive command from <NUM>% to -<NUM>%. Here, <NUM>% indicates a command not to drive the lever, +<NUM>% indicates a command to maximumly drive the lever in a positive direction, and -<NUM>% indicates a command to maximumly drive the lever in a negative direction. <FIG> illustrates, with a double-dashed chain line, a relation between the drive command and the operation amount of the actual machine operation lever which is represented by relational data before execution of processing.

Consequently, the work machine <NUM> can be remotely operated in such a manner that an operation of a hydraulic actuator in response to the operation of the operation lever is similar.

As shown by a solid line in <FIG>, data concerned with the calibration processing concerning the operation of the remote operation lever included in the remote operation mechanism <NUM> of the remote operation apparatus <NUM> is generated as relational data prescribing a relation between the operation amount of the remote operation lever and the drive command in such a manner that the drive command linearly changes relative to change in operation amount (swing angle) of the remote operation lever in a range of the drive command from <NUM>% to +<NUM>% and a range of the drive command from <NUM>% to -<NUM>%. <FIG> illustrates, with a double-dashed chain line, a relation between the operation amount of the remote operation lever and the drive command which is represented by relational data before execution of processing.

Consequently, the work machine <NUM> can be remotely operated in such a manner that the operation of the hydraulic actuator in response to the operation of the remote operation lever included in the remote operation mechanism <NUM> is like the operation of the hydraulic actuator in response to the operation of the actual machine operation lever included in the actual machine operation mechanism <NUM>. As a result of the above processing, the work machine <NUM> can be in a state of being remotely operated through the remote operation apparatus <NUM> as described above (see <FIG>).

<FIG> is a flowchart explaining a second embodiment of the second function as the calibration function of the remote operation support system with the above configuration.

In the remote operation apparatus <NUM>, the remote operation factor is transmitted together with the actual machine identifier to the remote operation support server <NUM> through the remote wireless communication equipment <NUM> by the remote control device <NUM> (<FIG>/STEP240). The remote operation factor and actual machine identifier may be transmitted from the remote operation apparatus <NUM> depending on the determination that the trigger operation (input of the actual machine identifier or designation of the remote operation mode) in the remote input interface <NUM> is present.

In the work machine <NUM>, the remote operation factor and actual machine identifier are received by the first support processing element <NUM> (<FIG>/C140), and the remote operation factor is transmitted to the work machine <NUM> identified by the actual machine identifier (<FIG>/STEP240).

In the corresponding work machine <NUM>, upon receiving the remote operation factor through the actual machine wireless communication equipment <NUM> by the actual machine control device <NUM> (<FIG>/C440), the actual machine operation factor is recognized by being read or searched from the storage device or database (<FIG>/STEP440). Then, the actual machine control device <NUM> determines whether the actual machine operation factor corresponds to the remote operation factor associated with the actual machine identifier (<FIG>/STEP441).

In a case where the determination result is positive (<FIG>/YES in STEP441), a second flag f2 is set to "<NUM>" (<FIG>/STEP443). On the other hand, in a case where the determination result is negative (<FIG>/NO in STEP441), the second flag f2 is set to "<NUM>" (<FIG>/STEP444). Specifically, in a case where the second flag f2 has a value of "<NUM>", it is meant that the remote operation factor and the actual machine operation factor correspond, and in a case where the second flag f2 has a value of "<NUM>", it is meant that the remote operation factor and the actual machine operation factor do not correspond.

If the second flag f2 is set to "<NUM>" (<FIG>/STEP443) or the second flag f2 is set to "<NUM>" (<FIG>/STEP444), the actual machine control device <NUM> transmits the second flag f2 through the actual machine wireless communication equipment <NUM> to the remote operation support server <NUM> (<FIG>/STEP445).

In the remote operation support server <NUM>, upon receiving the second flag f2 by the first support processing element <NUM>, data representing the second flag f2 is transmitted to the remote operation apparatus <NUM> (<FIG>/C141).

In the remote operation apparatus <NUM>, upon receiving the data representing the value of the second flag f2 (<FIG>/C240), the second support processing element <NUM> determines whether the second flag f2 is "<NUM>" (<FIG>/STEP241).

In a case where the determination result is negative (<FIG>/NO in STEP241), the remote control device <NUM> outputs a message that encourages change of the remote operation factor, through the remote output interface <NUM> (<FIG>/STEP242). In a case where it is determined that the remote operation factor is not changed (<FIG>/NO in STEP243), the message is continuously outputted through the remote output interface <NUM> (<FIG>/STEP242). In a case where an output duration of the message is more than a designated time, a series of processing may end.

On the other hand, in a case where it is determined that the remote operation factor is changed (<FIG>/YES in STEP243), processing of or after the transmission of the remote operation factor is repeated.

In a case where the second support processing element <NUM> determines that the second flag f2 has the value of "<NUM>" (<FIG>/YES in STEP241), it is determined in the remote operation apparatus <NUM> whether the operation pattern is changed by the remote operation mechanism <NUM> (<FIG>/STEP244).

In a case where the operation pattern is not changed (<FIG>/NO in STEP244), it is determined whether the actual machine operation factor is changed (<FIG>/STEP245). In a case where it is determined that the actual machine operation factor is not changed (<FIG>/NO in STEP245), a series of processing ends.

In cases where the operation pattern is changed (<FIG>/YES in STEP244) and where the actual machine operation factor is changed (<FIG>/YES in STEP245), the calibration request to adjust the amount of the output of the actual machine operation mode which is to be corrected relative to the operation amount inputted into the remote operation mechanism <NUM> in the remote operation apparatus <NUM> is transmitted together with the actual machine identifier to the remote operation support server <NUM> (<FIG>/STEP246).

In a case where the remote operation support server <NUM> receives the calibration request and the actual machine identifier (<FIG>/C142), the calibration command is transmitted to the work machine <NUM> that is the target of the remote operation of the remote operation apparatus <NUM> and to the remote operation apparatus <NUM> (<FIG>/STEP140).

In a case where the work machine <NUM> receives the calibration command (<FIG>/C441), calibration processing is executed (<FIG>/STEP446), and in a case where the remote operation apparatus <NUM> receives the calibration command (<FIG>/C241), the calibration processing is executed (<FIG>/STEP247).

In the second embodiment, the message that encourages the operator to change the remote operation factor is outputted. Therefore, the operator can recognize the content of the remote operation factor.

According to the remote operation support system with the configuration, in the case where the remote operation factor and the actual machine operation factor do not correspond, processing is executed to make the remote operation factor of the remote operation mechanism <NUM> included in the remote operation apparatus <NUM> correspond to the actual machine operation factor of the actual machine operation mechanism <NUM> included in the work machine <NUM>. Consequently, in a case where both the factors correspond, the remote operation of the work machine <NUM> by the remote operation apparatus <NUM> may be started without waiting for execution completion of calibration processing. Therefore, efficiency of processing of matching the setting of the remote operation apparatus with the work machine of the target of the remote operation is achieved.

According to the remote operation support system with the configuration, the first support processing element <NUM> can determine whether the actual machine operation mechanism of the work machine <NUM> that is the target of the remote operation by the remote operation apparatus <NUM> is changed. Consequently, in a case where the actual machine operation mechanism is not changed, the remote operation apparatus starts the remote operation of the work machine without executing the processing of making both the operation factors correspond, so that efficiency can be achieved. On the other hand, in a case where the actual machine operation mechanism is changed, processing of securely making the factors correspond is executed.

According to the remote operation support system with the configuration, in a case of executing the processing of making both the operation factors correspond, processing of matching the operation characteristics of the actual machine operation mechanism with the operation characteristics of the remote operation mechanism of the remote operation apparatus <NUM> is performed. Consequently, even in a case where the operation factor is changed, operability equivalent to before the change can be maintained.

According to the remote operation support system with the configuration, in a case where at least one of the operation pattern and the operation factor is changed, processing of making the operation characteristics of the actual machine operation mechanism correspond to the operation characteristics of the remote operation mechanism of the remote operation apparatus <NUM> is performed. Consequently, even in a case where only the operation pattern is changed, the operability equivalent to before the change can be maintained.

At least part of functional elements of the remote operation support server <NUM> may be constituted of the remote operation apparatus <NUM> and/or the work machine <NUM>. For example, the first support processing element <NUM> may be constituted of the remote control device <NUM> and/or the actual machine control device <NUM> as a first arithmetic processing device. The second support processing element <NUM> may be constituted of the remote control device <NUM> and/or the actual machine control device <NUM> as a second arithmetic processing device. In a case where the functional element of the remote operation support server <NUM> is mounted in the remote operation apparatus <NUM>, information may be communicated by wired communication through wired network mounted in the remote operation apparatus <NUM> in place of the wireless communication in the above embodiment. Similarly, in a case where the functional element of the remote operation support server <NUM> is mounted in the work machine <NUM>, information may be communicated by wired communication through wired network mounted in the work machine <NUM> in place of the wireless communication in the above embodiment.

For example, concerning the first embodiment of the second function, the functional elements represented by <FIG>/C121 and STEP122 may be included in the remote control device <NUM>. The functional elements represented by <FIG>/C120, C121 and STEP122 may be included in the actual machine control device <NUM>. The functional elements represented by <FIG>/C122 may be included in the remote control device <NUM> or the actual machine control device <NUM>.

Also, concerning the second embodiment of the second function, the functional elements represented by <FIG>/C142 may be included in the remote control device <NUM> or the actual machine control device <NUM>.

Concerning the first embodiment of the second function, the recognition result of the actual machine operation factor by the work machine <NUM> may be sent to the remote operation support server <NUM> (<FIG>/STEP422), prior to the remote operation of the remote operation apparatus <NUM>. In this case, the recognition result of the actual machine operation factor of the work machine <NUM> is stored in the database <NUM> of the remote operation support server <NUM>. Consequently, as shown in <FIG>, in a case where an actual machine operation factor acquisition request is transmitted from the remote operation apparatus <NUM> (<FIG>/STEP220) and the request is received by the remote operation support server <NUM>, the actual machine operation factor stored in the database <NUM> is recognized without communication with the work machine <NUM> (<FIG>/STEP520), so that the actual machine operation factor can be transmitted to the remote operation apparatus <NUM> (<FIG>/STEP521).

Concerning the first embodiment of the second function, the message indicating the content of the remote operation factor may be outputted through the remote output interface <NUM>.

Concerning the second function, in the above embodiment, in cases where both the factors do not correspond, where the operation pattern is changed and where the actual machine operation factor is changed, the calibration request is made in the remote operation apparatus <NUM>, but a work efficiency may be improved by exceptionally making no calibration request even in a case where these conditions are satisfied. For example, in a case where there are histories of setting the work machine <NUM> to be remotely operated by the remote operation apparatus <NUM> and both the operation factors, and using the operation pattern in the past, data concerned with calibration processing at that time may be used.

In the remote operation support system of the present invention, it is preferable that the first support processing element recognizes the determination result of whether the remote operation factor depending on the actual machine operation mechanism set as a target of the remote operation by the remote operation mechanism corresponds to the actual machine operation factor depending on the actual machine operation mechanism included in the work machine that is the target of the remote operation of the remote operation mechanism.

According to the remote operation support system with the configuration, the first support processing element can determine whether the actual machine operation mechanism that is the target of the remote operation by the remote operation apparatus is changed. Consequently, in a case where the actual machine operation mechanism is not changed, the remote operation apparatus can start the remote operation of the work machine without executing the processing of making both the operation factors correspond. On the other hand, in a case where the actual machine operation mechanism is changed, the processing of securely making the operation factors correspond is executed.

Also, in the remote operation support system of the present invention, it is preferable that the second support processing element executes processing of acquiring a correction value to make operation characteristics of the actual machine operation mechanism included in the work machine correspond to the operation characteristics of the remote operation mechanism included in the remote operation apparatus configured to remotely operate the work machine, on the requirement that the determination result recognized by the first support processing element is negative.

According to the remote operation support system with the configuration, in a case of executing the processing of making both the operation factors correspond, the processing of matching the operation characteristics of the actual machine operation mechanism with the operation characteristics of the remote operation mechanism is further performed. Consequently, the operability equivalent to that before the change can be maintained even in the case where the operation factor is changed.

Also, in the remote operation support system of the present invention, it is preferable that the first support processing element recognizes an operation pattern in the remote operation of the actual machine operation mechanism included in the work machine that is the target of the remote operation by the remote operation mechanism, and the second support processing element executes processing of acquiring a correction value to make operation characteristics of the actual machine operation mechanism included in the work machine correspond to operation characteristics of the remote operation mechanism included in the remote operation apparatus configured to remotely operate the work machine, on at least one of the requirement that the determination result recognized by the first support processing element is negative and a requirement that the operation pattern recognized by the first support processing element is changed.

According to the remote operation support system with the configuration, processing of making the operation characteristics of the actual machine operation mechanism correspond to the operation characteristics of the remote operation mechanism is performed in a case where at least one of the operation pattern and the operation factor is changed. Consequently, the operability equivalent to that before the change can be maintained even in a case where only the operation pattern is changed.

Also, in the remote operation support system of the present invention, it is preferable that the second support processing element causes a remote output interface included in the remote operation apparatus to output a message that encourages change of the remote operation factor, and recognizes a new remote operation factor changed through a remote input interface included in the remote operation apparatus.

According to the remote operation support system with the configuration, in a case where the remote operation factor is different from the actual machine operation factor, processing of making an operation method of the remote operation mechanism included in the remote operation apparatus correspond to an operation method of the actual machine operation mechanism included in the work machine is executed depending on user's will in the remote operation apparatus. Consequently, in a case where both the factors correspond, the remote operation apparatus can start the remote operation of the work machine without waiting for execution completion of the processing. Therefore, improvement in remote operation efficiency of the work machine can be achieved by improving an efficiency of the processing.

Also, in the remote operation support system of the present invention, it is preferable that the first support processing element recognizes the determination result of whether the remote operation factor corresponds to the actual machine operation factor based on communication with the work machine, after the second support processing element recognizes the new remote operation factor changed through the remote input interface.

Claim 1:
A remote operation support server (<NUM>) for supporting a remote operation of a work machine (<NUM>) by a remote operation apparatus (<NUM>), the remote operation support server (<NUM>) comprising:
a first support processing element (<NUM>) which is configured to recognize a determination result of whether a remote operation factor defining an operation setting of a remote operation mechanism included in the remote operation apparatus (<NUM>), which remote operation factor corresponds to the actual machine operation mechanism included in the work machine (<NUM>) intended to be remotely operated by the remote operation mechanism (<NUM>) included in the remote operation apparatus (<NUM>), corresponds to an actual machine operation factor defining an operation setting (<NUM>), which actual machine operation factor corresponds to the actual machine operation mechanism capable of receiving the remote operation, and
a second support processing element (<NUM>) which is configured to execute processing of making the actual machine operation factor correspond to the remote operation factor, based on communication with at least one of the remote operation apparatus (<NUM>) and the work machine (<NUM>), on a requirement that the determination result recognized by the first support processing element (<NUM>) is negative, and
a database (<NUM>) which is configured to hold and store captured image data.