Patent Description:
A display device for a construction machine has been suggested, the display device being capable of encouraging an inexperienced operator to perform an optimal operation which brings efficient excavation work (for example, see Patent Literature <NUM>). Specifically, while being associated with respective operation amounts of a first operation unit which operates a tractive force of a vehicle body and a second operation unit which commands and operates a work instrument in a vehicle-body front portion, "a target tractive force value", "a tractive force value in actual excavation work with respect to a target lifting force value", and "a lifting force value", which are in advance set, in excavation work are displayed while being compared with each other. Accordingly, it is intended to encourage an operator to improve fuel efficiency and to perform an operation with high work efficiency in work at each of phases (traveling of a vehicle and a lift by a work instrument). A device according to the prior art is disclosed in <CIT>.

However, in view of an improvement in operation techniques, it is preferable that an operator having little experience in operating work machines gain experience in operation by simulations while referring to situations where skilled operators or the like operate work machines.

Accordingly, an object of the present invention is to provide a server and so forth that enable an operator to gain experience in operation by a simulation while referring to a situation of an operation of a work machine by the operator himself/herself or another operator.

The present invention relates to a work assisting server for assisting an improvement in operation techniques of a work machine by an operator based on communication with a client.

A work assisting server of the present invention includes: a first assisting process element that recognizes a time series of captured images which are acquired through an image-capturing device cooperating with a first work machine based on communication with a first client used by a first operator for operating the first work machine and which represent an action manner and an environment changing manner of the first work machine and that causes a time series of work environment images which correspond to the captured images to be accumulatively stored and retained in a database; and a second assisting process element that causes an output interface of a second client to output a time series of designated work environment images, which are designated through an input interface of the second client, among time series of the work environment images which are stored and retained in the database based on communication with the second client and that causes the output interface of the second client to output a time series of simulation images which represent an action manner and an environment changing manner of a second work machine at a virtual work site, the action manner and the environment changing manner corresponding to a simulation operation by a second operator through the input interface of the second client, based on a designated work environment image at one designated time point, which is designated through the input interface of the second client, in the time series of the designated work environment images.

A work assisting system of the present invention includes: the work assisting server of the present invention; the first client; and the second client.

In the work assisting server and the work assisting system (hereinafter, appropriately referred to as "the work assisting server and so forth") of the present invention, the second operator can designate, through the input interface of the second client, one time series of the work environment images from archives of times series of the work environment images or movies that are registered in the database and that correspond to the captured images which represent an operation manner, the action manner, and the environment changing manner of the first work machine by the first operator. As for the time series of the work environment images, for example, a time series of work environment images throughout a period from an execution start time point when the first work machine starts executing a designated task to an execution finishing time point may be defined as a unit which configures an archive. The second operator can browse the time series of the designated work environment images through the output interface of the second client. The first operator may be the same operator as the second operator, that is, the second operator himself/herself in addition to another operator who is different from the second operator.

The second operator can, through the input interface of the second client, designate the designated work environment image at one designated time point in the time series of the designated work environment images and can perform a simulation operation of the second work machine at the virtual work site that is based on the designated work environment image at the designated time point. The second operator can cause the output interface of the second client to output the time series of the simulation images which represent the action manner and the environment changing manner of the second work machine at the virtual work site in accordance with the simulation operation manner.

As described above, the second operator browses a time series of the designated work environment images which can be used as a suitable reference, then designates a designated work environment image, which can be used as a more suitable reference, from the time series of the designated work environment images, performs a simulation operation of the second work machine at the virtual work site which is reproduced based on the designated work environment images, and can thereby intend an improvement in operation techniques of the work machine.

A work assisting system as one embodiment of the present invention, which is illustrated in <FIG>, is configured with a work assisting server <NUM> and plural remote operation devices <NUM> for performing remote operations of plural work machines <NUM>. The work assisting server <NUM>, the remote operation devices <NUM>, and the work machines <NUM> are configured to be capable of mutual network communication.

The work assisting server <NUM> includes a database <NUM>, a first assisting process element <NUM>, and a second assisting process element <NUM>. The database <NUM> stores and retains captured images, work environment images, route guiding images, and so forth in addition to positions and tracks of each of the plural work machines <NUM>. The database <NUM> may be configured with a database server separate from the work assisting server <NUM>. Each of the assisting process elements is configured with an arithmetic processing unit (a single-core processor or a multi-core processor or a processor core configuring that), reads necessary data and software from a storage device such as a memory, and executes a computation process following the software for the data as a target, the computation process being described later.

The remote operation device <NUM> which configures a client includes a remote control device <NUM>, a remote input interface <NUM>, and a remote output interface <NUM>. The remote control device <NUM> is configured with an arithmetic processing unit (a single-core processor or a multi-core processor or a processor core configuring that), reads necessary data and software from a storage device such as a memory, and executes a computation process following the software for the data as a target. The remote input interface <NUM> includes a remote operation mechanism <NUM>. The remote output interface <NUM> includes an image output device <NUM> and a remote wireless communication apparatus <NUM>.

The client may be configured with a portable terminal which cooperates with the remote operation device <NUM> or has a mutual communication function. The portable terminal may have a communication function to communicate with the work assisting server <NUM>.

The remote operation mechanism <NUM> includes a traveling operation device, a revolution operation device, a boom operation device, an arm operation device, and a bucket operation device. Each of the operation devices has operation levers which accept a rotation operation. An operation lever (traveling lever) of the traveling operation device is operated to move a lower traveling body <NUM> of the work machine <NUM>. The traveling lever may also have a traveling pedal. For example, a traveling pedal may be provided which is fixed to a base portion or a lower end portion of the traveling lever. An operation lever (revolution lever) of the revolution operation device is operated to move a hydraulic revolution motor which configures a revolution mechanism <NUM> of the work machine <NUM>. An operation lever (boom lever) of the boom operation device is operated to move a boom cylinder <NUM> of the work machine <NUM>. An operation lever (arm lever) of the arm operation device is operated to move an arm cylinder <NUM> of the work machine <NUM>. An operation lever (bucket lever) of the bucket operation device is operated to move a bucket cylinder <NUM> of the work machine <NUM>.

As illustrated in <FIG>, the operation levers configuring the remote operation mechanism <NUM> are arranged around a seat St on which an operator is seated, for example. The seat St is in a form of a high-back chair with armrests but may be in an arbitrary form on which the operator can be seated such as a form of a low-back chair without a headrest or a form of a chair without a backrest.

A pair of left and right traveling levers <NUM> corresponding to left and right continuous tracks are arranged laterally side by side in a left-right direction in front of the seat St. One operation lever may be used as plural operation levers. For example, a right operation lever <NUM> provided in front of a right frame of the seat St, which is illustrated in <FIG>, may function as the boom lever in a case where the right operation lever <NUM> is operated in a front-rear direction and may function as the bucket lever in a case where the right operation lever <NUM> is operated in the left-right direction. Similarly, a left operation lever <NUM> provided in front of a left frame of the seat St, which is illustrated in <FIG>, may function as the arm lever in a case where the left operation lever <NUM> is operated in the front-rear direction and may function as the revolution lever in a case where the left operation lever <NUM> is operated in the left-right direction. Lever patterns may arbitrarily be changed by an operation instruction from the operator.

As illustrated in <FIG>, the image output device <NUM> is configured with an obliquely right front image output device <NUM>, a front image output device <NUM>, and an obliquely left front image output device <NUM> which are respectively arranged in obliquely right front, in front, and in obliquely left front of the seat St, for example. The image output devices <NUM> to <NUM> may further include a speaker (sound output device).

The work machine <NUM> includes an actual machine control device <NUM>, an actual machine input interface <NUM>, an actual machine output interface <NUM>, and an actuation mechanism <NUM>. The actual machine control device <NUM> is configured with an arithmetic processing unit (a single-core processor or a multi-core processor or a processor core configuring that), reads necessary data and software from a storage device such as a memory, and executes a computation process following the software for the data as a target.

The work machine <NUM> is a crawler excavator (construction machine), for example, and includes, as illustrated in <FIG>, a crawler type lower traveling body <NUM> and an upper revolving body <NUM> which is revolvably mounted on the lower traveling body <NUM> via the revolution mechanism <NUM>. A cab (operator cab) <NUM> is provided to a front left side portion of the upper revolving body <NUM>. A work attachment <NUM> is provided to a front central portion of an upper revolving body <NUM>.

The actual machine input interface <NUM> includes an actual machine operation mechanism <NUM> and an actual machine image-capturing device <NUM>. The actual machine operation mechanism <NUM> includes plural operation levers, which are arranged similarly to the remote operation mechanism <NUM>, around a seat arranged in an internal portion of the cab <NUM>. A drive mechanism or a robot, which receives a signal corresponding to an operation manner of remote operation levers and moves actual machine operation levers based on the received signal, is provided to the cab <NUM>. The actual machine image-capturing device <NUM> is installed in the internal portion of the cab <NUM>, for example, and captures an image of an environment including at least a part of the actuation mechanism <NUM> through a front window of the cab <NUM>.

The actual machine output interface <NUM> includes an actual machine wireless communication apparatus <NUM>.

The work attachment <NUM> as the actuation mechanism includes a boom <NUM> which is attached to the upper revolving body <NUM> to be capable of derricking, an arm <NUM> which is rotatably coupled with a distal end of the boom <NUM>, and a bucket <NUM> which is rotatably coupled with a distal end of the arm <NUM>. To the work attachment <NUM>, the boom cylinder <NUM>, the arm cylinder <NUM>, and the bucket cylinder <NUM> which are configured with hydraulic cylinders capable of extending and contracting are attached.

The boom cylinder <NUM> is interposed between the boom <NUM> and the upper revolving body <NUM> such that the boom cylinder <NUM> extends and contracts by being supplied with hydraulic oil so as to rotate the boom <NUM> in a derricking direction. The arm cylinder <NUM> is interposed between the arm <NUM> and the boom <NUM> such that the arm cylinder <NUM> extends and contracts by being supplied with hydraulic oil so as to rotate the arm <NUM> around a horizontal axis with respect to the boom <NUM>. The bucket cylinder <NUM> is interposed between the bucket <NUM> and the arm <NUM> such that the bucket cylinder <NUM> extends and contracts by being supplied with hydraulic oil so as to rotate the bucket <NUM> around a horizontal axis with respect to the arm <NUM>.

Functions of the work assisting system in the above configuration will be described by using flowcharts illustrated in <FIG> and <FIG>. In the flowcharts, a block indicated by "CXX" is used for simplification of descriptions, denotes transmission and/or reception of data, and denotes a conditional branch where a process in a branching direction is executed while transmission and/or reception of the data are used as a condition.

"Recognition" of information by configuration elements (arithmetic processing resources or hardware resources) of the present invention represents a concept encompassing processes for preparing the information in all forms in which the information is usable in subsequent processes, such as receiving the information, reading out or searching for the information from storage devices and so forth, writing or registering the information (causing the information to be stored or retained) in storage devices and so forth, and estimating, determining, identifying, measuring, and predicting, and so forth the information by executing a computation process of basic information following a predetermined algorithm, the basic information being obtained by an output signal and/or reception from a sensor, a search, and so forth.

The remote operation device <NUM> as a first client determines whether or not a designating operation through the remote input interface <NUM> by the operator is made (STEP <NUM> in <FIG>). A "designating operation" is an operation for selecting the work machine <NUM> which cooperates with the remote operation device <NUM> and is a touch operation such as tapping, swiping, flicking, pinching out, or pinching in on a touch panel which configures the remote input interface <NUM>, for example. The touch panel displays an actual machine image of the work machine <NUM> which can become a target of a remote operation, for example, and it is determined whether or not an operation is made to a display position of the actual machine image in the touch panel. In a case where the determination result is a negative result such as interruption of the designating operation (NO in STEP <NUM> in <FIG>), a series of processes is finished. On the other hand, in a case where the determination result is affirmative (YES in STEP <NUM> in <FIG>), a work environment image request for requesting captured images necessary for an operation of the work machine <NUM> which cooperates with the remote operation device <NUM> is transmitted to the work assisting server <NUM> through the remote wireless communication apparatus <NUM> (STEP <NUM> in <FIG>). The work environment image request includes at least one of an identifier of the remote operation device <NUM> and an identifier of the operator.

In a case where the work assisting server <NUM> receives the work environment image request, the first assisting process element <NUM> transmits the work environment image request to the concerned work machine <NUM> (C10 in <FIG>).

In a case where the work machine <NUM> receives the work environment image request through the actual machine wireless communication apparatus <NUM> (C41 in <FIG>), the actual machine control device <NUM> acquires captured images through the actual machine image-capturing device <NUM> (STEP <NUM> in <FIG>). The actual machine control device <NUM> transmits captured image data which represent the captured images to the work assisting server <NUM> through the actual machine wireless communication apparatus <NUM> (STEP <NUM> in <FIG>).

In a case where the work assisting server <NUM> receives the captured image data (C11 in <FIG>), work environment image data (data which represent all or a part of the captured images themselves or simulated work environment images generated based on those) corresponding to the captured image data are transmitted to the remote operation device <NUM> (STEP <NUM> in <FIG>). Further, a time series of the work environment image data is registered in or stored and retained in the database <NUM> (STEP <NUM> in <FIG>).

In a case where the remote operation device <NUM> receives the work environment image data through the remote wireless communication apparatus <NUM> (C20 in <FIG>), the work environment images corresponding to the work environment image data are output to the image output device <NUM> (STEP <NUM> in <FIG>). Accordingly, for example, as illustrated in <FIG>, the work environment image, which includes the boom <NUM>, the arm <NUM>, the bucket <NUM>, and the arm cylinder <NUM> as a part of the work attachment <NUM> as the actuation mechanism whose image is captured by the actual machine image-capturing device <NUM> mounted on the work machine <NUM> which cooperates with the remote operation device <NUM>, is displayed on the image output device <NUM>.

In the remote operation device <NUM>, an operation manner of the remote operation mechanism <NUM> is recognized by the remote control device <NUM> (STEP <NUM> in <FIG>), and a remote operation command corresponding to the operation manner is transmitted to the work assisting server <NUM> through the remote wireless communication apparatus <NUM> (STEP <NUM> in <FIG>).

In a case where the work assisting server <NUM> receives the remote operation command, the first assisting process element <NUM> transmits the remote operation command to the work machine <NUM> (C12 in <FIG>).

In a case where in the work machine <NUM>, the actual machine control device <NUM> receives an operation command through the actual machine wireless communication apparatus <NUM> (C42 in <FIG>), actions of the work attachment <NUM> or the like are controlled (STEP <NUM> in <FIG>). For example, work is executed in which earth in front of the work machine <NUM> is scooped by the bucket <NUM>, the upper revolving body <NUM> is revolved, and earth is then dropped from the bucket <NUM> (see <FIG>). As illustrated in <FIG>, in the above work, the work environment image (a movie or intermittent still images) at a time point t = tk (k = <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>), for example, is displayed on the image output device <NUM>. The work environment image at t = t1 represents a situation where immediately after the bucket <NUM> is moved to an excavation start position, the work attachment <NUM> is extended forward, and the bucket <NUM> is pushed to the ground. The work environment image at t = t2 represents a situation where an operation to move the arm <NUM> to a position closer to the work machine <NUM> is performed. The work environment image at t = t3 represents a situation where sufficient earth and sand enter the bucket <NUM> and the bucket <NUM> is thereafter caused to take a retainment posture in which earth is grasped. The work environment image at t = t4 represents a situation where left revolution is performed while an operation to raise the boom <NUM> is performed. The work environment image at t = t5 represents a situation where an operation to move the arm <NUM> away is performed after the left revolution and the bucket <NUM> is positioned right above an earth discharging position. The work environment image at t = t6 represents a situation where the bucket takes an earth discharging posture and earth falls from the bucket <NUM>.

The remote operation device <NUM> as a second client determines whether or not a playback operation through the remote input interface <NUM> by the operator is made (STEP <NUM> in <FIG>). A "playback operation" includes a touch operation for designating one work environment image from a library of the work environment images, the library being registered in the database <NUM> and displayed on a touch panel, the touch panel configuring the remote input interface <NUM> and the remote output interface <NUM>, for example. A start of playback, a stop of playback, a temporary stop of playback, fast-forwarding, and rewinding of the work environment images are made possible by a remote input interface <NUM>.

In a case where the determination result is negative (NO in STEP <NUM> in <FIG>), a series of processes is finished. On the other hand, in a case where the determination result is affirmative (YES in STEP <NUM> in <FIG>), a playback request is transmitted to the work assisting server <NUM> through the remote wireless communication apparatus <NUM> (STEP <NUM> in <FIG>). The playback request accompanying the playback operation includes an image identifier for identifying the work environment image designated by the operator.

In a case where the work assisting server <NUM> receives the playback request (C13 in <FIG>), the second assisting process element <NUM> searches for a time series of the work environment images, which is a playback target, as a time series of a designated work environment images from the database <NUM> based on the image identifiers included in the playback request (STEP <NUM> in <FIG>).

The second assisting process element <NUM> transmits data which represent a time series of designated work environment images to the remote operation device <NUM> (STEP <NUM> in <FIG>).

In a case where the remote operation device <NUM> receives the work environment image data through the remote wireless communication apparatus <NUM> (C21 in <FIG>), the work environment images corresponding to the work environment image data are played back in or output to the image output device <NUM> (STEP <NUM> in <FIG>). Accordingly, for example, as illustrated in <FIG>, the respective work environment images (a movie or intermittent still images) at different time points t = t1, t2, t3, t4, t5, and t6, the work environment images including the boom <NUM>, the arm <NUM>, the bucket <NUM>, and the arm cylinder <NUM> as a part of the work attachment <NUM> as the actuation mechanism, are displayed on the image output device <NUM> in a time-series manner.

The remote control device <NUM> determines whether or not a playback stop operation through an operation by the remote input interface <NUM> is made (STEP <NUM> in <FIG>).

In a case where the determination result is negative (NO in STEP <NUM> in <FIG>), a playback request without the playback operation is transmitted to the work assisting server <NUM> through the remote wireless communication apparatus <NUM>. As a result, the time series of the work environment images is continuously output on the image output device <NUM> (see STEP <NUM> → C13 → STEP <NUM> → STEP <NUM> → C21 → STEP <NUM> in <FIG>).

On the other hand, in a case where the determination result is affirmative (YES in STEP <NUM> in <FIG>), the remote control device <NUM> transmits a playback stop request to the work assisting server <NUM> through the remote wireless communication apparatus <NUM> which configures the remote output interface <NUM> (STEP <NUM> in <FIG>). In this case, because the playback request without the playback operation is not transmitted to the work assisting server <NUM>, as a result, a playback of the time series of the work environment images on the image output device <NUM> is stopped. The playback stop request includes a playback stop time point of the time series of the work environment images as a designated time point.

In a case where the work assisting server <NUM> receives the playback stop request (C14 in <FIG>), the second assisting process element <NUM> recognizes a simulation situation based on the work environment image at the designated time point included in the playback stop request (STEP <NUM> in <FIG>). For example, an action manner and an environment changing manner of the work machine (second work machine) at a virtual work site, which correspond to the work environment image at a designated time point t = tk in the time series of the work environment images illustrated in <FIG>, are recognized as the simulation situation.

In addition, the second assisting process element <NUM> transmits simulation image data which represent the simulation situation to the remote operation device <NUM> (STEP <NUM> in <FIG>).

In a case where in the remote operation device <NUM>, the simulation image data are received by the remote wireless communication apparatus <NUM> which configures the remote output interface <NUM> (C22 in <FIG>), a simulation image is output on the image output device <NUM> which configures the remote output interface <NUM> (STEP <NUM> in <FIG>). Accordingly, for example, a simulation image, which represents the action manner and the environment changing manner of the work machine (second work machine) at the virtual work site at a time point t = τj (j = <NUM>, <NUM>,. ) indicated in <FIG>, is output on the image output device <NUM>. In this case, the image output device <NUM> may output a simulation image instead of the work environment image in a stop state and may output the work environment image in the stop state in a corner of the simulation image.

The remote control device <NUM> determines whether or not a simulation stop operation through the remote input interface <NUM> is made (STEP <NUM> in <FIG>).

In a case where the determination result is affirmative (YES in STEP <NUM> in <FIG>), the remote control device <NUM> transmits a simulation stop request to the work assisting server <NUM> through the remote wireless communication apparatus <NUM> (STEP <NUM> in <FIG>).

On the other hand, in a case where the determination result is negative (NO in STEP <NUM> in <FIG>), the remote control device <NUM> recognizes the operation manner of the remote operation mechanism <NUM> (STEP <NUM> in <FIG>), and a simulation operation command corresponding to the operation manner is transmitted to the work assisting server <NUM> through the remote wireless communication apparatus <NUM> (STEP <NUM> in <FIG>).

In a case where the work assisting server <NUM> receives the simulation operation command (C14 in <FIG>), the second assisting process element <NUM> recognizes the simulation situation corresponding to the simulation operation command (STEP <NUM> in <FIG>). For example, in a case where the operation manner of the remote operation mechanism <NUM> is to realize a situation where the boom <NUM> is caused to rise with respect to the upper revolving body <NUM> and the whole work attachment <NUM> is thereby caused to rise, a situation where the work attachment <NUM> acts in such a way in a simulation is recognized as the simulation situation.

The second assisting process element <NUM> transmits data which represent simulation images to the remote operation device <NUM> (STEP <NUM> in <FIG>). Then, the second assisting process element <NUM> determines whether or not the simulation stop request is made (STEP <NUM> in <FIG>). In a case where the determination result is affirmative (YES in STEP <NUM> in <FIG>), a series of processes is finished. In a case where the determination result is negative (NO in STEP <NUM> in <FIG>), processes subsequent to reception of the simulation operation command are repeated (see C14 → STEP <NUM> → STEP <NUM> in <FIG>).

In a case where in the remote operation device <NUM>, the simulation image data are received by the remote wireless communication apparatus <NUM> which configures the remote output interface <NUM> (C22 in <FIG>), a simulation image is output on the image output device <NUM> which configures the remote output interface <NUM> (STEP <NUM> in <FIG>). Accordingly, for example, a simulation image at a time point t = τj (j = <NUM>, <NUM>,. ) indicated in <FIG> is output on the image output device <NUM>.

The work environment image at t = τ1 represents, as a simulation image, a situation where immediately after the bucket <NUM> is moved to an excavation start position, the work attachment <NUM> is extended forward, and the bucket <NUM> is pushed to the ground. The work environment image at t = τ2 represents, as a simulation image, a situation where an operation to move the arm <NUM> to a position closer to the work machine <NUM> is performed. The work environment image at t = τ3 represents, as a simulation image, a situation where sufficient earth and sand enter the bucket <NUM> and the bucket <NUM> is thereafter caused to take a retainment posture in which earth is grasped. The work environment image at t = τ4 represents, as a simulation image, a situation where left revolution is performed while an operation to raise the boom <NUM> is performed. The work environment image at t = τ5 represents, as a simulation image, a situation where an operation to move the arm <NUM> away is performed after the left revolution and the bucket <NUM> is positioned right above an earth discharging position. The work environment image at t = τ6 represents, as a simulation image, a situation where the bucket takes an earth discharging posture and earth falls from the bucket <NUM>.

In the work assisting system in the above configuration and the work assisting server <NUM> configuring that, a time series of the work environment images representing an action manner and an environment changing manner of a first work machine, the first work machine being remotely operated by a first operator through the remote operation device <NUM> which configures the first client, is registered in the database <NUM> (see STEP <NUM> in <FIG>). Accordingly, archives of time series of various work environment images are accumulated and preserved in the database <NUM>. A "first work machine" denotes the work machine <NUM> which is remotely operated by the first operator and executes work at an actual work site.

A second operator of the remote operation device <NUM> configuring the second client can designate one time series of the work environment images from the archives registered in the database <NUM> through the remote input interface <NUM> (see STEP <NUM> in <FIG>). Accordingly, the second operator can browse the time series of the designated work environment images on the remote output interface <NUM> (see STEP <NUM> in <FIG> and <FIG>).

The second operator can perform a simulation operation of the second work machine at a virtual work site that is based on the designated work environment images, through the remote operation mechanism <NUM> which configures the remote input interface <NUM> (see <FIG>). A "second work machine" denotes a work machine which is subjected to a simulation operation by the second operator and is actuated at a virtual work site. The second operator can cause the remote output interface <NUM> to output a time series of the simulation images which represent the action manner and the environment changing manner of the work machine at the virtual work site in accordance with the simulation operation manner (see STEP <NUM> → STEP <NUM> → C14 → STEP <NUM> → STEP <NUM> → STEP <NUM> → C22 → STEP <NUM> in <FIG> and <FIG>).

As described above, the second operator browses a time series of the work environment images which can be used as a suitable reference, then designates one work environment image, which can be used as a more suitable reference, from the time series of the work environment images, performs a simulation operation of the work machine at a virtual work site which is reproduced based on the designated work environment images, and can thereby intend an improvement in operation techniques of the work machine.

In the above embodiment, the work assisting server <NUM> is configured with one or plural servers which are separate from each of the remote operation devices <NUM> and the work machines <NUM> (see <FIG>); however, as another embodiment, the work assisting server <NUM> may be a configuration element of the remote operation device <NUM> or the work machine <NUM>. Each of the configuration elements <NUM> and <NUM> of the work assisting server <NUM> can be a configuration element of each of two or more apparatuses in the remote operation devices <NUM> and the work machines <NUM>, the two or more apparatuses being capable of mutual communication.

Based on communication with the remote operation device <NUM> which configures the second client, the second assisting process element <NUM> may cause the remote output interface <NUM> which configures the remote operation device <NUM> to output time series, while superimposing the time series of the simulation images on the designated time series, which is subsequent to a designated time point, in the time series of the designated work environment images. The second operator can cause the remote output interface <NUM> to output the simulation images which represent the action manner and the environment changing manner of the second work machine at the virtual work site, while superimposing the simulation images on the designated time series of the work environment images which is subsequent to a designated time point, in accordance with the simulation operation manner through the remote operation mechanism <NUM>.

Thus, the second operator is caused to recognize the difference between the action manner of the first work machine or the operation manner of the first operator and the environment changing manner, which are to be referred to, and the action manner of the second work machine or the simulation operation manner of the second operator and the environment changing manner at the virtual work site, and a further improvement in his/her operation techniques is intended.

The second assisting process element <NUM> may evaluate the difference between the action manner of the first work machine which is represented by the designated time series of the designated work environment images and the action manner of the second work machine which is represented by the time series of the simulation images. For example, an interval between designated parts (for example, the buckets <NUM>) of the respective work attachments <NUM> of the first work machine and the second work machine may be converted into an actual spatial distance, and a cumulative value or a time average of the conversion results value may thereby be evaluated as the difference.

Furthermore, in a case where the difference exceeds a threshold value, the second assisting process element <NUM> may cause the remote output interface <NUM> to output information for confirming whether or not redoing of the simulation operation of the second work machine is necessary.

Accordingly, in a case where the difference between the action manner of the first work machine or the operation manner of the first operator and the environment changing manner and the action manner of the second work machine or the simulation operation manner of the second operator and the environment changing manner at the virtual work site exceeds the threshold value, because it is confirmed that the redoing is necessary through the remote input interface <NUM>, a further improvement in operation techniques of the second operator is intended. Further, in a case where it is confirmed that the redoing is necessary, the remote output interface <NUM> may be caused to output a reproduction of the designated time series of the designated work environment images. Accordingly, because the second operator is capable of redoing the simulation operation, a further improvement in operation technique is intended.

Based on communication with the remote operation device <NUM> which configures the second client, the second assisting process element <NUM> may cause an operation state (for example, inclination angles or positions of the levers <NUM>, <NUM>, and <NUM>) of the remote operation mechanism <NUM> which configures the second client at a start time point of the simulation operation of the second work machine to agree with an operation state, at a designated time point, of the remote operation mechanism <NUM> of the remote operation device <NUM> which configures the first client.

Claim 1:
A work assisting server (<NUM>) for assisting an improvement in an operation technique of a work machine by an operator based on communication with a client, the work assisting server comprising:
a first assisting process element (<NUM>) that is configured to recognize a time series of captured images which are acquired through an image-capturing (<NUM>) device cooperating with a first work machine based on communication with a first client used by a first operator for operating the first work machine and which represent an action manner and an environment changing manner of the first work machine and that is configured to cause a time series of work environment images which correspond to the captured images to be accumulatively stored and retained in a database; and
a second assisting process element (<NUM>) that is configured to cause an output interface of a second client to output a time series of designated work environment images, which are designated through an input interface of the second client, among time series of the work environment images which are stored and retained in the database based on communication with the second client and that is configured to cause the output interface of the second client to output a time series of simulation images which represent an action manner and an environment changing manner of a second work machine at a virtual work site, the action manner and the environment changing manner corresponding to a simulation operation by a second operator through the input interface (<NUM>) of the second client, based on a work environment image at one designated time point, which is designated through the input interface of the second client, in the time series of the designated work environment images.