Patent ID: 12258726

DETAILED DESCRIPTION

An aspect of this disclosure makes it possible to provide an operation support system for a construction machine capable of outputting support data with a higher rating regarding a target index.

Embodiments of the present invention are described below with reference to the accompanying drawings.

[Outline of Operation Support System]

First, an outline of an operation support system SYS is described with reference toFIG.1.

FIG.1is a drawing illustrating an example of the operation support system SYS.

The operation support system SYS includes multiple shovels100, a flying body200, and a management device300.

The operation support system SYS collects result information related to operation patterns (which is hereafter referred to as “operation pattern result information”) of predetermined types of operations (e.g., repetitive operations such as an excavation operation, a loading operation, and a compaction operation) and result information related to environmental conditions during the operations (which is hereafter referred to as “environmental condition result information”) from the multiple shovels100. An operation pattern indicates a pattern of a series of movements of the shovel100when performing a predetermined type of operation. For example, an operation pattern includes traces of movement of moving components such as a lower traveling body1, an upper rotating body3, a boom4, an arm5, and a bucket6during an operation. Specifically, operation pattern result information is, for example, detection information of various sensors indicating results of an operation pattern of the shovel100when the shovel100actually performs a predetermined type of operation. Environmental conditions may include external environmental conditions such as conditions related to the surrounding environment of the shovel100, as well as internal environmental conditions such as variable specifications (for example, an arm length and a bucket type) of the shovel100that affect the movement of the shovel100. The operation support system SYS performs machine learning based on the collected operation pattern result information and environmental condition result information to generate, for respective environmental conditions, operation patterns (optimum operation patterns) that are optimum for the shovel100to perform predetermined types of operations. An optimum operation pattern is an operation pattern generated so that the rating regarding a predetermined target index becomes relatively high. Examples of target indices include a high operation speed, a good fuel economy, a long lifetime of an attachment, a low frequency of occurrence of impact load, a large loading amount in a loading operation, and a slow growth of a crack in an attachment after the occurrence of the crack. Then, the operation support system SYS outputs an optimum operation pattern under the current environmental conditions of the shovel100to be supported based on the generated operation patterns and assists the operator so that the shovel100operates according to the optimum operation pattern.

The operation support system SYS may include other types of construction machines (e.g., an asphalt finisher and a bulldozer) in place of or in addition to the shovels100. Also, the flying body200may be provided in each of multiple sites. That is, the operation support system SYS may include multiple flying bodies200.

<Outline of Shovel>

The shovel100(an example of a construction machine) includes a lower traveling body1, an upper rotating body3mounted on the lower traveling body1to be rotatable via a rotating mechanism2, a boom4, an arm5, and a bucket6as attachments (work devices), and a cabin10.

The lower traveling body1includes, for example, a pair of right and left crawlers, and travels when the crawlers are hydraulically driven by traveling hydraulic motors1A and1B (seeFIG.2).

The upper rotating body3rotates with respect to the lower traveling body1by being driven by a rotation hydraulic motor2A (seeFIG.2).

The boom4is mounted on the front center of the upper rotating body3such that the boom4is pivotable vertically, the arm5is attached to an end of the boom4such that the arm5can rotate vertically, and the bucket6is attached to an end of the arm5such that the bucket6can rotate vertically. The boom4, the arm5, and the bucket6are hydraulically-driven by a boom cylinder7, an arm cylinder8, and a bucket cylinder9, respectively.

The cabin10is a cockpit for an operator, and is mounted on the front left side of the upper rotating body3.

The shovel100can communicate with the management device300through a predetermined communication network NW such as a mobile communication network including a base station as a terminal, a satellite communication network using communication satellites in the sky, or the Internet. This enables the shovel100to send (upload) various types of information including the operation pattern result information and the environmental condition result information to the management device300. Details are described later.

<Outline of Flying Body>

The flying body200flies over a work site where the shovel100operates. The flying body200may be configured to fly according to operation commands from a remote controller being held by the operator on the ground of the work site, or to fly automatically according to, for example, a predetermined flight route.

As described later, the flying body200includes a camera240and captures images of the work site (which are hereafter referred to as “work site images”).

Also, the flying body200can communicate with the management device300via the communication network NW. This enables the flying body200to send (upload), for example, a work site image captured by the camera240, information (“flying body orientation information”) indicating the orientation of the flying body when the work site image is captured, and information (“flying body location information”) indicating the position of the flying body when the work site image is captured. Details are described later.

<Outline of Management Device>

The management device300is a terminal device installed in a location that is geographically away from the shovel100. The management device300is, for example, a server device installed in a management center provided outside of a work site where the shovel100operates, and is mainly composed of one or more server computers. In this case, the server device may be an in-house server run by a business operator managing the operation support system SYS or by a related business operator related to the business operator, or may be a so-called cloud server.

As described above, the management device300can communicate with each of the shovel100and the flying body200via the communication network NW. This enables the management device300to receive the operation pattern result information and the environmental condition result information uploaded from the shovel100and the work site image uploaded from the flying body200, and to generate an optimum operation pattern for the shovel100to be supported based on the received information. Details are described later.

[Configuration of Operation Support System]

Next, a configuration of the operation support system SYS is described with reference toFIGS.1and2.

FIG.2is a drawing illustrating an example of a configuration of the operation support system SYS.

InFIG.2, a mechanical power line is indicated by a double line, a high-pressure hydraulic line is indicated by a thick solid line, a pilot line is indicated by a dotted line, and an electric drive-control line is indicated by a thin solid line.

<Configuration of Shovel>

A hydraulic drive system for hydraulically driving hydraulic actuators of the shovel100according to the present embodiment includes an engine11, a main pump14, a regulator14a, and a control valve system17. Also, as described above, the hydraulic drive system of the shovel100according to the present embodiment includes hydraulic actuators such as traveling hydraulic motors1A and1B, a rotation hydraulic motor2A, a boom cylinder7, an arm cylinder8, and a bucket cylinder9that hydraulically drive the lower traveling body1, the upper rotating body3, the boom4, the arm5, and the bucket6, respectively.

The engine11is the main power source in the hydraulic drive system, and is mounted on, for example, a rear part of the upper rotating body3. Specifically, the engine11rotates constantly at a preset target rotation speed under the control of an engine control device (ECU: Engine Control Unit)74described later to drive the main pump14and the pilot pump15. The engine11is, for example, a diesel engine fueled by light oil.

The regulator14acontrols the discharge rate of the main pump14. For example, the regulator14aadjusts the angle (tilt angle) of a swash plate of the main pump14according to a control command from the controller30.

Similarly to the engine11, the main pump14is mounted on, for example, a rear part of the upper rotating body3, and supplies hydraulic oil to the control valve system17through a high-pressure hydraulic line16. The main pump14is driven by the engine11as described above. The main pump14is, for example, a variable displacement hydraulic pump. As described above, the stroke length of the piston is adjusted by adjusting the tilt angle of the swash plate with the regulator14aunder the control of the controller30, and the discharge flow rate (discharge pressure) of the main pump14is thereby controlled.

The control valve system17is, for example, a hydraulic control device that is provided in the middle of the upper rotating body3and controls the hydraulic drive system in response to an operation of an operating device26by the operator. As described above, the control valve system17is connected to the main pump14via the high-pressure hydraulic line16and supplies hydraulic oil supplied from the main pump14selectively to the hydraulic actuators (the traveling hydraulic motors1A and1B, the rotation hydraulic motor2A, the boom cylinder7, the arm cylinder8, and the bucket cylinder9) according to the operation state of the operating device26. Specifically, the control valve system17includes multiple control valves that control the flow rates and the flow directions of the hydraulic oil supplied from the main pump14to the respective hydraulic actuators. For example, the control valve system17includes a control valve175that corresponds to the boom4(boom cylinder7) (seeFIG.9). Also, for example, the control valve system17includes a control valve176corresponding to the arm5(arm cylinder8) (seeFIG.9). Also, for example, the control valve system17includes control valve174corresponding to the bucket6(bucket cylinder9) (seeFIG.9). Also, for example, the control valve system17includes a control valve173corresponding to the upper rotating body3(rotation hydraulic motor2A) (seeFIG.9). Further, for example, the control valve system17includes a right traveling control valve and a left traveling control valve corresponding to the right crawler and the left crawler of the lower traveling body1.

The operation system of the shovel100according to the present embodiment includes the pilot pump15, the operating device26, and an operation valve31.

For example, the pilot pump15is provided in a rear part of the upper rotating body3and supplies a pilot pressure to the operating device26and the operation valve31via the pilot line25. The pilot pump15is, for example, a fixed displacement hydraulic pump, and is driven by the engine11as described above.

The operating device26is provided near the cockpit of the cabin10, and is an operation input unit used by the operator to operate various moving components (e.g., the lower traveling body1, the upper rotating body3, the boom4, the arm5, and the bucket6). In other words, the operating device26is an operation input unit used by the operator to operate hydraulic actuators (e.g., the traveling hydraulic motors1A and1B, the rotation hydraulic motor2A, the boom cylinder7, the arm cylinder8, and the bucket cylinder9) for driving the moving components. Pilot lines on the secondary side of the operating device26are connected to the control valve system17. With this configuration, pilot pressures corresponding to the operation states of the lower traveling body1, the upper rotating body3, the boom4, the arm5, and the bucket6by the operating device26are input to the control valve system17. Therefore, the control valve system17can drive the hydraulic actuators according to the operation states of the operating device26.

The operation valve31adjusts the flow path area of the pilot line25according to a control command (for example, a control current) from the controller30. With this configuration, the operation valve31can output a pilot pressure corresponding to a control command to a secondary pilot line based on a primary pilot pressure supplied from the pilot pump15. The secondary ports of the operation valve31are connected to the right and left pilot ports of each control valve of the control valve system17corresponding to a hydraulic actuator, and the operation valve31applies a pilot pressure corresponding to a control command from the controller30to the pilot ports of the control valve. With this configuration, even when the operating device26is not being operated by the operator, the controller30can drive a hydraulic actuator by supplying the hydraulic oil discharged from the pilot pump15via the operation valve31to the pilot ports of the corresponding control valve in the control valve system17.

In addition to the operation valve31, an electromagnetic relief valve may be provided to relieve an excess hydraulic pressure generated in the hydraulic actuator into a hydraulic oil tank. This makes it possible to actively suppress the operation of a hydraulic actuator when the amount of operation of the operating device26performed by the operator is excessive. For example, an electromagnetic relief valve may be provided to relieve the excess pressure in the bottom side oil chamber and the rod side oil chamber of each of the boom cylinder7, the arm cylinder8, and the bucket cylinder9into the hydraulic oil tank.

The control system of the shovel100of the present embodiment includes a controller30, an ECU74, a discharge pressure sensor14b, an operation pressure sensor15a, a display device40, an input device42, an imaging device80, a state detection device S1, and a communication device T1.

The controller30performs drive control of the shovel100. The functions of the controller30may be implemented by hardware, software, or a combination of them. For example, the controller30is mainly composed of a computer that includes a processor such as a central processing unit (CPU), a memory device such as a random access memory (RAM), a non-volatile secondary storage such as a read-only memory (ROM), and interfaces for various inputs and outputs. The controller30implements various functions by, for example, executing various programs installed in the secondary storage on the CPU. The same applies to the ECU74, a control device210of the flying body200, and a control device310of the management device300described later.

For example, the controller30sets a target rotation speed based on an operation mode preset according to a predetermined operation by the operator, and outputs a control command to the ECU74to control the engine11, via the ECU74, to rotate at a constant speed.

Also, for example, the controller30outputs a control command to the regulator14aas needed to change the discharge amount of the main pump14and thereby perform a so-called total horsepower control or a negative control.

Also, for example, the controller30may include a function (upload function) to upload various types of information related to the shovel100to the management device300. Specifically, the controller30may be configured to send (upload) operation pattern result information and environmental condition result information during a predetermined type of operation of the shovel100to the management device300via the communication device T1. The controller30includes an information transmitter301as a functional unit that is related to the uploading function and implemented by, for example, executing one or more programs installed in the secondary storage on the CPU.

Also, for example, the controller30controls the machine guidance function that guides the manual operation of the shovel100via the operating device26by the operator. The controller30may also control the machine control function that automatically assists the manual operation of the shovel100via the operating device26by the operator. The controller30includes an operation pattern acquirer302and a machine guidance unit303as functional units that are related to the machine guidance function and the machine control function and implemented by, for example, executing one or more programs installed in the secondary storage on the CPU.

A part of the functions of the controller30may be implemented by another controller (control device). That is, the functions of the controller30may be distributed to multiple controllers. For example, the machine guidance function and the machine control function described above may be implemented by dedicated controllers (control devices).

The ECU74controls various actuators (for example, a fuel injection device) of the engine11in response to a control command from the controller30, and causes the engine11to rotate at a constant target rotation speed (set rotation speed) (constant rotation control). The ECU74performs the constant rotation control of the engine11based on the rotation speed of the engine11detected by an engine rotation speed sensor11a.

The discharge pressure sensor14bdetects the discharge pressure of the main pump14. The detection signal corresponding to the discharge pressure detected by the discharge pressure sensor14bis input to the controller30.

As described above, the operation pressure sensor15adetects secondary pilot pressures at the operating device26, that is, pilot pressures at the operating device26corresponding to the operation states of respective moving components (hydraulic actuators). Detection signals of pilot pressures at the operating device26corresponding to the operation states of, for example, the lower traveling body1, the upper rotating body3, the boom4, the arm5, and the bucket6detected by the operation pressure sensor15aare input to the controller30.

The display device40is connected to the controller30, disposed in a position where the display device40is easily visible by the operator seated in the cabin10, and displays various information images under the control of the controller30. The display device40is, for example, a liquid crystal display or an organic EL (electroluminescence) display.

The input device42is provided within the reach of the operator seated in the cabin10, receives various operations input by the operator, and outputs signals corresponding to the operations. For example, the input device42is integrated with the display device40. Alternatively, the input device42may be provided separately from the display device40. The input device42includes, for example, a touch panel mounted on a display of the display device40, a knob switch provided at an end of a lever included in the operating device26, a button switch, a lever, and a toggle disposed around the display device40. The signals corresponding to the operations on the input device42are input to the controller30.

The imaging device80captures images of scenes surrounding the shovel100. The imaging device80includes a camera80F capturing an image of the front side of the shovel100, a camera80L capturing an image of the left side of the shovel100, a camera80R capturing an image of the right side of the shovel100, and a camera80B capturing an image of the rear side of the shovel100.

The camera80F is attached to, for example, the ceiling of cabin10, i.e., a part inside of the cabin10. Also, the camera80F may be attached to a part outside of the cabin10such as the roof of the cabin10or a side surface of the boom4. The camera80L is attached to the upper left edge of the upper rotating body3, the camera80R is attached to the upper right edge of the upper rotating body3, and the camera80B is attached to the upper rear edge of the upper rotating body3.

The imaging device80(each of the cameras80F,80B,80L, and80R) is, for example, a monocular wide-angle camera having a very wide angle of view. Also, the imaging device80may be a stereo camera or a distance image camera. The images (which are hereafter referred to as “surrounding images”) of scenes around the shovel100captured by the imaging device80are input to the controller30.

The state detection device S1outputs detection information related to various states of the shovel100. The detection information output from the state detection device S1is input to the controller30.

For example, the state detection device S1detects posture states and operation states of attachments. Specifically, the state detection device S1may detect the depression/elevation angles of the boom4, the arm5, and the bucket6(which are hereafter referred to as a “boom angle”, an “arm angle”, and a “bucket angle”, respectively). That is, the state detection device S1may include a boom angle sensor S11, an arm angle sensor S12, and a bucket angle sensor S13that detect the boom angle, the arm angle, and the bucket angle, respectively (seeFIG.9). The state detection device S1may also detect the acceleration and the angular acceleration of each of the boom4, the arm5, and the bucket6. In this case, the state detection device S1may include, for example, a rotary encoder, an acceleration sensor, an angular acceleration sensor, a 6-axis sensor, and an inertial measurement unit (IMU) attached to each of the boom4, the arm5, and the bucket6. Further, the state detection device S1may include cylinder sensors each of which detects the cylinder position, the velocity, and the acceleration of one of the boom cylinder7, the arm cylinder8, and the bucket cylinder9that drive the boom4, the arm5, and the bucket6, respectively.

Further, for example, the state detection device S1detects the attitude states of bodies, i.e., the lower traveling body1and the upper rotating body3. Specifically, the state detection device S1may detect the inclined states of the bodies with respect to a horizontal plane. In this case, the state detection device S1may include, for example, an inclination sensor that is attached to the upper rotating body3and detects inclination angles (which are hereafter referred to as a “longitudinal inclination angle” and a “lateral inclination angle”) around two axes in the longitudinal direction and the lateral direction of the upper rotating body3.

Also, for example, the state detection device S1detects the rotating state of the upper rotating body3. Specifically, the state detection device S1detects the rotation angular velocity and the rotation angle of the upper rotating body3. In this case, the state detection device S1may include, for example, a gyro sensor, a resolver, and a rotary encoder attached to the upper rotating body3. That is, the state detection device S1may include a rotation angle sensor S15that detects the rotation angle of the upper rotating body3.

Also, for example, the state detection device S1detects the state of force being applied to the shovel100via attachments. Specifically, the state detection device S1may detect the working pressures (cylinder pressures) of hydraulic actuators. In this case, the state detection device S1may include pressure sensors that detect pressures in the rod side oil chambers and the bottom side oil chambers of the boom cylinder7, the arm cylinder8, and the bucket cylinder9.

Also, for example, the state detection device S1may include sensors that detect displacements of spools of control valves in the control valve system17. Specifically, the state detection device S1may include a boom spool displacement sensor S16that detects a displacement of a boom spool constituting a control valve175. The state detection device S1may also include an arm spool displacement sensor S17that detects a displacement of an arm spool constituting a control valve176. The state detection device S1may also include a bucket spool displacement sensor S18that detects a displacement of a bucket spool constituting a control valve174. Also, the state detection device S1may include a rotation spool displacement sensor S19that detects a displacement of a rotation spool constituting a control valve173. Further, the state detection device S1may include a right traveling spool displacement sensor and a left traveling spool displacement sensor that detect displacements of a right traveling spool and a left traveling spool that constitute a right traveling control valve and a left traveling control valve, respectively.

Further, for example, the state detection device S1detects the position of the shovel100and the orientation of the upper rotating body3. In this case, the state detection device S1may include, for example, a GNSS (Global Navigation Satellite System) compass, a GNSS sensor, and a direction sensor that are attached to the upper rotating body3.

The communication device T1communicates with an external device through the communication network NW. The communication device T1is, for example, a mobile communication module that supports mobile communication standards such as the long term evolution (LTE), the 4th Generation (4G), and the 5th Generation (5G) or a satellite communication module for connecting to a satellite communication network. The same applies to the communication device220of the flying body200.

The information transmitter301transmits operation pattern result information and environmental condition result information during a predetermined type of operation of the shovel100to the management device300via the communication device T1. The operation pattern result information transmitted by the information transmitter301includes, for example, various types of detection information input from the state detection device S1. Also, the environmental condition result information transmitted by the information transmitter301includes, for example, surrounding images of the shovel.100input from the imaging device80. Also, the environmental condition result information transmitted by the information transmitter301may include information related to internal environmental conditions of the shovel100such as variable specifications including a large bucket specification, a long arm specification, and a quick coupling specification. For example, the information transmitter301continuously determines whether a predetermined target-type operation is being performed. When it is determined that the target-type operation is being performed, the information transmitter301records operation pattern result information (i.e., various types of detection information input from the state detection device S1) and environmental condition information (i.e., surrounding images of the shovel100input from the imaging device80) during a period where the operation is performed in, for example, an internal memory in association with each other. Here, the information transmitter301may also store, in the internal memory, date and time information related to the start and end of the target-type operation and position information of the shovel100during the operation in association with the set of the operation pattern result information and the environmental condition result information. This enables the management device300to extract a work site image corresponding to the set of the operation pattern result information and the environmental condition result information sent from the shovel100from among work site images uploaded from the flying body200. The date and time information may be obtained from, for example, a predetermined time measuring unit (for example, a real time clock (RTC)) inside of the controller30. Then, the information transmitter301transmits the set of the recorded operation pattern result information and the recorded environmental condition result information to the management device300via the communication device T1at a predetermined timing such as a timing when the shovel100is keyed off (stopped). Also, the information transmitter301may be configured to send the set of the recorded operation pattern result information and the recorded environmental condition result information to the management device300via the communication device T1each time after the target-type operation is completed.

The environmental condition result information may include detection information detected by other sensors mounted on the shovel100in place of or in addition to the imaging device80. For example, the shovel100may be equipped with other sensors such as a millimeter-wave radar and a LIDAR (Light Detecting and Ranging) device, and the environmental condition result information may include detection information of these distance sensors. The same applies to the current environmental condition information described later. The environmental condition result information may also include weather information. The weather information may include information detected by, for example, a raindrop detection sensor and an illuminance sensor that may be included in the state detection device S1. The information transmitter301may be configured to transmit only the operation pattern result information to the management device300. In this case, the management device300can generate environmental condition result information corresponding to the operation pattern result information sent from the shovel100based on captured images captured by the flying body200flying over the work site of the shovel100. Also, the information transmitter301may consecutively upload detection information detected by the state detection device S1and surrounding images of the shovel100captured by the imaging device80to the management device300via the communication device T1. In this case, the management device300may extract information corresponding to a period when the target-type operation is performed from the information uploaded from the shovel100, and generate operation pattern result information and environmental information.

When a predetermined type of operation is to be performed, the operation pattern acquirer302obtains, from the management device300, an operation pattern (optimum operation pattern) that is optimum for the current environmental conditions related to a predetermined target index. For example, in response to a predetermined operation (which is hereafter referred to as an “acquisition request operation”) performed by the operator on the input device42, the operation pattern acquirer302sends a signal (acquisition request signal) requesting to obtain an operation pattern to the management device300via the communication device T1. The acquisition request signal includes information (which is hereafter referred to as “current environmental condition information”) related to the current environmental conditions of the shovel100. With this configuration, the management device300can provide, to the shovel100, an optimum operation pattern that matches the current environmental conditions of the shovel100. The current environmental condition information includes, for example, the latest surrounding images of the shovel100captured by the imaging device80. The current environmental condition information may also include information related to internal environmental conditions of the shovel100, such as variable specifications including a large bucket specification, a long arm specification, and a quick coupling specification. Also, the current environmental condition information may include information, i.e., weather information, detected by, for example, a raindrop detection sensor and an illuminance sensor that may be included in the state detection device S1. Then, the operation pattern acquirer302obtains information on an operation pattern that is transmitted from the management device300in response to the acquisition request signal and received by the communication device T1.

The operation pattern acquirer302may not necessarily send the current environmental condition information to the management device300together with the acquisition request signal. In this case, the management device300may determine the current environmental conditions (external environmental conditions) of the shovel100based on a work site image that corresponds to the work site of the shovel100and is uploaded from the flying body200. Also, the management device300may obtain weather information as an environmental condition of the work site of the shovel100from, for example, a server or a website related to weather information based on flying body position information uploaded from the flying body200.

The machine guidance unit303performs control processes related to the machine guidance function and the machine control function. That is, the machine guidance unit303assists the operator in operating various operating elements (the lower traveling body1, the upper rotating body3, and attachments including the boom4, the arm5, and the bucket6) via the operating device26.

For example, when the operator is operating the arm5via the operating device26, the machine guidance unit303may automatically operate at least one of the boom4and the bucket6so that an end (e.g., the tip or the back side) of the bucket6matches a predetermined target design surface (which is hereafter simply referred to as a “design surface”). Also, the machine guidance unit303may automatically operate the arm5regardless of the operation state of the operating device26for operating the arm5. That is, the machine guidance unit303may be triggered by the operation of the operating device26by the operator to cause an attachment to perform a predetermined operation.

More specifically, the machine guidance unit303acquires various types of information from the state detection device S1, the imaging device80, the communication device T1, and the input device42. Also, the machine guidance unit303calculates a distance between the bucket6and the design surface based on the obtained information. Then, the machine guidance unit303appropriately controls the operation valve31according to the calculated distance between the bucket6and the design surface, and individually and automatically adjusts the pilot pressures applied to the control valves corresponding to the hydraulic actuators to cause the hydraulic actuators to operate automatically. The operation valve31includes, for example, a boom proportional valve31A corresponding to the boom4(the boom cylinder7) (seeFIG.9). The operation valve31also includes, for example, an arm proportional valve31B corresponding to the arm5(the arm cylinder8) (seeFIG.9). The operation valve31also includes, for example, a bucket proportional valve31C corresponding to the bucket6(the bucket cylinder9) (seeFIG.9). The operation valve31also includes, for example, a rotation proportional valve31D corresponding to the upper rotating body3(the rotation hydraulic motor2A) (seeFIG.9). Further, the operation valve31includes, for example, a right traveling proportional valve and a left traveling proportional valve corresponding to the right-side crawler and the left-side crawler of the lower traveling body1.

For example, the machine guidance unit303may automatically expand and contract at least one of the boom cylinder7, the arm cylinder8, and the bucket cylinder9in response to the opening and closing operations of the arm5with the operating device26to assist an excavation operation. The excavation operation is an operation of excavating the ground with the tip of the bucket6along the design surface. The machine guidance unit303automatically expands and contracts at least one of the boom cylinder7and the bucket cylinder9when, for example, the operator is manually operating the operating device26to operate the arm5in the closing direction (which is hereafter referred to as an “arm closing operation”).

Also, the machine guidance unit303may automatically expand and contract at least one of the boom cylinder7, the arm cylinder8, and the bucket cylinder9to assist a finishing operation (compaction operation) on a slope or a horizontal surface. The compaction operation is an operation of pulling the bucket6along the design surface while pressing the back side of the bucket6against the ground. The machine guidance unit303automatically expands and contracts at least one of the boom cylinder7and the bucket cylinder9when, for example, the operator is manually operating the operating device26to perform the arm closing operation. This makes it possible to move the bucket6along the design surface, which is a finished slope or horizontal surface, while pressing the back side of the bucket6against an unfinished slope or horizontal surface with a predetermined pressing force.

The machine guidance unit303may also automatically rotate the rotation hydraulic motor2A to cause the upper rotating body3to face the design surface. In this case, the machine guidance unit303may cause the upper rotating body3to face the design surface in response to an operation of a predetermined switch included in the input device42. Also, the machine guidance unit303may cause the upper rotating body3to face the design surface and start the machine control function in response to a simple operation of a predetermined switch.

Further, for example, when a predetermined type of operation (for example, an excavation operation, a loading operation, or a compaction operation) is being performed, the machine guidance unit303controls at least a part of operations of attachments, the upper rotating body3, and the lower traveling body1to match the operation pattern (optimum operation pattern) obtained by the operation pattern acquirer302in response to an operation of the operating device26by the operator. With this configuration, the operator can make the operation of the shovel100match an operation pattern that is optimum for the current environmental conditions of the shovel100and output from the management device300(an optimum controller3103described later) so that the rating of a predetermined target index such as the operation speed becomes relatively high regardless of the proficiency of the operator in maneuvering the shovel100.

Further, the machine guidance unit303may display the movement of the shovel100corresponding to the optimum operation pattern on the display device40for the operator while controlling the movement of the shovel100based on the optimum operation pattern. For example, when controlling the movement of the shovel100based on the optimum operation pattern, the machine guidance unit303displays, on the display device40, a video of a simulation result of a simulator3102D corresponding to the optimal operation pattern. This enables the operator to perform an operation while confirming the actual operation pattern with a video displayed on the display device40.

<Configuration of Flying Body>

The flying body200is an autonomous flying body that can be flown by remote control or autopilot and may be, for example, a multicopter or an airship. In the present embodiment, as illustrated inFIG.1, the flying body200is a quadcopter.

The flying body200includes a control device210, a communication device220, an autonomous navigation device230, a camera240, and a positioning device250.

The control device210performs various controls related to the flying body200.

For example, the control device210transmits work site images consecutively input from the camera240to the management device300via the communication device220. Also, the control device210may be configured to send a work site image input from the communication device220via and the camera240to the management device300when a work site image transmission request is received from the management device300. Further, the control device210may be configured to send work site images buffered for a certain period of time at once via the communication device220to the management device300at a predetermined timing. Also, the control device210may be configured to transmit work site images to the management device300together with position information and date and time information corresponding to the respective work site images. In this case, the date and time information may be obtained from, for example, a predetermined time measuring unit (for example, an RTC) inside of the control device210.

The communication device220communicates with an external device via the communication network NW. Specifically, the communication device220communicates with the management device300under the control of the control device210. The communication device220is connected to the control device210, and various types of information received from the outside are input to the control device210.

The autonomous navigation device230is a device for realizing autonomous navigation of the flying body200. The autonomous navigation device230includes, for example, a flight control device, an electric motor, and a battery. Also, the flying body200may be equipped with a GNSS receiver in order to independently determine the position of the flying body200. Also, when an external power source on the ground is used via a wired connection instead of a battery, the flying body200may include a converter that performs voltage conversion. Further, the flying body200may include a solar panel. The flight control device includes various sensors such as a gyro sensor, an acceleration sensor, a barometric pressure sensor, and an ultrasonic sensor, and provides an attitude maintenance function and an altitude maintenance function. The electric motor receives power from the battery and rotates the propellers. When receiving information related to a target flight position from, for example, the control device210, the autonomous navigation device230controls the rotation speeds of the four propellers separately, and causes the flying body200to move to the target flight position while maintaining the attitude and the altitude of the flying body200. The information related to the target flight position includes, for example, the latitude, longitude, and altitude of the target flight position. The control device210acquires the information on the target flight position from the outside via the communication device220. The autonomous navigation device230may change the orientation of the flying body200when receiving information on a target orientation from the control device210.

The camera240captures images of scenes of a work site below the flight area of the flying body200. The camera240may be attached to, for example, the lower surface of the flying body200to be able to capture images of scenes vertically below the flying body200. The images (work site images) captured by the camera240are input to the control device210.

The positioning device250detects the position and the orientation of the flying body200. For example, the positioning device250may include a GNSS compass, a GNSS sensor, and a direction sensor (geomagnetic sensor). The information detected by the positioning device250is input to the control device210.

The positioning device250may be built in the autonomous navigation device230(flight control device).

<Configuration of Management Device>

The management device300includes a control device310, a communication device320, an operation input device330, and a display device340.

The control device310performs various controls in the management device300.

For example, the control device310performs machine learning (supervised learning and reinforcement learning) based on the collected operation pattern result information and environmental condition result information and thereby controls a function (which is hereafter referred to as a “machine learning function”) for generating an operation pattern (optimum operation patterns) that is optimum for a predetermined type of operation performed by the shovel100for each of multiple environmental conditions. Also, the control device310controls a function (which is hereafter referred to as an “operation support function”) that outputs the optimum operation pattern under the current environmental condition of the shovel100to be supported based on the generated operation patterns.

The control device310includes an information acquirer3101, an operation pattern generator3102, and an optimum controller3103as functional units that are related to the machine learning function and the operation support function and are implemented by executing one or more programs installed in the secondary storage on the CPU. The control device310also includes a storage3100as a storage area that is related to the machine learning function and the operation support function and defined in a non-volatile storage device in the control device310. Details of the configuration of the management device300related to the machine learning function and the operation support function are described later (seeFIG.3).

The storage3100may instead be provided outside of the control device310.

The communication device320communicates via the communication network NW with external devices such as the shovel100and the flying body200. The communication device320is connected to the control device310, and various types of information received from the outside are input to the control device310.

The operation input device330receives operations input by, for example, the operator or the administrator of the management device300, and outputs signals corresponding to the input operations. The operation input device330is connected to the control device310, and the signals corresponding to the input operations are input to the control device310.

The display device340is, for example, a liquid crystal display or an organic EL display and displays various information images under the control of the control device310.

[Examples of Machine Learning Function and Operation Support Function]

Next, examples of the machine learning function and the operation support function in the operation support system SYS are described with reference toFIG.3andFIG.4.

FIG.3is a block diagram illustrating an example of a functional configuration related to the machine learning function and the operation support function in the operation support system SYS. Specifically,FIG.3is a block diagram illustrating an example of a functional configuration of the machine learning function and the operation support function in the management device300(the control device310).

The information acquirer3101(an example of a result information acquisition unit) obtains uploaded data such as operation pattern result information and environmental condition result information from one or more shovels100and work site images from one or more flying bodies200. Then, the information acquirer3101stores the operation pattern result information, the environmental condition result information, and the work site images in a shovel operation related information data base (DB)3100A in the storage3100to enable data extraction such that these information items are organized for each operation of the of shovel100and each operation type. The information acquirer3101may be configured to extract, from the work site images, only work site images obtained at a date and time near the date and time of the operation pattern result information and the environmental condition result information uploaded from the shovel100at the corresponding work site, and incorporate the extracted work site images in the environmental condition result information. Also, the information acquirer3101may be configured to access a server or a website related to weather information via the communication device320, obtain weather information at the same date and time as the operation pattern result information and the environmental condition information uploaded from the shovel100, and incorporate the weather information in the environmental condition result information. Hereafter, the set of operation pattern result information and environmental condition result information stored in the shovel operation related information DB3100A for each operation is referred to as “shovel operation related information” for descriptive purposes.

Based on the operation pattern result information and the environmental condition result information for each operation stored in the shovel operation related information DB3100A, the operation pattern generator3102(an example of a generator) generates an optimum operation pattern having a relatively high rating regarding the target index for each target operation type and for each target index, i.e., an optimum operation pattern with the maximized rating for each of different environmental conditions.

The operation pattern generator3102includes an operation evaluation unit3102A, a supervised learning unit3102B, a reinforcement learning unit3102C, and a simulator3102D.

The operation evaluation unit3102A extracts, for each target index, shovel operation related information with a relatively high rating regarding the target index, specifically, with a rating greater than or equal to a predetermined standard, from sets of shovel operation related information stored in the shovel operation related information DB3100A for respective operations. Specifically, feature values related to the rating are defined for each target index, and the operation evaluation unit3102A extracts the feature values from the shovel operation related information and evaluates the shovel operation related information based on the extracted feature values. For example, when the target index is the operation speed in the loading operation, the feature values may include a rotation speed, a loading amount, an excavation path (depth, position, and length), an angle in the middle of excavation, a boom raising position, and a bucket path (during lifting rotation, lowering rotation, soil removing, and suspension movement), an engine speed, and a pump horsepower. Also, when the lifetime of an attachment in the excavation operation is the target index, the feature values may include an insertion angle of the tip of the bucket6and the magnitude of an excavation force. The operation evaluation unit3102A stores the extracted shovel operation related information as training data for supervised learning in the training DB3100B in which data is organized for respective operation types and target indices for data extraction.

The supervised learning unit3102B performs known machine learning (supervised learning) for each work type and each target index based on the training data stored in the training DB3100B, and generates an operation pattern (“supervised learning operation pattern”) with a relatively-high rating regarding the target index for each of multiple different environmental conditions. In this process, a function of the operation evaluation unit3102A may be used to determine the rating regarding the target index. The same applies to the reinforcement learning unit3102C. The generated supervised learning operation patterns are stored in the supervised learning operation pattern DB3100C in which data is organized so as to be extractable for each environmental condition.

The reinforcement learning unit3102C performs reinforcement learning for each operation type and each target index based on a predetermined evaluation condition and the supervised learning operation patterns stored in the supervised learning operation pattern DB3100C for the different environmental conditions. Then, the reinforcement learning unit3102C generates an operation pattern (optimum operation pattern) with a higher target index for each of the different environmental conditions. Specifically, the reinforcement learning unit3102C causes the simulator3102D to repeatedly perform the simulation of a target-type operation, autonomously selects operation patterns with higher rewards, i.e., with higher ratings related to the target index, and finally generates an optimum operation pattern under a certain environmental condition. Also, the reinforcement learning unit3102C can generate an optimum operation pattern under an environmental condition that is not included in the supervised learning operation patterns by using the simulator3102D. The generated optimum operation patterns are stored in the optimum operation pattern DB3100D in which data is organized for respective operation types and environmental conditions for data extraction.

The simulator3102D can perform, for each operation type, a simulation of movements included in an operation pattern of the shovel100based on input conditions such as environmental conditions, operation conditions, and the operation pattern. With this configuration, the simulator3102D can generate operation patterns. Therefore, the reinforcement learning unit3102C can perform not only reinforcement learning based on the past operation patterns (operation pattern result information) obtained by the information acquirer3101, but also reinforcement learning based on information on operation patterns newly generated by the simulator3102D.

For example,FIG.4is a drawing illustrating an example of an operation simulation of the shovel100in an excavation operation by the simulator3102D.

As illustrated inFIG.4(A) throughFIG.4(D), in the operation simulation in the excavation operation, the simulator3102D simulates, based on input conditions, a series of movements where the shovel100places the bucket6on the ground and causes the bucket6to hold a load such as soil by pulling the bucket6.

In this case, the simulator3102D performs simulations by generating multiple operation setting conditions by changing operation settings such as the tip angle (an insertion angle and an angle in the middle of excavation) of the bucket6, an excavation path (depth, position, and length), a boom raising position, an engine speed, and a pump horsepower. As a result, the simulator3102D obtains feature values and target indices for the respective operation setting conditions. In this way, the simulator3102D can generate virtual operation pattern information (which is hereafter referred to as “operation pattern virtual information”). Then, the simulator3102D inputs the operation pattern virtual information to the reinforcement learning unit3102C, and the reinforcement learning unit3102C can obtain an optimum operation pattern.

Although the present embodiment includes a case where operation pattern result information from the information acquirer3101is used, the operation pattern result information is not necessarily used. That is, the operation pattern generator3102may obtain an optimum operation pattern based only on the operation pattern virtual information generated by the simulator3102D.

In response to an acquisition request signal received from the shovel100via the communication device320, the optimum controller3103(an example of an environmental information acquisition unit and an output unit) outputs an optimum operation pattern with which a target index for a type of operation specified by the acquisition request signal becomes relatively high (maximized) under the current environmental condition of the shovel100. The target index may be specified in advance or may be specified by the acquisition request signal sent from the shovel100. Specifically, the optimum controller3103outputs an optimum operation pattern under the current environmental condition of the shovel100based on optimum operation patterns for multiple different environmental conditions stored in the optimum operation pattern DB3100D.

For example, the optimum controller3103extracts, from the optimum operation pattern DB3100D, an optimum operation pattern that matches the current environmental condition of the shovel100(specifically, the environmental condition corresponding to current environmental condition information included in the acquisition request signal), and outputs the optimum operation pattern. Also, when the optimum operation pattern DB3100D does not include an optimum operation pattern that matches the current environmental condition of the shovel100, the optimum controller3103may extract one or more optimum operation patterns corresponding to an environmental condition that is relatively close to the current environmental condition of the shovel100. The optimum controller3103may be configured to apply a correction to the extracted one or more optimum operation patterns based on a difference between the environmental condition corresponding to the extracted optimum operation patterns and the current environmental condition of the shovel100and thereby output an optimum operation pattern corresponding to the current environmental condition of the shovel100.

Also, for example, the optimum controller3103may be configured to autonomously output a unique optimum operation pattern that maximizes the target index under the current environment of the shovel100based on multiple optimum operation patterns stored in the optimum operation pattern DB3100D by using the simulator3102D according to a method (algorithm) similar to that of the reinforcement learning unit3102C. In other words, the optimum controller3103may be comprised mainly of an artificial intelligence (AI) that autonomously outputs an optimum operation pattern, which maximizes the target index under the current environment of the shovel100, based on multiple optimum operation patterns stored in the optimum operation pattern DB3100D by using the simulator3102D. With this configuration, the optimum controller3103can output an optimum operation pattern with a higher rating regarding the target index without using a method such as correction even when the optimum operation pattern DB3100D does not include an optimum operation pattern corresponding to the current environment of the shovel100.

The optimum controller3103sends the output optimum operation pattern via the communication device320to the shovel100that has sent the acquisition request signal.

Also, the optimum controller3103feeds back or adds the output optimum operation pattern to the optimum operation pattern DB3100D and thereby updates the optimum operation pattern DB3100D. As a result, for example, an operation pattern corresponding to a new environmental condition may be added to the optimum operation pattern DB3100D, or an operation pattern in the optimum operation pattern DB3100D may be updated to an operation pattern with a higher rating regarding the target index. Accordingly, the optimum controller3103becomes able to output an optimum operation pattern with a higher rating regarding the target index as the optimum operation pattern DB3100D is updated.

The series of operations of the operation pattern generator3102described above are also repeatedly performed as new information is obtained from, for example, the shovel100by the information acquirer3101and the shovel operation related information DB3100A is updated. Therefore, the optimum operation pattern DB3100D is also updated as a result of the series of operations of the operation pattern generator3102. Accordingly, the optimum controller3103becomes able to output an optimum operation pattern with a higher rating regarding the target index as the optimum operation pattern DB3100D is updated.

[Effects of Operation Support System]

Next, with reference toFIG.5, effects of the operation support system SYS (specifically, the operation support system SYS illustrated inFIG.2andFIG.3) of the present embodiment are described.

FIG.5is a drawing for explaining the effects of the operation support system SYS. Specifically,FIG.5compares the lifetime of an attachment when the shovel100is operated by the machine control function using the optimum operation patterns output from the operation support system SYS (the management device300) of the present embodiment with the lifetime of the attachment when the shovel100is manually operated by operators (a novice operator and a skilled operator).

InFIG.5, the length of each vertical bar indicates a variation range, and each black dot indicates an average value.

As described above, the operation support system SYS (the management device300) outputs an optimum operation pattern with a relatively high (maximized) rating regarding a target index under the current environmental condition of shovel100to support the operation of the shovel100. For example, the operation support system SYS (the management device300) outputs an optimum operation pattern for a loading operation performed by the shovel100by using a soil mound in an actual work site as the current environmental condition and an operation speed as the target index to cause the shovel100to perform a series of optimum operations including holding soil, rotating the upper rotating body3, and removing the soil. Also, for example, for a shovel100where a crack is found in an attachment, the operation support system SYS may output an operation pattern that slows the crack growth to slow down the crack growth in the shovel100. That is, the operation support system SYS (the management device300) of the present embodiment can cause the shovel100to perform a given type of operation according to an operation pattern having a relatively high rating regarding a target index regardless of the operation skill level of the operator. Therefore, the operation support system SYS can improve the operation efficiency, the energy efficiency (fuel efficiency), and the durability of a shovel. Also, the operation support system SYS can assist the operator in an operation where, for example, an excavation operation, a loading operation, and a compaction operation are repeated. This in turn makes it possible to alleviate the fatigue of the operator. Also, when the slowness of the growth of a crack formed in an attachment of the shovel100is used as the target index, the operation support system SYS can slow down the growth of the crack as much as possible and buy time and can therefore prevent a situation where actual work at a work site is stopped in order to assess the state of the crack and repair the crack.

Specifically, as illustrated inFIG.5, when, for example, the target index is the lifetime of an attachment, the lifetime of the attachment is short and varies widely in the case of a novice operator.

In the case of a skilled operator, the lifetime of the attachment is longer than that in the case of the novice operator, and the variation of the lifetime is smaller than that in the case of the novice operator. However, due to manual operations, a certain degree of variation may still occur.

When the shovel100is operated by the machine control function using a supervised learning operation pattern, the average lifetime of the attachment increases and the variation becomes considerably small. However, because training data is based on past records, the lifetime cannot exceed the maximum lifetime in the case of the skilled operator.

On the other hand, when the shovel100is operated by the machine control function using an optimum operation pattern output by the operation support system SYS (the management device300) of the present embodiment, both of the average value and the maximum value of the lifetime of the attachment become greater than those achieved in the case where the supervised learning operation pattern is used.

As described above, the operation support system SYS (the management device300) updates the optimum operation pattern DB3100D. Accordingly, as illustrated inFIG.5, when the shovel100is operated by the machine control function using an optimum operation pattern based on the updated optimum operation pattern DB, the average lifetime of the attachment can be further increased and the variation can be further reduced compared with the case before the update.

[Other Examples of Machine Learning Function and Operation Support Function]

Next, other examples of a machine learning function and an operation support function in the operation support system SYS are described with reference toFIG.6andFIG.7.

FIG.6is a block diagram illustrating another example of a functional configuration related to the machine learning function and the operation support function in the operation support system SYS. Specifically,FIG.6is a block diagram illustrating another example of a functional configuration related to the machine learning function and the operation support function in the management device300(the control device310).

In this example, the management device300(the control device310) generates an optimum arrangement pattern in addition to an optimum operation pattern. Here, an “arrangement” indicates a combination of operation contents (operation patterns), and an “operation content” indicates a combination of movement patterns. Specifically, an “arrangement” is a combination of operation patterns determined taking into account, for example, a construction order (an excavation position, an excavation amount, a temporary yard position, a temporary placement amount, a slope position, etc.), a turning position of a dump truck, the number of turns made by a dump truck, the number of dump trucks, the number of construction machines (e.g., shovels), a soil property, and presence/absence and positions of buried objects at a construction site. That is, the management device300(the control device310) generates an optimum arrangement pattern and optimum operation patterns corresponding to respective operation patterns included in the optimum arrangement pattern. Hereafter, the “arrangement pattern” and the “operation pattern” are collectively referred to as an “arrangement-operation pattern”. Also, in this example, the arrangement pattern and the optimum arrangement pattern are mainly described, and repeated descriptions of the operation pattern and the optimum operation pattern may be omitted.

The control device310includes an information acquirer3101X, an arrangement-operation pattern generator3102X, and an optimum controller3103X as functional units. Also, the control device310uses a storage3100X. The storage3100X includes a shovel arrangement-operation related information DB3100AX, a training DB3100BX, a supervised learning arrangement-operation pattern DB3100CX, and an optimum arrangement-operation pattern DB3100DX.

The information acquirer3101X obtains uploaded data such as operation pattern result information and environmental condition result information from one or more shovels100and work site images from one or more flying bodies200. Also, the information acquirer3101X generates result information related to arrangements (which is hereafter referred to as “arrangement result information”) from the obtained operation pattern result information. The information acquirer3101X may obtain arrangement result information from one or more shovels100. Then, the information acquirer3101X stores (registers) the operation pattern information, the environmental condition result information, the arrangement result information, and the work site images in the shovel arrangement-operation related information DB constructed in the storage3100X. The shovel arrangement-operation related information DB is organized such that data can be extracted for each arrangement constituted by a series of operation contents of the shovel100and for each (type of) combination of operation contents constituting the arrangement. The shovel arrangement-operation related information data base (DB)3100AX in the storage3100X stores the operation pattern result information, the environmental condition result information, and the work site images. Hereafter, a set of arrangement result information and environmental condition result information stored for each arrangement in the shovel arrangement-operation related information DB3100AX is referred to as “shovel arrangement related information” for descriptive purposes.

Based on the shovel arrangement related information stored in the shovel arrangement-operation related information DB3100AX, the arrangement-operation pattern generator3102X generates an optimum arrangement pattern with a relatively-high rating regarding a target index for each (type of) combination of operation contents and for each target index, i.e., an optimum arrangement pattern with the maximized rating for each of different environmental conditions.

The arrangement-operation pattern generator3102X includes an arrangement-operation evaluation unit3102AX, a supervised learning unit3102BX, a reinforcement learning unit3102CX, and a simulator3102DX.

The arrangement-operation evaluation unit3102AX extracts, for each target index, shovel arrangement related information whose rating regarding the target index is relatively high or greater than or equal to a predetermined standard from sets of shovel arrangement related information stored in the shovel arrangement-operation related information DB3100AX. Specifically, feature values related to the rating are defined for each target index, and the arrangement-operation evaluation unit3102AX extracts the feature values from the shovel arrangement related information and evaluates the shovel arrangement related information based on the extracted feature values. For example, target indices related to an arrangement may include (shortness of) an operation time, (smallness of) the number of workers, (smallness of) the amount of necessary fuel, and (smallness of) the amount of CO2 emission. Also, feature values related to an arrangement may include the number of excavations, the number of rotations, a rotation angle, the amount of soil for each excavation, and the amount of soil for each loading. The arrangement-operation evaluation unit3102AX stores the extracted shovel arrangement related information in the training DB3100BX as training data for supervised learning. In the training DB3100BX, training data (shovel arrangement related information) is organized for each (type of) combination of operation contents and for each target index such that data is extractable.

The supervised learning unit3102BX performs known machine learning (supervised learning) for each (type of) combination of operation contents and for each target index based on the training data stored in the training DB3100BX. Then, the supervised learning unit3102BX generates, as a result of the supervised learning, an arrangement pattern (which is hereafter referred to as a “supervised learning arrangement pattern”) having a relatively high rating regarding the target index for each of multiple different environmental conditions. A function of the arrangement-operation evaluation unit3102AX may be used to determine the rating regarding the target index. The same applies to the reinforcement learning unit3102CX. The generated supervised learning arrangement patterns are stored in the supervised learning arrangement-operation pattern DB3100CX in which data is organized for each environmental condition such that the data is extractable.

The reinforcement learning unit3102CX performs reinforcement learning for each (type of) combination of operation contents and for each target index based on the supervised learning arrangement patterns stored in the supervised learning arrangement-operation pattern DB3100CX for the different environmental conditions and generates an operation pattern (optimum operation pattern) with a higher target index for each of the different environmental conditions. Specifically, the reinforcement learning unit3102CX causes the simulator3102DX to repeatedly perform a simulation on an arrangement with a target combination, autonomously selects operation patterns with higher rewards, i.e., with higher ratings regarding the target index, and finally generates an optimum operation pattern under a certain environmental condition. Also, the reinforcement learning unit3102CX can generate an optimum operation pattern under an environmental condition that is not included in the supervised learning arrangement patterns by using the simulator3102DX. The generated optimum operation patterns are stored in the optimum arrangement-operation pattern DB3100DX in which data is organized for each (type of) combination of operation contents and for each environmental condition such that the data is extractable.

The simulator3102DX can perform an operation simulation related to an arrangement of the shovel100for each (type of) combination of operation contents based on input conditions such as environmental conditions, construction conditions, and the arrangement pattern. With this configuration, the simulator3102DX can generate arrangement patterns. Therefore, the reinforcement learning unit3102CX can perform not only reinforcement learning based on past arrangement patterns (arrangement result information) obtained by the information acquirer3101X but also reinforcement learning based on information on arrangement patterns newly generated by the simulator3102DX.

For example,FIG.7is a drawing for explaining an example of an operation simulation of the shovel100regarding an arrangement of a construction site by the simulator3102DX.

As illustrated inFIG.7, in this example, a simulation related to an arrangement by the shovel100at a work site710along a general road720is performed.

In the work site710, a slope SL for carrying out soil to the general road720is formed, and a dump truck DT enters the work site710to carry out soil. Also, an excavation area711is set in the work site710, and temporary yards712and713for soil are set around the excavation area711. Further, another excavation area714is set in the work site710, and a temporary yard715for soil is set near the excavation area714.

In this example, the simulator3102DX simulates, under these construction conditions, an arrangement corresponding to a combination of an excavation operation on the excavation area711, a soil removing operation for moving soil to the temporary yards712and713, and a loading operation for loading the soil in the temporary yards712and713onto the dump truck DT. Also, under the same construction conditions, the simulator3102DX simulates an arrangement corresponding to a combination of an excavation operation on the excavation area714, a soil removing operation for moving soil to the temporary yard715, and a loading operation for loading the soil in the temporary yard715onto the dump truck DT.

In this case, the simulator3102DX generates, for example, multiple different operation setting conditions including the position, the orientation, the movement path, and the order of operations of the shovel100to perform the simulation. As a result, the simulator3102DX obtains feature values and a target index for each operation setting condition. In this manner, the simulator3102DX can generate virtual arrangement pattern information (which is hereafter referred to as “arrangement pattern virtual information”). Then, the simulator3102DX inputs the arrangement pattern virtual information to the reinforcement learning unit3102CX, and the reinforcement learning unit3102CX can obtain an optimum arrangement pattern. When a simulation is performed, the position of the general road720is set as an unchangeable element, and the temporary yards712and713and the excavation area714are set as changeable elements.

The present embodiment includes an example in which arrangement pattern result information based on information acquired by the information acquirer3101X is used. However, the arrangement pattern result information is not necessarily used. That is, the arrangement-operation pattern generator3102X can also obtain an optimum arrangement pattern based only on arrangement pattern virtual information generated by the simulator3102DX.

In response to an acquisition request signal received from the shovel100via the communication device320, the optimum controller3103X outputs an optimum arrangement pattern with which the target index becomes relatively high (or is maximized) under the current environmental condition of the shovel100for a combination of operation contents of a type specified by the acquisition request signal. The target index may be predetermined or may be specified by the acquisition request signal sent from the shovel100. Specifically, the optimum controller3103X outputs an optimum operation pattern under the current environmental condition of the shovel100based on optimum operation patterns stored in the optimum arrangement-operation pattern DB3100DX for the different environmental conditions.

For example, the optimum controller3103X extracts an optimum arrangement pattern that matches the current environmental condition of the shovel100(specifically, an environmental condition corresponding to current environmental condition information included in the acquisition request signal) from the optimum arrangement-operation pattern DB3100DX and outputs the extracted optimum arrangement pattern. When the optimum arrangement pattern matching the current environmental condition of the shovel100is not included in the optimum arrangement-operation pattern DB3100DX, the optimum controller3103X may extract one or more optimum arrangement patterns corresponding to environmental conditions that are relatively close to the current environmental condition of the shovel100. Then, the optimum controller3103X may apply a predetermined correction to the extracted one or more optimum arrangement patterns based on differences between the environmental conditions corresponding to the extracted optimum arrangement patterns and the current environmental condition of the shovel100and thereby output an optimum arrangement pattern corresponding to the current environmental condition of the shovel100.

Also, for example, the optimum controller3103X may autonomously output a unique optimum arrangement pattern that maximizes the target index under the current environment of the shovel100based on multiple optimum arrangement patterns stored in the optimum arrangement-operation pattern DB3100DX by using the simulator3102DX according to a method (algorithm) similar to that of the reinforcement learning unit3102CX. That is, the optimum controller3103X may be comprised mainly of an artificial intelligence that autonomously outputs an optimum arrangement pattern that maximizes the target index under the current environment of the shovel100by using the simulator3102DX based on multiple optimum arrangement patterns stored in the optimum arrangement-operation pattern DB3100DX. With this configuration, the optimum controller3103X can output an optimum arrangement pattern with a higher rating regarding the target index without using a method such as correction even when the optimum arrangement-operation pattern DB3100DX does not include an optimum arrangement pattern corresponding to the current environment of the shovel100.

The optimum controller3103X sends the output optimum arrangement pattern via the communication device320to the shovel100that has sent the acquisition request signal.

In addition, the optimum controller3103X feeds back or adds the output optimum arrangement pattern to the optimum arrangement-operation pattern DB3100DX and thereby updates the optimum arrangement-operation pattern DB3100DX. As a result, an operation pattern corresponding to a new environmental condition may be added to the optimum arrangement-operation pattern DB3100DX and an operation pattern in the optimum arrangement-operation pattern DB3100DX may updated to an operation pattern with a higher rating regarding the target index. Thus, the optimum controller3103X can output an optimum arrangement pattern with a higher rating regarding the target index as the optimum arrangement-operation pattern DB3100DX is updated.

Also, the series of operations of the arrangement-operation pattern generator3102X are also repeatedly performed as new information is obtained by the information acquirer3101X from, for example, the shovel100and the shovel arrangement-operation related information DB3100AX is updated. Accordingly, the optimum arrangement-operation pattern DB3100DX is also updated by the series of operations of the arrangement-operation pattern generator3102X. Thus, the optimum controller3103X can output an optimum arrangement pattern with a higher rating regarding the target index as the optimum arrangement-operation pattern DB3100DX is updated.

[Still Other Examples of Machine Learning Function and Operation Support Function]

Next, with reference toFIG.8andFIG.9, still other examples of the machine learning function and the operation support function in the operation support system SYS are described.

FIG.8andFIG.9are block diagrams illustrating still other examples of functional configurations related to the machine learning function and the operation support function in the operation support system SYS. Specifically,FIG.8is a block diagram illustrating a configuration of the operation support system SYS according to this example, andFIG.9is a block diagram illustrating components of the shovel100in the configuration of the operation support system SYS that are not illustrated inFIG.8.

In this example, the machine learning function and the operation support function are provided in the shovel100. Below, components unique to this example are mainly described, and repeated descriptions may be omitted.

The controller30of the shovel100includes, as functional units implemented by executing one or more programs installed in the secondary storage on the CPU, a current ground shape acquirer F1, a target ground shape acquirer F2, a comparison unit F3, an operation start determining unit F4, an arrangement—operation setting unit F5, a movement determining unit F6, a movement command generator F7, a movement limiting unit F8, a command value calculator F9, a current bucket position calculator F10, a boom current command generator F11, a boom spool displacement calculator F12, a boom angle calculator F13, an arm current command generator F21, an arm spool displacement calculator F22, an arm angle calculator F23, a bucket current command generator F31, a bucket spool displacement calculator F32, a bucket angle calculator F33, a rotation current command generator F41, a rotation spool displacement calculator F42, and a rotation angle calculator F43.

The current ground shape acquirer F1(an example of an environmental information acquisition unit) obtains information (e.g., three-dimensional data such as three-dimensional dots and surfaces) on a current ground shape (“current ground shape”) around the shovel100based on images captured by the imaging device80.

The target ground shape acquirer F2obtains a ground shape (for example, a target construction surface) (which is hereafter referred to as a “target ground shape”) that is to be formed in a construction site.

The comparison unit F3compares the current ground shape with the target ground shape, and outputs information on a difference (hereafter, “difference information”) between the current ground shape and the target ground shape to a learning unit F100.

The operation start determining unit F4determines the start of an operation based on a command received from the management device300via the communication device T1.

The arrangement-operation setting unit F5sets an arrangement in the work site and operations included in the arrangement according to a command received from the management device300via the communication device T1. The arrangement and the operations to be set are input to the learning unit F100and the movement determining unit F6.

The movement determining unit F6determines movements according to the arrangement and the operations set by the arrangement-operation setting unit F5in response to a command from the learning unit F100. The determined movements are input to the learning unit F100and the movement command generator F7.

The movement command generator F7(an example of a controller) generates movement commands for the shovel100, i.e., movement commands for actuators for driving driven components of the shovel100based on a command from the learning unit F100, the movements determined by the movement determining unit F6, and the current position (hereafter, “bucket current position”) of a working part (e.g., the tip or the back side) of the bucket6calculated by the current bucket position calculator F10. The generated movement commands are input to the learning unit F100and the movement limiting unit F8.

The movement limiting unit F8limits the movements (including stopping the movements) of the shovel100corresponding to the movement commands generated by the movement command generator F7according to predetermined movement limiting conditions. The movement limiting conditions may include, for example, “a possibility that a part other than the working part of the shovel100comes into contact with a surrounding object as a result of an movement of the shovel100corresponding to a movement command”. Also, the movement limiting conditions may include, for example, “the angular velocity of the motion axis of an attachment exceeds an allowable range as a result of a movement of the shovel100corresponding to a movement command”. Specifically, when a movement limiting condition is satisfied, the movement limiting unit F8corrects the movement command generated by the movement command generator F7such that the movement of the shovel100is restricted and outputs the corrected movement command to the command value calculator F9. On the other hand, when no movement limiting condition is satisfied, the movement limiting unit F8outputs the movement command generated by the movement command generator F7to the command value calculator F9without change.

Based on movement commands or corrected movement commands input from the movement limiting unit F8, the command value calculator F9outputs command values to the respective driven components (the boom4, the arm5, the bucket6, the upper rotating body3, and the right and left crawlers of the lower traveling body1). Specifically, the command value calculator F9outputs a boom command value α * for the boom4, an arm command value β * for the arm5, a bucket command value γ * for the bucket6, a rotation command value δ* for the upper rotating body3, a right-travel command value ε1* for the right crawler, and a left-travel command value ε2* for the left crawler.

The current bucket position calculator F10calculates the current position (current bucket position) of the working part of the bucket6. Specifically, the current bucket position calculator F10calculates the current bucket position based on a boom angle α, an arm angle β, a bucket angle γ, a right drive wheel rotation angle ε1, and a left drive wheel rotation angle ε2that are fed back from the boom angle calculator F13, the arm angle calculator F23, the bucket angle calculator F33, and the rotation angle calculator F43.

The boom current command generator F11outputs a boom current command to a boom proportional valve31A.

The boom spool displacement calculator F12calculates the amount of displacement of the boom spool constituting the control valve175corresponding to the boom cylinder7based on an output from the boom spool displacement sensor S16.

The boom angle calculator F13calculates the boom angle α based on an output from the boom angle sensor S11.

The boom current command generator F11basically generates a boom current command for the boom proportional valve31A such that the difference between the boom command value α * generated by the command value calculator F9and the boom angle α calculated by the boom angle calculator F13becomes zero. In this process, the boom current command generator F11adjusts the boom current command such that the difference between a target boom spool displacement amount derived from the boom current command and a boom spool displacement amount calculated by the boom spool displacement calculator F12becomes zero. Then, the boom current command generator F11outputs the adjusted boom current command to the boom proportional valve31A.

The boom proportional valve31A changes the opening area according to the boom current command, and applies a pilot pressure corresponding to the magnitude of the boom command current to the pilot port of the control valve175. The control valve175moves the boom spool according to the pilot pressure and causes hydraulic oil to flow into the boom cylinder7. The boom spool displacement sensor S16detects the displacement of the boom spool and feeds back the detection result to the boom spool displacement calculator F12of the controller30. The boom cylinder7expands and contracts in response to the inflow of the hydraulic oil and thereby moves the boom4up and down. The boom angle sensor S11detects the rotation angle of the boom4moving up and down, and feeds back the detection result to the boom angle calculator F13of the controller30. The boom angle calculator F13feeds back the calculated boom angle α to the current bucket position calculator F10.

The arm current command generator F21outputs an arm current command to the arm proportional valve31B.

The arm spool displacement calculator F22calculates the amount of displacement of the arm spool constituting the control valve176corresponding to the arm cylinder8based on an output from the arm spool displacement sensor S17.

The arm angle calculator F23calculates the arm angle β based on an output from the arm angle sensor S12.

The arm current command generator F21basically generates an arm current command for the arm proportional valve31B such that the difference between the arm command value β * generated by the command value calculator F9and the arm angle β calculated by the arm angle calculator F23becomes zero. In this process, the arm current command generator F21adjusts the arm current command such that the difference between a target arm spool displacement amount derived from the arm current command and an arm spool displacement amount calculated by the arm spool displacement calculator F22becomes zero. Then, the arm current command generator F21outputs the adjusted arm current command to the arm proportional valve31B.

The arm proportional valve31B changes the opening area according to the arm current command, and applies a pilot pressure corresponding to the magnitude of the arm command current to the pilot port of the control valve176. The control valve176moves the arm spool according to the pilot pressure and causes hydraulic oil to flow into the arm cylinder8. The arm spool displacement sensor S17detects the displacement of the arm spool and feeds back the detection result to the arm spool displacement calculator F22of the controller30. The arm cylinder8expands and contracts in response to the inflow of the hydraulic oil and thereby opens and closes the arm5. The arm angle sensor S12detects the rotation angle of the opening and closing arm5, and feeds back the detection result to the arm angle calculator F23of the controller30. The arm angle calculator F23feeds back the calculated arm angle β to the current bucket position calculator F10.

The bucket current command generator F31outputs a bucket current command to the bucket proportional valve31C.

The bucket spool displacement calculator F32calculates the amount of displacement of the bucket spool constituting the control valve174corresponding to the bucket cylinder9based on the output from the bucket spool displacement sensor S18.

The bucket angle calculator F33calculates the bucket angle γ based on an output from the bucket angle sensor S13.

The bucket current command generator F31basically generates a bucket current command for the bucket proportional valve31C such that the difference between the bucket command value γ * generated by the command value calculator F9and the bucket angle γ calculated by the bucket angle calculator F33becomes zero. In this process, the bucket current command generator F31adjusts the bucket current command such that the difference between a target bucket spool displacement amount derived from the bucket current command and a bucket spool displacement amount calculated by the bucket spool displacement calculator F32becomes zero. Then, the bucket current command generator F31outputs the adjusted bucket current command to the bucket proportional valve31C.

The bucket proportional valve31C changes the opening area according to the bucket=rent command, and applies a pilot pressure corresponding to the magnitude of the bucket command current to the pilot port of the control valve174. The control valve174moves the bucket spool according to the pilot pressure and causes hydraulic oil to flow into the bucket cylinder9. The bucket spool displacement sensor S18detects the displacement of the bucket spool and feeds back the detection result to the bucket spool displacement calculator F32of the controller30. The bucket cylinder9expands and contracts according to the inflow of the hydraulic oil and thereby opens and closes the bucket6. The bucket angle sensor S13detects the rotation angle of the opening and closing bucket6, and feeds back the detection result to the bucket angle calculator F33of the controller30. The bucket angle calculator F33feeds back the calculated bucket angle γ to the current bucket position calculator F10.

The rotation current command generator F41outputs a rotation current command to the rotation proportional valve31D.

The rotation spool displacement calculator F42calculates the amount of displacement of the rotation spool constituting the control valve173corresponding to the rotation hydraulic motor2A based on an output from the rotation spool displacement sensor S19.

The rotation angle calculator F43calculates the rotation angle δ based on an output from the rotation angle sensor S15.

The rotation current command generator F41basically generate a rotation current command for the rotation proportional valve31D such that the difference between the rotation command value δ * generated by the command value calculator F9and the rotation angle δ calculated by the rotation angle calculator F43becomes zero. In this process, the rotation current command generator F41adjusts the rotation current command such that the difference between a target rotation spool displacement amount derived from the rotation current command and a rotation spool displacement amount calculated by the rotation spool displacement calculator F42becomes zero. Then, the rotation current command generator F41outputs the adjusted rotation current command to the rotation proportional valve31D.

The rotation proportional valve31D changes the opening area according to the rotation current command, and applies a pilot pressure corresponding to the magnitude of the rotation command current to the pilot port of the control valve173. The control valve173moves the rotation spool according to the pilot pressure, and causes hydraulic oil to flow into the rotation hydraulic motor2A. The rotation spool displacement sensor S19detects the displacement of the rotation spool and feeds back the detection result to the rotation spool displacement calculator F42of the controller30. The rotation hydraulic motor2A rotates according to the inflow of the hydraulic oil and thereby causes the upper rotating body3to rotate. The rotation angle sensor S15detects the rotation angle of the rotating upper rotating body3and feeds back the detection result to the rotation angle calculator F43of the controller30. The rotation angle calculator F43feeds back the calculated rotation angle δ to the current bucket position calculator F10.

Each of the right crawler and the left crawler of the lower traveling body1also has a feedback loop similar to those of other driven components (operating bodies) such as the boom4, the arm5, the bucket6, and the upper rotating body3. That is, feedback loops are formed based on inputs of the right travel command value ε1* and the left travel command value ε2* generated by the command value calculator F9. The feedback loops feed back the right drive wheel rotation angle ε1and the left drive wheel rotation angle ε2, which indicate the rotation positions (rotation angles) of the drive wheels of the right crawler and the left crawler, to the current bucket position calculator F10.

Thus, the controller30provides a three-stage feedback loop for each driven component (operating body). That is, the controller30forms feedback loops for spool displacement amounts, feedback loops for rotation angles of driven components (operating bodies), and a feedback loop for the position of the working part (e.g., the position of the tip) of the bucket6. With this configuration, the controller30can accurately control the movement of the working part of the bucket6during automatic control.

The learning unit F100(an example of a result information acquisition unit, a generator, and an output unit) implements a machine learning function and an operation support function. That is, the learning unit F100includes functions similar to the information acquirer3101X, the arrangement-operation pattern generator3102X, and the optimum controller3103X of the management device300(the control device310) described above.

Specifically, unlike the management device300described above, the learning unit F100causes the machine (the shovel100) to perform actual operations and an actual arrangement and performs reinforcement learning based on result information obtained in the actual operations and the actual arrangement. The result information includes result information related to the arrangement, the operations, and the movements of the shovel100that is fed back from the arrangement-operation setting unit55, the movement determining unit F6, and the movement command generator F7. The result information also includes result information related to environmental conditions such as current ground shape information around the shovel100that is input from the current ground shape acquirer F1via the comparison unit F3. The result information further includes result information regarding the results of the arrangement, the operations, and the movements of the shovel100such as difference information from the comparison unit F3. With this configuration, the learning unit F100can generate an operation pattern (optimum operation pattern) and an arrangement pattern (optimum arrangement pattern) with which the target index becomes relatively high for each operation type or each (type of) combination of operation contents and for each environmental condition based on the result information. Then, based on the difference information input from the comparison unit F3, the learning unit F100outputs commands corresponding to the optimum operation pattern and the optimum arrangement pattern under the current environmental condition (i.e., the current ground shape) to the arrangement-operation setting unit F5, the movement determining unit F6, and the movement command generator F7. With this configuration, the controller30(the movement command generator F7) can automatically or semi-automatically control its own machine (the shovel100) based on the optimum operation pattern and the optimum arrangement pattern.

[Effects of Operation Support System]

Next, with reference toFIG.10, the effects of the operation support system SYS (specifically, the operation support system SYS illustrated inFIG.8andFIG.9) of the present embodiment are described.

FIG.10is a drawing for explaining the effects of the operation support system SYS. Specifically,FIG.10is a drawing for explaining the effects of the operation support system SYS (the learning unit F100provided in the shovel100) illustrated inFIG.8and FIG.9.

In this example, the shovel100is performing an operation of constructing a slope (a target construction surface1001) by embankment. Normally, a slope is constructed continuously in the depth direction ofFIG.10. Therefore, the construction area is divided into multiple sections in the traveling direction of the shovel100. When the construction is completed from the foot of the slope to the shoulder of the slope in one section, the shovel100moves to the adjacent section, and the construction of the adjacent section is started.

The shovel100discharges soil held in the bucket6along the slope to fill the difference between the target construction surface1001and the current shape of the slope.

However, in this example, the discharged soil collapses toward the foot of the slope, resulting in a shape (an actual shape1003) different from an expected shape1002. This is caused by, for example, a difference in the soil property or a change in the weather.

Here, the learning unit F100of the shovel100performs reinforcement learning using movements, operations, and an arrangement of this time and their result (the actual shape1003) as result information.

For example, the learning unit F100of the shovel100updates the optimum operation pattern and the optimum arrangement pattern by adding a scooping operation to the soil discharging operation under the same environmental condition. As a result, a scooping operation for recovery is performed, and the soil is formed into the expected shape1002. Then, when the construction in the next section is started, an operation pattern with a high reward under environmental information such as a similar difference in the soil property or similar weather is extracted, and the construction operation is continued based on the extracted operation pattern. Therefore, even in a recovery operation, the recovery can be quickly performed with the reinforcement learning by the learning unit F100. Also, the results in one section can be used in the next section.

Thereafter, under the control of the controller30(the learning unit F100), the shovel100performs the scooping operation together with the soil discharging operation when performing a construction work under a similar environmental condition. Thus, the shovel100(the controller30) can autonomously improve an operation of the shovel100such that the target index (for example, work efficiency) becomes relatively high.

[Variations and Improvements]

Embodiments of the present invention are described above. However, the present invention is not limited to the specifically disclosed embodiments, and variations and modifications may be made without departing from the scope of the present invention.

For example, although the operating device26in the above-described embodiments is a hydraulic device that outputs a pilot pressure according to an operation by the operator, the operating device26may be an electric device that outputs an electric signal. In this case, the control valve system17may include electromagnetic pilot control valves. Also, the controller30can control the machine guidance function or the machine control function by directly determining the operation state based on an electric signal input from an electric operating device.

In the embodiments and variations described above, the shovel100uploads operation pattern result information to the management device300, obtains an optimum operation pattern from the management device300, and performs controls related to the machine control function based on the optimum operation pattern. However, the present invention is not limited to this example. For example, a shovel that uploads operation pattern result information to the management device300may be different from a shovel whose operation is to be supported by the operation support system SYS (the management device300). In this case, the shovel that uploads the operation pattern result information to the management device300does not necessarily include the machine guidance function and the machine control function.

Also, in the embodiments and variations described above, operation pattern result information, environmental condition result information, and work site images are uploaded from the shovel100and the flying body200to the management device300. However, the present invention is not limited to this example. For example, operation pattern result information, environmental condition result information, and work site images recorded in the shovel100and the flying body200may be read and stored in an external storage device outside of the shovel100and the flying body200by, for example, a service person according to a predetermined method. Then, the service person may go to a facility corresponding to the management device300and transfer data such as the operation pattern result information, the environmental condition result information, and the work site images from the storage device to the management device300.

Also, in the embodiments and variations described above, operation pattern result information is formed based on detection information output from the state detection device S1provided in the shovel100. However, the present invention is not limited to this example. For example, the operation pattern result information may be formed based on detection information of a sensor (for example, a camera, a LIDAR, or a millimeter wave radar) that observes the operation of the shovel100from the outside. In this case, the detection information of the sensor may be uploaded, or recorded in a predetermined storage device and transferred to the management device300by a worker who visits a facility corresponding to the management device300.

Also, in the embodiments and variations described above, the function of the optimum controller3103is provided in the management device300. However, the function of the optimum controller3103may be provided in the shovel100. In this case, a data set corresponding to the optimum operation pattern DB3100D is sent from the management device300to the shovel100in advance. Also, when the optimum operation pattern DB3100D is updated at the management device300, an updated data set is sent from the management device300to the shovel100.

Further, in the embodiments and variations described above, the shovel100is configured such that all of the moving components such as the lower traveling body1, the upper rotating body3, the boom4, the arm5, and the bucket6are hydraulically driven. However, some of the moving components may be electrically driven. That is, the configurations described in the above embodiments may also be applied to a hybrid shovel and an electric shovel.