AUTOMATIC TRAVELING SYSTEM, AUTOMATIC TRAVELING METHOD, AND AUTOMATIC TRAVELING PROGRAM

An acquisition processing part acquires a captured image from a camera which is installed on a work vehicle. A detection processing part detects an obstacle on the basis of the captured image which is acquired by the acquisition processing part. When an obstacle is detected by the detection processing part, a reception processing part receives a traveling stop instruction for stopping automatic traveling of the work vehicle or a traveling continuation instruction for continuing automatic traveling of the work vehicle. A traveling processing part stops the automatic traveling of the work vehicle when the reception processing part receives the traveling stop instruction, and continues the automatic traveling of the work vehicle when the reception processing part receives the traveling continuation instruction.

TECHNICAL FIELD

The present invention relates to an automatic traveling system, an automatic traveling method, and an automatic traveling program for causing a work vehicle to automatically travel.

BACKGROUND ART

A work vehicle includes a function of stopping automatic traveling, in a case where the work vehicle detects an obstacle during automatic traveling along a predetermined traveling route in a field. Conventionally, there is known a technique that allows an operator of a work vehicle to switch between a normal mode in which automatic traveling of the work vehicle is stopped in a case where the work vehicle detects an obstacle, and a release mode in which traveling of the work vehicle is continued even when the work vehicle detects an obstacle (see, for example, Patent Literature 1).

CITATION LIST

Patent Literature

DISCLOSURE OF INVENTION

Problems to be Solved by the Invention

However, in the conventional technique, in a case where the release mode is set, automatic traveling is continued, even when the work vehicle detects an obstacle. Therefore, there is a risk that the work vehicle may come into contact with the obstacle. As described above, while, in a system that uniformly stops automatic traveling, in a case where a work vehicle detects an obstacle, work efficiency is lowered, in a system that enables to continue automatic traveling, even when a work vehicle detects an obstacle, safety of the work vehicle may be lowered.

An object of the present invention is to provide an automatic traveling system, an automatic traveling method, and an automatic traveling program that enable to prevent lowering of work efficiency, while securing safety of a work vehicle.

Means for Solving the Problems

An automatic traveling system according to the present invention includes an acquisition processing part, a detection processing part, a reception processing part, and a traveling processing part. The acquisition processing part acquires a captured image from an imaging part installed on a work vehicle. The detection processing part detects an obstacle, based on the captured image to be acquired by the acquisition processing part. The reception processing part receives a traveling stop instruction to stop automatic traveling of the work vehicle, or a traveling continuation instruction to continue automatic traveling, in a case where the obstacle is detected by the detection processing part. The traveling processing part stops automatic traveling of the work vehicle, in a case where the reception processing part receives the traveling stop instruction, and continues automatic traveling of the work vehicle, in a case where the reception processing part receives the traveling continuation instruction.

An automatic traveling method according to the present invention is a method including, by one or more processors: acquiring a captured image from an imaging part installed on a work vehicle; detecting an obstacle, based on the captured image to be acquired; receiving a traveling stop instruction to stop automatic traveling of the work vehicle, or a traveling continuation instruction to continue automatic traveling, in a case where the obstacle is detected; and stopping automatic traveling of the work vehicle, in a case where the traveling stop instruction is received, and continuing automatic traveling of the work vehicle, in a case where the traveling continuation instruction is received.

An automatic traveling program according to the present invention is a program causing one or more processors to execute: acquiring a captured image from an imaging part installed on a work vehicle; detecting an obstacle, based on the captured image to be acquired; receiving a traveling stop instruction to stop automatic traveling of the work vehicle, or a traveling continuation instruction to continue automatic traveling, in a case where the obstacle is detected; and stopping automatic traveling of the work vehicle, in a case where the traveling stop instruction is received, and continuing automatic traveling of the work vehicle, in a case where the traveling continuation instruction is received.

Effect of the Invention

According to the present invention, it is possible to provide an automatic traveling system, an automatic traveling method, and an automatic traveling program that enable to prevent lowering of work efficiency, while securing safety of a work vehicle.

DESCRIPTION OF EMBODIMENTS

The following embodiment is an example embodying the present invention, and is not intended to limit the technical scope of the present invention.

As illustrated inFIG.1, an automatic traveling system1according to an embodiment of the present invention includes a work vehicle10and an operation terminal20. The work vehicle10and the operation terminal20can communicate with each other via a communication network N1. For example, the work vehicle10and the operation terminal20can communicate with each other via a mobile phone network, a packet network, or a wireless LAN.

In the present embodiment, a case is described as an example, in which the work vehicle10is a tractor. Note that, as another embodiment, the work vehicle10may be a rice transplanter, a combine harvester, a construction machine, a snowplow, or the like. The work vehicle10is a so-called robot tractor provided with a configuration capable of automatically traveling (autonomously traveling) along a predetermined traveling route R within a field F (seeFIG.3). For example, the work vehicle10can automatically travel along the traveling route R that is generated in advance within the field F, based on position information on a current position of the work vehicle10to be computed by a positioning device17.

For example, the work vehicle10travels in parallel back and forth from a work start position S to a work end position G in a work area of the field F illustrated inFIG.3. An outer periphery of the field F is, for example, a headland area, and the work vehicle10travels while turning. The traveling route R is not limited to the route illustrated inFIG.3, but is set as appropriate according to a content of work.

As illustrated inFIGS.1and2, the work vehicle10includes a vehicle control device11, a storage part12, a traveling device13, a work machine14, a camera15, a communication part16, the positioning device17, a detection processing device19, and the like. The vehicle control device11is electrically connected to the storage part12, the traveling device13, the work machine14, the positioning device17, the detection processing device19, and the like. Note that, the vehicle control device11and the positioning device17may be wirelessly communicable. The camera15is also electrically connected to the detection processing device19.

The storage part12is a non-volatile storage part such as a hard disk drive (HDD) or a solid state drive (SSD) that stores various pieces of information. The storage part12stores a control program such as an automatic traveling program for causing the vehicle control device11to perform automatic traveling processing to be described later (seeFIG.10). For example, the automatic traveling program is non-transitorily recorded on a computer-readable recording medium such as a CD or a DVD, read by a predetermined reading device (not illustrated), and stored in the storage part12. Note that, the automatic traveling program may be downloaded from a server (not illustrated) to the work vehicle10via the communication network N1, and stored in the storage part12. Also, data on the traveling route R to be generated in the operation terminal20, determination data for determining an obstacle, and the like are stored in the storage part12. The determination data are data for determining a person, which is an example of an obstacle, and, for example, are data indicating a feature of a person, data on an image (sample image) indicating a part or whole of a person, and the like. The determination data may be stored in the detection processing device19. In addition, work information (e.g., a turning pattern, a work order, and the like) may be stored in the storage part12. Note that, the obstacle according to the present invention is an object that interferes with traveling of the work vehicle10, and, for example, is a person, an animal, a device (such as other work vehicles), and the like.

The traveling device13is a driving part that causes the work vehicle10to travel. As illustrated inFIG.2, the traveling device13includes an engine131, front wheels132, rear wheels133, a transmission134, a front axle135, a rear axle136, a steering wheel137, and the like. Note that, the front wheel132and the rear wheel133are provided on each of left and right sides of the work vehicle10. Further, the traveling device13is not limited to a wheel type in which the front wheels132and the rear wheels133are provided, but may be a crawler type in which a crawler is provided on left and right sides of the work vehicle10.

The engine131is a drive source such as a diesel engine or a gasoline engine that is driven by using fuel to be replenished in an unillustrated fuel tank. The traveling device13may include an electric motor as a drive source together with the engine131or in place of the engine131. Note that, the engine131is connected to an unillustrated generator, and electric power is supplied from the generator to an electrical component such as the vehicle control device11provided in the work vehicle10, a battery, and the like. Note that, the battery is charged by electric power to be supplied from the generator. Further, electrical components such as the vehicle control device11and the positioning device17provided in the work vehicle10can also be driven by electric power to be supplied from the battery even after the engine131is stopped.

A driving force of the engine131is transmitted to the front wheels132via the transmission134and the front axle135, and is transmitted to the rear wheels133via the transmission134and the rear axle136. The driving force of the engine131is also transmitted to the work machine14via a PTO shaft (not illustrated). In a case where the work vehicle10performs automatic traveling, the traveling device13performs a traveling operation in accordance with a command of the vehicle control device11.

The work machine14is, for example, a mower, a cultivator, a plow, a fertilizer applicator, a seeder, or the like, and is detachably attachable to the work vehicle10. This allows the work vehicle10to perform various types of work by using each of the work machines14. In the present embodiment, a case is described as an example in which the work machine14is a mower.

For example, the work machine14is mounted in such a way that the work machine14is offset to the left or to the right with respect to the work vehicle10. For example, the work vehicle10performs mowing work by traveling within a field in a state that the directly-mounted work machine14is mounted in such a way the work machine14is offset to the left or to the right. Note that, the work machine14is not limited to a directly-mounted work machine (seeFIG.2) fixed to the work vehicle10, but may also be a towed work machine towed by the work vehicle10.

The camera15is a digital camera that captures an image of a subject, and outputs the image as digital image data. The camera15continuously captures a subject at a predetermined frame rate, generates frame images of a predetermined resolution, and sequentially transmits the frame images to the detection processing device19. The camera15is an example of the imaging part according to the present invention.

In the present embodiment, five cameras15are installed at different locations of the work vehicle10. Specifically, as illustrated inFIG.4, a camera15F (hereinafter, also referred to as “camera1”) is installed on a front portion of the work vehicle10, a camera15B (hereinafter, also referred to as “camera2”) is installed on a rear portion of the work vehicle10, a camera15R (hereinafter, also referred to as “camera3”) is installed on a right side portion of the work vehicle10, a camera15L (hereinafter, also referred to as “camera4”) is installed on a left side portion of the work vehicle10, and a camera15C (hereinafter, also referred to as “camera5”) is installed on a front right portion of the work machine14. Note that, the camera15may be installed on the work vehicle10by, for example, a double-sided tape or the like.

In addition, each camera15has a predetermined imaging range (detection area) capable of imaging. For example, the camera15F captures a detection area K1in front of the work vehicle10, the camera15B captures a detection area K2behind the work vehicle10, the camera15R captures a detection area K3on the right side of the work vehicle10, the camera15L captures a detection area K4on the left side of the work vehicle10, and the camera15C captures a detection area K5on a front right portion of the work machine14. Each camera15captures each detection area at a predetermined frame rate, and sequentially transmits the captured images to the detection processing device19. The detection processing device19transmits the captured images and a detection result (determination result) to be described later to the vehicle control device11and the operation terminal20.

Note that, the detection processing device19becomes communicable with the cameras15when the cameras15are electrically connected to the work vehicle10. When the detection processing device19becomes communicable with the cameras15, the detection processing device19acquires the number of the cameras15, and identification information (device information) of each camera15. The detection processing device19also outputs the acquired information on the number of the cameras15and the acquired identification information to the vehicle control device11, and the vehicle control device11outputs the information on the number of the cameras15and the identification information to the operation terminal20. The operator may add a camera15, or change the installation position.

The steering wheel137is an actuating part to be actuated by the operator or the vehicle control device11. In the traveling device13, for example, an angle of the front wheel132is changed by an unillustrated hydraulic power steering mechanism or the like in response to an actuation of the steering wheel137by the vehicle control device11, and a traveling direction of the work vehicle10is changed.

Further, the traveling device13includes, in addition to the steering wheel137, an unillustrated shift lever, accelerator, brake, and the like to be actuated by the vehicle control device11. In the traveling device13, a gear of the transmission134is switched to a forward gear, a backward gear, or the like in response to an actuation of the shift lever by the vehicle control device11, and a traveling mode of the work vehicle10is switched to forward, backward, and the like. The vehicle control device11also controls a speed of the engine131by actuating the accelerator. The vehicle control device11also actuates the brake, and brakes rotation of the front wheels132and the rear wheels133by using an electromagnetic brake.

The positioning device17is a communication device including a positioning control part171, a storage part172, a communication part173, a positioning antenna174, and the like. For example, as illustrated inFIG.2, the positioning device17is provided on an upper portion of a cabin18in which the operator boards. An installation location of the positioning device17is not limited to the cabin18. Furthermore, the positioning control part171, the storage part172, the communication part173, and the positioning antenna174of the positioning device17may be disposed in a distributed manner at different positions in the work vehicle10. Note that, as described above, the battery is connected to the positioning device17, and the positioning device17is operable even when the engine131is stopped. Also, for example, a mobile phone terminal, a smartphone, a tablet terminal, or the like may be substituted as the positioning device17.

The positioning control part171is a computer system including one or more processors, and a storage memory such as a non-volatile memory and a RAM. The storage part172is a non-volatile memory or the like that stores a program for causing the positioning control part171to perform positioning processing, and data such as positioning information and movement information. For example, the program is non-transitorily recorded in a computer-readable recording medium such as a CD, or a DVD, read by a predetermined reading device (not illustrated), and stored in the storage part172. Note that, the program may be downloaded from a server (not illustrated) to the positioning device17via the communication network N1, and stored in the storage part172.

The communication part173is a communication interface that connects the positioning device17to the communication network N1in a wired or wireless manner, and performs data communication with an external device such as a base station server via the communication network N1in accordance with a predetermined communication protocol.

The positioning antenna174is an antenna that receives a radio wave (a GNSS signal) to be transmitted from a satellite.

The positioning control part171computes a current position of the work vehicle10, based on a GNSS signal to be received from a satellite by the positioning antenna174. For example, in a case where the work vehicle10automatically travels within the field F, and the positioning antenna174receives a radio wave (such as a transmission time, and orbit information) to be transmitted from each of a plurality of satellites, the positioning control part171computes a distance between the positioning antenna174and each satellite, and computes a current position (a latitude and a longitude) of the work vehicle10, based on the computed distance. The positioning control part171also computes a current position of the work vehicle10by utilizing correction information associated with a base station (reference station) near the work vehicle10. Positioning by a real-time kinematic method (RTK-GPS positioning method (RTK method)) may also be performed. In this way, the work vehicle10performs automatic traveling by utilizing positioning information by the RTK method.

The vehicle control device11and the detection processing device19include control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various pieces of arithmetic processing. The ROM is a non-volatile storage part that stores in advance a control program such as a BIOS and an OS for causing the CPU to execute various pieces of arithmetic processing. The RAM is a volatile or non-volatile storage part that stores various pieces of information, and is used as a transitory storage memory (work area) in which the CPU executes various pieces of processing. Further, the vehicle control device11and the detection processing device19control the work vehicle10by causing the CPU to execute various control programs stored in advance in the ROM or the storage part12.

The detection processing device19acquires a captured image from the camera15, and determines whether a detection area includes an obstacle (e.g., a person), based on the captured image. Specifically, as illustrated inFIG.1, the detection processing device19includes various processing parts such as an acquisition processing part111and a detection processing part112. The detection processing device19functions as the various processing parts by causing the CPU to execute various pieces of processing according to the automatic traveling program. In addition, some or all of the processing parts may be constituted of an electronic circuit. Note that, the automatic traveling program may be a program for causing a plurality of processors to function as the processing parts.

The acquisition processing part111acquires a captured image from one or more cameras15. For example, the acquisition processing part111acquires in order a captured image of each of the detection areas K1to K5from each of the five cameras15F,15B,15R,15L, and15C installed on the work vehicle10and the work machine14frame by frame. The acquisition processing part111stores, in the storage part12, the acquired captured images together with an imaging time.

The acquisition processing part111also outputs image data on the acquired captured images to the operation terminal20. The acquisition processing part111is an example of the acquisition processing part according to the present invention.

The detection processing part112detects an obstacle (e.g., a person) in the captured image to be acquired by the acquisition processing part111. Specifically, the detection processing part112determines whether a person is included in a detection area, based on the captured image. For example, the detection processing part112determines presence or absence of a person in a detection area by analyzing the captured image, and collating the captured image with the determination data stored in the storage part12or the detection processing device19. The detection processing part112also performs determination processing for each of the captured images to be acquired in order by the acquisition processing part111. For example, the detection processing part112performs the determination processing for a captured image by the camera15F, then, performs the determination processing for a captured image by the camera15B, then, performs the determination processing for a captured image by the camera15R, then, performs the determination processing for a captured image by the camera15L, and then, performs the determination processing for a captured image by the camera15C. The detection processing part112transmits a detection result (determination result) to the operation terminal20. The detection processing part112is an example of the detection processing part according to the present invention.

As illustrated inFIG.1, the vehicle control device11includes various processing parts such as the traveling processing part113. Note that, the vehicle control device11functions as the various processing parts by causing the CPU to execute various pieces of processing according to the automatic traveling program. Also, some or all of the processing parts may be constituted of an electronic circuit. Note that, the automatic traveling program may be a program for causing a plurality of processors to function as the processing parts.

The traveling processing part113controls traveling of the work vehicle10. Specifically, when the traveling processing part113acquires a work start instruction from the operation terminal20, the traveling processing part113causes the work vehicle10to start automatic traveling. For example, when the operator presses a work start button on an operation screen of the operation terminal20, the operation terminal20outputs a work start instruction to the work vehicle10. When the traveling processing part113acquires the work start instruction from the operation terminal20, the traveling processing part113causes the work vehicle10to start automatic traveling. This allows the work vehicle10to start automatic traveling along the traveling route R, and a work by the work machine14is started. Note that, the traveling route R along which the work vehicle10travels is generated, for example, by the operation terminal20. The work vehicle10acquires the traveling route R from the operation terminal20, and automatically travels within the field F along the traveling route R.

Further, when the traveling processing part113acquires a traveling stop instruction from the operation terminal20, the traveling processing part113causes the work vehicle10to stop automatic traveling. For example, when the operator presses a traveling stop button on the operation screen of the operation terminal20, the operation terminal20outputs a traveling stop instruction to the work vehicle10. Also, for example, in a case where the detection processing part112detects a person, and in a case where the operator does not give either a traveling stop instruction or a traveling continuation instruction on the operation screen of the operation terminal20for a predetermined time, the operation terminal20outputs a traveling stop instruction to the work vehicle10. When the traveling processing part113acquires the traveling stop instruction from the operation terminal20, the travel processing part113causes the work vehicle10to stop automatic traveling. This allows the work vehicle10to stop automatic traveling, and a work by the work machine14is stopped. Specific examples of the traveling stop instruction and the traveling continuation instruction are described later.

Further, when the traveling processing part113acquires a traveling resume instruction from the operation terminal20, the traveling processing part113causes the work vehicle10to resume automatic traveling. For example, when the detection processing part112detects a person, and the operator presses a traveling resume button on the operation screen of the operation terminal20after the work vehicle10stops traveling, the operation terminal20outputs a traveling resume instruction to the work vehicle10. When the traveling processing part113acquires the traveling resume instruction from the operation terminal20, the traveling processing part113causes the work vehicle10to resume automatic traveling. This causes the work vehicle10to resume automatic traveling along the traveling route R again. The traveling processing part113is an example of the traveling processing part according to the present invention.

As illustrated inFIG.1, the operation terminal20is an information processing device including an operation control part21, a storage part22, an operation display part23, a communication part24, and the like. The operation terminal20may be constituted of a mobile terminal such as a tablet terminal or a smartphone.

The communication part24is a communication interface that connects the operation terminal20to the communication network N1in a wired or wireless manner, and performs data communication with an external device such as one or more work vehicles10via the communication network N1in accordance with a predetermined communication protocol.

The operation display part23is a user interface including a display part such as a liquid crystal display or an organic EL display that displays various pieces of information, and an operation part such as a touch panel that receives an operation, a mouse, or a keyboard. An operator can perform, on an operation screen to be displayed on the display part, an operation of registering various pieces of information (such as work vehicle information, field information, and work information to be described later) by operating the operation part. Further, the operator can give a work start instruction, a traveling stop instruction, a traveling resume instruction, and the like to the work vehicle10by operating the operation part. Furthermore, the operator can recognize a traveling state of the work vehicle10that is automatically traveling along the traveling route R within the field F by a traveling trajectory to be displayed on the operation terminal20at a location away from the work vehicle10.

The storage part22is a non-volatile storage part such as an HDD or an SSD that stores various pieces of information. The storage part22stores a control program such as an automatic traveling program for causing the operation control part21to perform automatic traveling processing (seeFIG.10) to be described later. For example, the automatic traveling program is non-transitorily recorded in a computer-readable recording medium such as a CD or a DVD, read by a predetermined reading device (not illustrated), and stored in the storage part22. Note that, the automatic traveling program may be downloaded from a server (not illustrated) to the operation terminal20via the communication network N1, and stored in the storage part22.

The operation control part21includes control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various pieces of arithmetic processing. The ROM is a non-volatile storage part that stores in advance a control program such as a BIOS and an OS for causing the CPU to execute various pieces of arithmetic processing. The RAM is a volatile or non-volatile storage part that stores various pieces of information, and is used as a transitory storage memory (work area) in which the CPU executes various pieces of processing. Further, the operation control part21controls the operation terminal20by causing the CPU to execute various control programs stored in advance in the ROM or the storage part22.

As illustrated inFIG.1, the operation control part21includes various processing parts such as a vehicle setting processing part211, a field setting processing part212, a work setting processing part213, a route generation processing part214, an output processing part215, a display processing part216, and a reception processing part217. Note that, the operation control part21functions as the various processing parts by causing the CPU to execute various pieces of processing according to the control program. Also, some or all of the processing parts may be constituted of an electronic circuit. Note that, the control program may be a program for causing a plurality of processors to function as the processing parts.

The vehicle setting processing part211sets information (hereinafter, referred to as work vehicle information) related to the work vehicle10. The vehicle setting processing part211sets information such as a model of the work vehicle10, a position of the positioning antenna174mounted on the work vehicle10, a type of the work machine14, a size and a shape of the work machine14, a position of the work machine14relative to the work vehicle10, a vehicle speed and an engine speed of the work vehicle10during work, a vehicle speed and an engine speed of the work vehicle10during turning, and the like by performing a registration operation on the operation terminal20by the operator.

The field setting processing part212sets information (hereinafter, referred to as field information) related to the field F. The field setting processing part212sets information such as a position and a shape of the field F, the work start position S where work is started, the work end position G (seeFIG.3) where work is finished, and a work direction by performing a registration operation on the operation terminal20.

Note that, the work direction means a direction in which the work vehicle10is caused to travel while allowing the work machine14to perform work in a work area, which is an area acquired by excluding a non-work area such as a headland, or a non-cultivated land from the field F.

Information on a position and a shape of the field F can be automatically acquired, for example, by allowing the operator to board the work vehicle10, and drive the work vehicle10in such a way as to travel around along an outer periphery of the field F, and recording a transition of position information of the positioning antenna174at the time. Further, a position and a shape of the field F can also be acquired by allowing the operator to operate the operation terminal20in a state that a map is displayed on the operation terminal20, and based on a polygonal shape acquired by determining a plurality of points on the map. An area to be determined by the acquired position and shape of the field F is an area (traveling area) where the work vehicle10is allowed to travel.

The work setting processing part213sets information (hereinafter, referred to as work information) on how the work is specifically performed. The work setting processing part213is configured in such a way that presence or absence of a cooperative work between a work vehicle10(unmanned tractor) and a manned work vehicle10, the number of skips, which is the number of work routes to be skipped in a case where the work vehicle10turns in a headland, a width of the headland, a width of a non-cultivated land, and the like are settable, as the work information.

The route generation processing part214generates the traveling route R, which is a route along which the work vehicle10is caused to automatically travel, based on the setting information. The traveling route R is, for example, a work route (seeFIG.3) from the work start position S to the work end position G. The traveling route R illustrated inFIG.3is a route along which the work vehicle10is caused to travel back and forth in parallel in the work area of the field F. The route generation processing part214can generate and store the traveling route R of the work vehicle10, based on pieces of the setting information set by the vehicle setting processing part211, the field setting processing part212, and the work setting processing part213.

Specifically, the route generation processing part214generates the traveling route R (seeFIG.3), based on the work start position S and the work end position G registered in field settings. The traveling route R is not limited to the route illustrated inFIG.3.

The work vehicle10is configured in such a way that data on the traveling route R generated in the operation terminal20are transferred to the work vehicle10and stored in the storage part12, and the work vehicle10can autonomously travel along the traveling route R while detecting a current position of the work vehicle10by the positioning antenna174. Note that, a current position of the work vehicle10usually coincides with a position of the positioning antenna174.

The work vehicle10according to the present embodiment travels in the substantially rectangular-shaped field F as illustrated inFIG.3. The work vehicle10is configured to be able to automatically travel, in a case where a current position is located within the field F, and is not able to automatically travel, in a case where a current position is located outside the field F (such as on a public road). The work vehicle10is also configured to be able to automatically travel, in a case where, for example, a current position coincides with the work start position S.

In a case where a current position of the work vehicle10coincides with the work start position S, and a work start button is pressed on the operation screen by the operator and a work start instruction is given, the work vehicle10starts automatic traveling by the traveling processing part113, and a work by the work machine14is started (seeFIG.2). In other words, the operation control part21permits automatic traveling of the work vehicle10on a condition that a current position coincides with the work start position S. Note that, the condition for permitting automatic traveling of the work vehicle10is not limited to the aforementioned condition.

The output processing part215outputs, to the work vehicle10, information on the traveling route R generated by the route generation processing part214. The output processing part215can also instruct the work vehicle10to start and stop automatic traveling, and the like by transmitting a control signal to the work vehicle10via the communication part24. This allows the work vehicle10to automatically travel.

For example, the traveling processing part113causes the work vehicle10to automatically travel from the work start position S to the work end position G, based on the traveling route R acquired from the operation terminal20. Also, the traveling processing part113may cause the work vehicle10to automatically travel from the work end position G to an entrance of the field F, when the work vehicle10finishes the work. In a case where the work vehicle10is automatically traveling, the operation control part21can receive, from the work vehicle10, a state (such as a position and a traveling speed) of the work vehicle10, and cause the operation display part23to display the state.

The display processing part216causes the operation display part23to display various pieces of information. For example, the display processing part216causes the operation display part23to display an operation screen for registering work vehicle information, field information, work information, and the like, a setting screen D1(seeFIGS.5A and5B) for setting a layout of image display columns L1that display a captured image of each of the plurality of cameras15installed on the work vehicle10, a traveling state screen D2(seeFIG.6AtoFIG.8B) that displays the captured images, a traveling selection screen D3(seeFIG.9) for selecting a traveling stop instruction to stop traveling of the work vehicle10, or a traveling continuation instruction to continue traveling, and the like.

The reception processing part217receives various operations from the operator. For example, the reception processing part217receives an operation (an example of a setting operation according to the present invention) of setting a layout of the image display columns L1from the operator. Also, the reception processing part217receives, from the operator, a work start instruction of causing the work vehicle10to start working. The reception processing part217also receives, from the operator, a traveling stop instruction of causing the work vehicle10during automatic traveling to stop traveling. The reception processing part217also receives, from the operator, a traveling stop instruction or a traveling continuation instruction of the work vehicle10, in a case where the work vehicle10detects a person. The reception processing part217also receives, from the operator, a traveling resume instruction of causing the work vehicle10that has stopped traveling to resume traveling. When the reception processing part217receives each of the instructions, the output processing part215outputs each of the instructions to the work vehicle10. The reception processing part217is an example of the reception processing part according to the present invention.

When the traveling processing part113of the work vehicle10acquires a work start instruction from the operation terminal20, the traveling processing part113causes the work vehicle10to start traveling and working. Also, when the traveling processing part113acquires a traveling stop instruction from the operation terminal20, the traveling processing part113causes the work vehicle10to stop traveling and working. Also, when the traveling processing part113acquires a traveling continuation instruction from the operation terminal20, the traveling processing part113causes the work vehicle10to continue traveling and working. Also, when the traveling processing part113acquires a traveling resume instruction from the operation terminal20, the traveling processing part113causes the work vehicle10to resume traveling and working.

Herein, a specific example is described. For example, in a case where the five cameras15F,15B,15R,15L, and15C (cameras1to5) are installed on the work vehicle10as illustrated inFIG.4, and the display processing part216acquires the number of the cameras15and identification information to be acquired from the work vehicle10, the display processing part216causes the operation display part23to display the setting screen D1for setting a layout of the image display columns L1. For example, as illustrated inFIG.5A, the display processing part216causes to display five image display columns L1that display captured images of the camera15F (camera1), the camera15B (camera2), the camera15R (camera3), the camera15L (camera4), and the camera15C (camera5) side by side in any order. Also, as illustrated inFIG.5B, for example, the display processing part216may arrange the five image display columns L1at a position associated with an installation layout of the cameras15on the work vehicle10. The operator may also change the layout of the image display columns L1on the setting screen D1. For example, the operator may arrange each image display column L1at a desired position on the setting screen D1by a drag-and-drop operation, and the like.

Further, as illustrated on a left portion of the setting screen D1, the display processing part216may desirably display an image (camera placement image Pc) capable of identifying a placement position of each camera15relative to the work vehicle10. This allows the operator to easily recognize a correlation between the placement position of each camera15and the layout of the image display columns L1. Note that, the display processing part216may omit the camera placement image Pc on the setting screen D1.

The display processing part216arranges and displays a plurality of captured images P1according to the setting operation on the setting screen D1. For example, when the layout of the image display columns L1illustrated inFIG.5Ais set, as illustrated inFIG.6A, the display processing part216causes to display a camera image display column A1that displays a captured image of each camera15on the traveling state screen D2that displays an automatic traveling state of the work vehicle10. The traveling state screen D2displays a current state of the work vehicle10during automatic traveling in real time. The operator can recognize a current traveling status and work status of the work vehicle10on the traveling state screen D2. The display processing part216causes to display each image display column L1in the camera image display column A1according to the layout set on the setting screen D1.FIG.6Billustrates an example of the traveling state screen D2on which the captured image P1is displayed in each image display column L1. The captured image P1to be displayed in each image display column L1is updated in real time.

In a case where the image display column L1is set on the layout illustrated inFIG.5B, the display processing part216causes to display each image display column L1illustrated inFIG.7A.FIG.7Billustrates an example of the traveling state screen D2on which the captured image P1is displayed in each image display column L1.

Herein, in a case where an obstacle (in this case, a person) is detected in at least one of the captured images of the five cameras15, the operation terminal20performs the following processing. For example, in a case where the detection processing part112of the work vehicle10detects a person in a captured image acquired from the camera15L (camera4), the detection processing part112outputs, to the operation terminal20, a determination result indicating that a person is detected, and identification information of the camera15L (camera4) associated with the captured image. The acquisition processing part111also outputs, to the operation terminal20, image data on the captured image acquired from each camera15.

When the display processing part216of the operation terminal20acquires the image data from the work vehicle10, the display processing part216causes to display the captured image P1in each image display column L1. Also, when the display processing part216acquires the determination result and the identification information from the work vehicle10, as illustrated inFIG.8A, the display processing part216causes to display the image display column L1and the captured image P1associated with the camera15L (camera4) in an enlarged manner. This allows the image of the person detected as an obstacle to be displayed in an enlarged manner.

As another embodiment, the display processing part216may cause to display the image display column L1associated with the camera15L (camera4) in a highlighted manner among the plurality of image display columns L1. For example, as illustrated inFIG.8B, the display processing part216causes to display a frame of the image display column L1associated with the camera15L (camera4) in a bold line. The display processing part216may also cause to display the image display column L1associated with the camera15L (camera4) in color, by turning light, or by blinking. The display processing part216may also cause to display only the image display column L1associated with the camera15L (camera4) in the camera image display column A1, and hide the image display columns L1associated with the other cameras15.

When the display processing part216acquires, from the work vehicle10, a determination result indicating that a person is detected, the display processing part216causes to display, on the operation display part23, the traveling selection screen D3for selecting a traveling stop instruction to stop traveling of the work vehicle10, or a traveling continuation instruction to continue traveling.FIG.9illustrates an example of the traveling selection screen D3. The display processing part216causes to display, on the traveling selection screen D3, a message indicating that an obstacle is detected, a message for selecting whether to stop traveling or continue traveling, a traveling stop button B1to stop traveling, and a traveling continuation button B2to continue traveling. When the operator presses the traveling stop button B1on the traveling selection screen D3, the reception processing part217receives a traveling stop instruction. When the reception processing part217receives the traveling stop instruction, the output processing part215outputs the traveling stop instruction to the work vehicle10. When the traveling processing part113of the work vehicle10acquires the traveling stop instruction from the operation terminal20, the traveling processing part113causes the work vehicle10to stop traveling and working.

In contrast, when the operator presses the traveling continuation button B2on the traveling selection screen D3, the reception processing part217receives a traveling continuation instruction. When the reception processing part217receives the traveling continuation instruction, the output processing part215outputs the traveling continuation instruction to the work vehicle10. When the traveling processing part113of the work vehicle10acquires the traveling continuation instruction from the operation terminal20, the traveling processing part113does not stop traveling and working of the work vehicle10, and the work vehicle10continues traveling and working. Note that, in a case where the reception processing part217receives the traveling continuation instruction, the output processing part215may be configured not to output the traveling continuation instruction to the work vehicle10. Thus, since the traveling processing part113does not perform processing of causing the work vehicle10to stop traveling, consequently, the work vehicle10continues traveling and working.

Herein, it is conceivable that the operator does not press either the traveling stop button B1or the traveling continuation button B2on the traveling selection screen D3. For example, it is conceivable that the operator does not notice that an obstacle is detected or that the traveling selection screen D3is displayed. In this case, in order to avoid a risk of contact between the work vehicle10and the obstacle, the output processing part215outputs the traveling stop instruction to the work vehicle10. Specifically, in a case where the reception processing part217does not receive either the traveling stop instruction or the traveling continuation instruction, the traveling processing part113causes the work vehicle10to stop automatic traveling. For example, in a case where the operator does not press either the traveling stop button B1or the traveling continuation button B2until a predetermined time elapses after the display processing part216has displayed the traveling selection screen D3, or after the detection processing part112has detected a person, the output processing part215outputs the traveling stop instruction to the work vehicle10.

In this way, in a case where an obstacle (person) is detected by the detection processing part112, and in a case where the reception processing part217does not receive either the traveling stop instruction or the traveling continuation instruction until a predetermined time elapses, the traveling processing part113causes the work vehicle10to stop automatic traveling. In other words, the work vehicle10stops traveling and working, in a case where an obstacle is detected, and an instruction is not received from the operator.

Herein, the predetermined time may be a fixed time that is set in advance, or may be a time depending on a size of an obstacle when the obstacle is detected. For example, in a case where a size of the obstacle in the captured image is large, a distance between the work vehicle10and the obstacle becomes short, as compared with a case where the size of the obstacle in the captured image is small. Therefore, the operation control part21sets the predetermined time short, as the size of the obstacle in the captured image increases. Also, the operation control part21sets the predetermined time long, as the size of the obstacle in the captured image decreases. This allows a reception time of the traveling stop instruction and the traveling continuation instruction to be shortened, in a case where the distance between the work vehicle10and the obstacle is short, and a reception time of the traveling stop instruction and the traveling continuation instruction to be extended, in a case where the distance between the work vehicle10and the obstacle is long. Note that, the traveling processing part113may decelerate the work vehicle10during the predetermined time.

Note that, the operation control part21may inform the operator of information indicating that the work vehicle10has detected an obstacle. For example, the operation control part21may output a sound (warning sound), when the work vehicle10has detected an obstacle.

When the operator gives the traveling resume instruction on the operation terminal20after the work vehicle10has stopped by the traveling stop instruction, the reception processing part217receives the traveling resume instruction, and the output processing part215outputs the traveling resume instruction to the work vehicle10. When the traveling processing part113of the work vehicle10acquires the traveling resume instruction from the operation terminal20, the traveling processing part113causes the work vehicle10to resume traveling and working.

Note that, the operation terminal20may be accessible to a website (agricultural support site) of agricultural support services provided by a server (not illustrated) via the communication network N1. In this case, the operation terminal20is able to function as an operation terminal for the server by causing the operation control part21to execute a browser program. Further, the server includes each processing part described above, and performs each piece of processing.

As another embodiment, each function of the vehicle control device11and the detection processing device19described above may be included in the operation control part21of the operation terminal20. In other words, for example, the operation control part21may acquire image data on a captured image from each camera15of the work vehicle10, and perform processing of detecting an obstacle. Also, each function (the acquisition processing part111and the detection processing part112) of the detection processing device19may be included in the vehicle control device11.

An example of the automatic traveling processing to be performed by the vehicle control device11, the detection processing device19, and the operation control part21is described with reference toFIG.10. For example, the automatic traveling processing is started by the vehicle control device11, the detection processing device19, and the operation control part21, in a case where the work vehicle10starts automatic traveling. Note that, the invention of the present application may be regarded as an invention directed to an automatic traveling method in which the vehicle control device11, the detection processing device19, and the operation control part21perform a part or all of the automatic traveling processing, or as an invention directed to an automatic traveling program for causing the vehicle control device11, the detection processing device19, and the operation control part21to perform a part or all of the automatic traveling method. Also, the automatic driving processing may be executed by one or more processors.

In step S1, the vehicle control device11causes the work vehicle10to start working. For example, when the operator presses the work start button on the operation screen of the operation terminal20, the operation control part21outputs a work start instruction to the work vehicle10. When the vehicle control device11acquires the work start instruction from the operation terminal20, the vehicle control device11causes the work vehicle10to start automatic traveling. This allows the work vehicle10to start automatic traveling along the traveling route R, and a work by the work machine14is started.

Next, in step S2, the detection processing device19acquires a captured image from each of the cameras15installed on the work vehicle10. For example, the detection processing device19acquires a captured image (frame image) from each of the five cameras15F,15B,15R,15L, and15C continuously at a predetermined frame rate.

Next, in step S3, the detection processing device19detects an obstacle (e.g., a person) in the acquired captured images. Specifically, the detection processing device19determines whether a person is included in a detection area, based on the captured images. In a case where the detection processing device19detects a person (S3: Yes), the processing proceeds to step S4. On the other hand, in a case where the detection processing device19does not detect a person (S3: No), the processing is shifted to step S31.

In step S31, the vehicle control device11determines whether the work vehicle10has finished the work. In a case where the work vehicle10has finished the work (S31: Yes), the processing is terminated. On the other hand, in a case where the work vehicle10has not finished the work (S31: No), the processing returns to step S2. In this way, in a case where the detection processing device19does not detect a person, the work vehicle10performs traveling and working, while acquiring captured images until a predetermined work is finished.

Further, the detection processing device19outputs, to the operation terminal20, image data on the acquired captured image, identification information of the camera15associated with the captured image, and a determination result indicating whether a person is included in the captured image. When the operation control part21acquires the image data from the detection processing device19of the work vehicle10, the operation control part21causes to display the captured image P1in each image display column L1(seeFIG.6B).

In step S4, the operation control part21causes to display the captured image including a person in an enlarged manner. For example, in a case where a person is detected in a captured image associated with the camera15L (camera4), as illustrated inFIG.8A, the operation control part21enlarges the image display column L1associated with the camera15L (camera4), and causes to display the captured image P1in an enlarged manner. This allows the image of the person detected as an obstacle to be displayed in an enlarged manner.

Next, in step S5, the operation control part21causes to display, on the operation display part23, the traveling selection screen D3for selecting a traveling stop instruction to stop traveling of the work vehicle10, or a traveling continuation instruction to continue traveling (seeFIG.9).

Next, in step S6, the operation control part21determines whether the traveling stop instruction is received from the operator on the traveling selection screen D3. For example, when the operator presses the traveling stop button B1on the traveling selection screen D3illustrated inFIG.9, the operation control part21receives the traveling stop instruction (S6: Yes), and the processing is shifted to step S9. On the other hand, in a case where the operator does not press the traveling stop button B1on the traveling selection screen D3illustrated inFIG.9(S6: No), the processing is shifted to step S7.

Next, in step S7, the operation control part21determines whether the traveling continuation instruction is received from the operator on the traveling selection screen D3. For example, when the operator presses the traveling continuation button B2on the traveling selection screen D3illustrated inFIG.9, the operation control part21receives the traveling continuation instruction (S7: Yes), and the processing is shifted to step S71. On the other hand, in a case where the operator does not press the traveling continuation button B2on the traveling selection screen D3illustrated inFIG.9(S7: No), the processing is shifted to step S8.

In step S71, the operation control part21does not cause the work vehicle10to stop traveling and working, but causes the work vehicle10to continue traveling and working. Thereafter, the processing returns to step S2.

In step S8, the operation control part21determines whether a predetermined time has elapsed. Specifically, the operation control part21determines whether the predetermined time has elapsed without receiving either the traveling stop instruction or the traveling continuation instruction since the operation control part21displayed the traveling selection screen D3, or since the detection processing device19detected a person. In a case where the predetermined time has elapsed without receiving either the traveling stop instruction or the traveling continuation instruction (S8: Yes), the processing is shifted to step S9. In a case where the predetermined time has not elapsed (S8: No), the processing returns to step S6.

In step S9, the operation control part21outputs the traveling stop instruction to the work vehicle10, and the vehicle control device11causes the work vehicle10to stop traveling and working.

In step S10, the operation control part21determines whether a traveling resume instruction is received from the operator. When the operation control part21receives the traveling resume instruction from the operator (S10: Yes), the processing returns to step S2. On the other hand, in a case where the operation control part21does not receive the traveling resume instruction from the operator (S10: No), the processing is terminated. The vehicle control device11, the detection processing device19, and the operation control part21repeatedly perform pieces of the processing of steps S1to S10until the work vehicle10finishes working.

As described above, the automatic traveling system1according to the present embodiment acquires the captured image P1from the camera15installed on the work vehicle10, and detects an obstacle, based on the captured image P1. Also, the automatic traveling system1receives a traveling stop instruction to stop automatic traveling of the work vehicle10, or a traveling continuation instruction to continue automatic traveling, in a case where the obstacle is detected. Further, the automatic traveling system1causes the work vehicle10to stop automatic traveling, in a case where the traveling stop instruction is received, and causes the work vehicle10to continue automatic traveling, in a case where the traveling continuation instruction is received. This allows the operator to select whether to stop automatic traveling or to continue automatic traveling, in a case where the work vehicle10has detected an obstacle during automatic traveling. For example, in a case where the obstacle is an obstacle that does not interfere with automatic traveling, the work vehicle10continues automatic traveling without stopping by allowing the operator to give an instruction to continue automatic traveling, thereby preventing lowering of work efficiency. In addition, for example, in a case where the obstacle is an obstacle that interferes with automatic traveling, the operator can secure safety by stopping automatic traveling, removing the obstacle, and the like. Then, the operator can resume automatic traveling, in a case where safety of the work vehicle10is secured. Thus, it is possible to prevent lowering of work efficiency, while securing safety of the work vehicle10.

Further, in a case where the work vehicle10has detected an obstacle during automatic traveling, and in a case where the operator does not perform either an operation of stopping automatic traveling, or an operation of continuing automatic traveling within a predetermined time, the automatic traveling system1stops automatic traveling. This can avoid a situation in which the work vehicle10continues automatic traveling and comes into contact with an obstacle without being noticed by the operator.

The present invention is not limited to the embodiment described above, but the following embodiments may be available.

In the embodiment described above, in a case where an obstacle (person) is detected by the detection processing part112, and in a case where the reception processing part217does not receive either the traveling stop instruction or the traveling continuation instruction until a predetermined time has elapsed, the traveling processing part113causes the work vehicle10to stop automatic traveling. As another embodiment, in a case where an obstacle is detected by the detection processing part112, and in a case where the reception processing part217does not receive either the traveling stop instruction or the traveling continuation instruction until the distance between the work vehicle10and the obstacle reaches a predetermined distance, the traveling processing part113may cause the work vehicle10to stop automatic traveling.

The operation control part21may compute a change in distance between the work vehicle10and the obstacle, based on successive frame images to be acquired from the work vehicle10. For example, as illustrated inFIG.11, the operation control part21may compute the distance, based on a size of a frame W1surrounding a person in the captured image P1.

Herein, the predetermined distance may be a predetermined fixed distance, or may be a distance depending on a size of an obstacle when the obstacle is detected. For example, in a case where a size of the obstacle in the captured image is large, a distance between the work vehicle10and the obstacle becomes short, as compared with a case where the size of the obstacle in the captured image is small. Therefore, the operation control part21sets the predetermined distance short, as the size of the obstacle in the captured image increases. Also, the operation control part21sets the predetermined distance long, as the size of the obstacle in the captured image decreases. This allows a reception time of the traveling stop instruction and the traveling continuation instruction to be shortened, in a case where the distance between the work vehicle10and the obstacle is short, and a reception time of the traveling stop instruction and the traveling continuation instruction to be extended, in a case where the distance between the work vehicle10and the obstacle is long. Note that, the traveling processing part113may decelerate the work vehicle10during the predetermined distance.

As another embodiment, for example, in a case where the detection processing part112detects a specific person, and the operator gives a traveling continuation instruction, and thereafter, in a case where the detection processing part112detects the same person again, the work vehicle10may continue automatic traveling without acquiring a traveling continuation instruction from the operator. This improves convenience because it is not necessary to request the operator to select a traveling stop instruction or a traveling continuation instruction, for example, each time the same obstacle that does not interfere with automatic traveling is detected.

Further, in the present embodiment, obstacle detection and determination are performed based on an image captured by a camera, but, as another embodiment, a camera and Lidar (obstacle detection sensor) may be used together. Specifically, obstacle detection may be performed by the Lidar, and obstacle determination may be performed by the camera. The obstacle detection sensor may be a sensor utilizing an ultrasonic wave or the like.