REMOTE CONTROL SYSTEM

A remote control system allows a forklift to be operated remotely in an intuitive and easy way even when an obstacle is present between the forklift and the object, The remote control system includes a guidance route display section that displays on a display unit a guidance route connecting the forklift and the object. The guidance route display section is configured to display a guidance route straightly connecting the forklift and the object if a judgement section judges that an obstacle is not present, or, if the judgement section judges that an obstacle is present, display a guidance route connecting the forklift and the object in a manner of avoiding the obstacle.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority benefits of Japanese Patent Application No. 2021-163952, filed on Oct. 5, 2021. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of this specification.

FIELD OF THE INVENTION

This invention relates to a remote control system for remotely operating a forklift.

Description of Related Art

A forklift is for an operator to board and perform operations of carrying and unloading a cargo. Hence, many years of experience are required for the operator to operate the forklift quickly and accurately. On the other hand, since the number of skilled operators is limited, the number of skilled operators may be insufficient in a locality with a small population. Hence, it is required that a skilled operator remotely operate a local forklift from an area where there are many skilled operators.

Further, when handling a tank filled with a gas that adversely affects the human body or an explosive that explodes if dropped, it is preferred that the operator operates the forklift remotely away from the warehouse where the cargo is handled.

Therefore, a remote control system for remotely operating a forklift is known (see, for example, Patent Literature 1). In the remote control system, an operation apparatus is provided in a remote base station away from the warehouse where the cargo is handled by the forklift, and the operator operates the forklift from the remote base station using the operation apparatus.

The forklift has a control device, a wireless communication device, a camera, a sensor, and a driving device for operating its traveling and the raising/lowering of its fork, etc. The operation apparatus in the base station includes a handle, a lever, a pedal, a control unit and a display unit, etc. For example, the display unit is provided with two display monitors, one of which displays an image of the front of the forklift taken by a camera mounted on the forklift, and the other of which displays the information detected by the sensor and so on.

When carrying and unloading with a forklift, the operator operates the forklift using the handle or the lever, etc., while checking the object to be handled in front of the forklift based on the image on the display unit. On the other hand, depending on the direction of the forklift, the object to be handled may not be visible from the display unit, so that there is a problem that operating the forklift is difficult. Further, if there is an obstacle between the forklift and the object, operating the forklift is also difficult, as it is required to approach the object in a manner of avoiding the obstacle while checking the positions of both the object and the obstacle.

CITATION LIST

Patent Literature

SUMMARY OF THE INVENTION

Accordingly, this invention provides a remote control system that allows a forklift to be operated intuitively and easily even if an obstacle is present between the forklift and the object, while the forklift is operated remotely.

The remote control system is for remotely operating a forklift, and includes a camera provided on the forklift, a display unit displaying images taken by the camera, a forklift coordinates acquisition section acquiring the coordinates of the forklift, an object coordinates acquisition acquiring the coordinates of a designated object, a judgement section judging whether or not an obstacle is present between the forklift and the object, and a guidance route display section displaying a guidance route connecting the forklift and the object on the display unit. The guidance route display section is configured to display a guidance route straightly connecting the forklift and the object if the judgement section judges that the obstacle is not present between the forklift and the object, or display a guidance route connecting the forklift and the object in a manner of avoiding the obstacle if the judgement section judges that the obstacle is present between the forklift and the object.

In an embodiment, the judgement section forms a virtual route straightly connecting the forklift and the object and having a width corresponding to the width of the forklift, and judges that the obstacle is not present between the forklift and the object if the entirety or a portion of the obstacle is not present inside the virtual route, or that the obstacle is present between the forklift and the object if the entirety or a portion of the obstacle is present inside the virtual route, and the guidance route display section displays the guidance route in a manner that the entirety or a portion of the obstacle is not present inside the guidance route.

In an embodiment, the guidance route display section displays the guidance route having a width corresponding to the width of the forklift.

In an embodiment, the guidance route display section displays first and second guidance routes as the above guidance route, wherein the first guidance route avoids the right side of the obstacle and the second guidance route avoids the left side of the obstacle.

In an embodiment, the first guidance route and the second guidance route have different colors, shapes or blink rates

With the remote control system of his invention, while the forklift is operated remotely, the forklift can be operated intuitively and easily even if an obstacle is present between the forklift and the object.

DESCRIPTION OF THE EMBODIMENTS

The remote control system according to an embodiment of this invention is described below with reference to the drawings.

Referring toFIG.1, the remote control system includes forklifts1that travel and do cargo handling work in the facility100. The remote control system is provided with a base station200provided at a remote location away from the facility100. The base station200is provided with an operation apparatus2. An operator can remotely operate a predetermined forklift1by using the operation apparatus2.

Although the facility100is a warehouse in this embodiment, the facility in this invention may alternatively be a factory or an outdoor work place, etc. In addition, though the forklifts1are reach forklifts in this embodiment, they may be counterbalance forklifts or the like alternatively. When an operator operates a forklift1from a remote base station200, for example, a tank filled with a gas that adversely affects the human body or an explosive that explodes if dropped can be loaded and unloaded in the facility100.

Referring to FIG,2, the forklift1includes a camera10, an obstacle sensor11, a laser scanner12, a driving device13, a control unit14, and a wireless communication unit15, etc. Referring toFIG.3, the control unit14includes an image processing section141, a forklift coordinates acquisition section142, a driving control section143and an obstacle coordinates acquisition section144, etc. The control unit14is composed of a CPU (as a central processing device), an input/output interface, ROM and RAM, etc., and stores a program for processing information. The camera10, the obstacle sensor11, the laser scanner12, the driving device13and the wireless communication unit15are connected to the control unit14.

The driving device13includes a traveling motor for driving the driving wheels16provided at the rear part of the body of the forklift1, a plurality of hydraulic cylinders for raising, lowering, tilting, advancing and retreating the fork17provided at the front part of the body of the forklift1, and so on. An operation signal from the operation apparatus2provided in the base station200is sent to the control unit14via the wireless communication unit15, this operation signal is processed by the driving control section143, and the driving device13of the traveling motor and the hydraulic cylinders is driven based on the operation of the operation apparatus2. As a result, the driving device13of the traveling motor, the hydraulic cylinders and so on is driven in conjunction with the operation of the operation apparatus2by the operator, and the forklift1can be operated.

The camera10is arranged at a position of the line of sight of an operator who operates on board the forklift1, and takes pictures of the front of the forklift1from the position of the line of sight of such operator. The camera10includes, for example, a CCD image sensor or a CMOS image sensor. The image taken by the camera10is processed by the image processing section141of the control unit14and displayed on the display unit20(FIG.4) provided in the operation apparatus2. Thereby, the operator who remotely controls the forklift1using the operating apparatus2at the base station200is able to confirm, by means of the display unit20, the front of the forklift1from the same line of sight as when he or she is on board the forklift1to operate the forklift1.

The forklift1is provided with a laser scanner12, and a plurality of reflectors101are installed in the facility100. The laser scanner12transmits/receives laser to/from the reflectors101while rotating the laser horizontally by 360°. As a result, the forklift1can recognize the plurality of reflectors101arranged along the traveling path in the facility100using the laser scanner12. Here, the reflectors101are fixed to the walls in the facility100, and their position information stored in the map of the forklift coordinates acquisition section142of the control unit14. With the recognition of the plurality of reflectors101by the forklift1using the laser scanner12, the forklift coordinates acquisition section142can measure and acquire the position coordinates of the forklift1based on the principle of triangulation.

The forklift1is provided with an obstacle sensor11, which includes, for example, an optical sensor. The obstacle sensor11is capable of detecting an obstacle present in a predetermined area in front of the forklift1and also measure the distance and the direction from the forklift1to the obstacle. The obstacle coordinates acquisition section144of the control unit14can measure and acquire the position coordinates of the obstacle based on the detection signals of the obstacle sensor11and the position coordinates of the forklift1.

Referring toFIG.4, the operation apparatus2includes a display unit20, an operation unit21, an object designation unit22, a control unit23, and a wireless communication unit24, etc. The control unit23includes an image processing section231, a judgement section232, a guidance route display section233, and an object coordinates acquisition section234, etc. The control unit23is composed of a CPU (a central processing device), an input/output interface, ROM, RAM and so on, and stores a program for processing information. The display unit20, the operation unit21, the object designation unit22and the wireless communication unit24are connected to the control unit23.

The display unit20may include, for example, two display monitors, wherein one display monitor displays an image of the front of the forklift1taken by the camera10mounted on the forklift1, and the other display monitor displays information detected by various sensors and information necessary for operation, etc.

The operation unit21includes a handle, a lever and a pedal, etc., and the operator can operate the forklift in the same manner as when actually boarding the forklift and operating. The operation signal from the operation unit21is transmitted by the wireless communication unit24via the control unit23and received by the wireless communication unit15of the forklift1, Then, the operator can remotely operate the driving device13and so on of the forklift1, as described above, by operating the handle, the lever and so on of the operation unit21.

The object designation unit22is, for example, a mouse. As described later, after the operator uses the mouse as the object designating unit22to designate an object110with the pointer220displayed on the display unit20, the guidance route display section233displays, on the display unit20, a guidance route GR or guidance routes GR1and GR2connecting the designated object110and the forklift1(FIGS.6,8and10). When the forklift1travels to the object110, the operator can virtually travel along the guidance route GR, GR1or GR2displayed on the display unit20, so that the forklift1can be operated intuitively and easily.

First Embodiment

The first embodiment of the remote control system will be described below.

Referring toFIG.5, the forklift coordinates acquisition section142constantly measures and acquires the absolute coordinates (Fx, Fy) of the forklift1(step S1). An image taken by the camera10is always displayed on the display unit20. The operator remotely controls the forklift1by the operation unit21based on the image displayed on the display unit20.

Referring toFIG.6, after the objects110to be handled is displayed on the display unit20, the operator uses the mouse as the object designation unit22and the pointer220displayed on the display unit20to designate an object110(step S2). Specifically, after the operator puts the pointer220on the object110and clicks the mouse, the object110is designated.

The image processing section231identifies the boundary with the floor surface, the wall surface or the like to recognize the shape of the designated object110(step S3). Then, the object coordinates acquisition section234acquires the camera coordinates of the object110displayed on the display unit20, and the absolute coordinates (Ox, Oy) of the object110are measured and acquired based on the absolute coordinates (Fx, Fy) of the forklift1(step S4).

Referring toFIG.7, the judgement section232connects the absolute coordinates (Fx, Fy) of the forklift1and the absolute coordinates (Ox, Oy) of the object110to form a virtual route VR having a width corresponding to the width W of the forklift1(step S5). Then, the obstacle sensor11measures and acquires the coordinate position of the obstacle120, and the image processing section231identifies the boundary with the floor surface, the wall surface or the like, so that the shape of the obstacle120existing other than the object110is recognized (step S6). After that, the judgement section232judges whether or not the obstacle120is present between the forklift1and the object110(step S7). The presence or absence of the obstacle120is judged as follows.

The judgement section232judges that the obstacle120is not present between the forklift1and the object110if the entirety or a portion of the obstacle120is not present inside the formed virtual route VR (step S7). Because a travel route having a width corresponding to the width W of the forklift1is formed when the forklift1actually travels, with the virtual route VR having the width W, the actual travel route of the forklift1can be conceived virtually and intuitively.

Referring toFIGS.6and7, if the judgement section232judges that the obstacle120is not present between the forklift1and the object110, a guidance route GR straightly connecting the forklift1and the object110is formed (step S8). The guidance route GR has a width corresponding to the width W of the forklift1, so that if it is judged that the obstacle120is not present, the guidance route GR coincides with the virtual route YR. Because a travel route having a width corresponding to the width W of the forklift1is formed when the forklift1actually travels, with the guidance route GR having the width W, the actual travel route of the forklift1can be conceived virtually and intuitively.

Referring shown in FIG,6, the guidance route display section233converts the formed guidance route GR into camera coordinates and displays it on the display unit20. Then, as the forklift1moves, the absolute coordinates (Fx, Fy) of the forklift1are changed, and the guidance route GR is displayed on the display unit20following the absolute coordinates (Fx, Fy) of the forklift1that are changed at any time.

On the other hand, the judgement section232judges that the obstacle120is present between the forklift1and the object110if the entirety or a portion of the obstacle120is present inside the formed virtual route VR (step S7). If the judgement section232judges that the obstacle120is present, then the guidance route GR connecting the forklift1and the object110is formed in a manner of avoiding the obstacle120(step S9). Referring toFIGS.8and9, the guidance route GR has a width corresponding to the width W of the forklift1, and is formed by curves not overlapping with the recognized obstacle120. For example, a guidance route GR can be formed by connecting both end points of the forklift1with points separated from the obstacle120by predetermined distances and both end points of the object110by spline curves.

Referring toFIG.8, the guidance route display section233converts the formed guidance route GR into camera coordinates and displays it on the display unit20. Then, as the forklift1moves, the absolute coordinates (Fx, Fy) of the forklift1are changed, and the guidance route is displayed on the display unit20following the absolute coordinates (Fx, Fy) of the forklift1that are changed at any time.

Second Embodiment

A second embodiment of the remote control system will be described below. For the same configuration as the first embodiment, detailed description thereof is omitted to avoid duplicate explanations.

Referring toFIG.5, the forklift coordinates acquisition section142constantly measures and acquires the absolute coordinates (Fx, Fy) of the forklift1(step S1). Then, the operator designates an object110by the object designation unit22(step S2).

The image processing section231recognizes the shape of the designated object110(step S3), Then, the object coordinates acquisition section234measures and acquires the absolute coordinates (Ox, Oy) of the object110(step S4).

The judgement section232forms a virtual route VR connecting the absolute coordinates (Fx, Fy) of the forklift1and the absolute coordinates (Ox, Oy) of the object110(step S5). Then, the image processing section231recognizes the shape of the obstacle120(step S6). After that, the judgement section232judges whether or not the obstacle120is present between the forklift1and the object110(step S7). If the judgement section232judges that the obstacle120is not present, a guidance route GR straightly connecting the forklift1and the object110is formed as in the first embodiment (step S8).

On the other hand, if the judgement section232judges that the obstacle120is present, a curved guidance route GR connecting the forklift1and the object110is formed in a manner of avoiding the obstacle120(step S9). In the second embodiment, as shown inFIG.11, two guidance routes are formed at this time, including a first guidance route GR1avoiding the right side of the obstacle120, and a second guidance route GR2avoiding the left side of the obstacle120. As a result, the operator can intuitively determine to drive on the right side or the left side of the obstacle120while checking the conditions of the road surface and the surroundings and so on.

Referring toFIG.10, the guidance route display section233converts the formed first and second guidance routes GR1and GR2into camera coordinates and displays them on the display unit20. Then, as the forklift1moves, the absolute coordinates (Fx, Fy) of the forklift1are changed, and the first and second guidance routes GR1and GR2are displayed on the display unit20following the absolute coordinates (Fx, Fy) of the forklift1that are changed at any time. The first and second guidance routes GR1and GR2are displayed in different colors, shapes or blink rates to allow the operator to easily recognize each of the guidance routes GR1and GR2.

Although the preferred embodiments of this invention have been described above, the configuration of this invention is not limited to them. For example, the configuration may be modified as follows.Though the obstacle sensor11is an optical sensor in the above embodiments, for example, an obstacle120other than the object110also may be recognized by the image processing section141based on the image taken by the camera10, so the obstacle sensor11also can be constituted of the image processing section141.Though the guidance route GR has a band shape having a width corresponding to the width W of the forklift1in the above embodiments, it also may be represented by an arrow, a blinking line or a thin line, etc.Though the guidance route GR is formed by curved lines to avoid the obstacle120in the above embodiments, it also may be formed by a bent line or the like.Though the object designation unit22is a mouse in the above embodiments, if, for example, the display monitor20is a touch panel, then the operator may press an object110displayed on the display monitor20with a finger to designate the object110, and thus the object designation unit22also may be constituted of a touch sensor.

Effects of this invention are described below.

The remote control system according to this invention includes a camera10provided on the forklift1, a display unit20displaying the images taken by the camera10, and a forklift coordinates acquisition section142acquiring the coordinates of the forklift1. Further, the remote control system designates an object110displayed on the display unit20, and also includes an object coordinates acquisition section234acquiring the coordinates of the object110, a judgement section232judging whether or not the obstacle120is present between the forklift1and the object110, and a guidance route display section233displaying on the display unit20a guidance route GR connecting the forklift1and the object110. The guidance route display section233is configured to display a guidance route GR straightly connecting the forklift1and the object110if the judgement section232judges that the obstacle120is not present between the forklift1and the object110, or display a guidance route GR connecting the forklift1and the object110in a manner of avoiding the obstacle120if the judgement section232judges that the obstacle120is present between the forklift1and the object110.

As described above, even when the obstacle120is present between the forklift1and the object110, a guidance route GR connecting the forklift1and the object110in a manner of avoiding the obstacle120is displayed on the display unit20. As a result, n the forklift1travels to the object110, the operator virtually travel along the guidance route GR displayed on the display unit20, so that the forklift1can be operated intuitively and easily.

It is preferred that the judgement section232connects the forklift1and the object110to form a virtual route YR having a width corresponding to the width W of the forklift1and then judges that the obstacle120is not present if the entirety or a portion of the obstacle120is not present inside the virtual route VR, or judges that the obstacle120is present if the entirety or a portion of the obstacle120is present inside the virtual route VR, and the guidance route display section233displays the guidance route GR in a manner that the entirety or a portion of the obstacle120is not present inside the guidance route GR.

Because a travel route having a width corresponding to the width W of the forklift1is formed when the forklift1actually travels, with the virtual route VR having the width W, an actual travel route of the forklift1can be conceived virtually and intuitively.

Further, it is desirable that the guidance route display section233displays the guidance route GR having a width corresponding to the width W of the forklift1.

When the forklift1actually travels, a travel route having a width corresponding to the width W of the forklift1is formed. Since the guidance route GR has a width W, the operator can virtually and intuitively conceive an actual travel route of the forklift1, so that the forklift1can easily travel.

Furthermore, it is preferred that the guidance route display section233displays a first guidance route GR1that avoids the right side of the obstacle120and a second guidance route GR2that avoids the left side of the obstacle120.

The operator can intuitively determine to drive on the right side or the left side of the obstacle120according to the conditions of the road surface and the surroundings, and so on.

The first and second guidance routes GR1and GR2have different colors, shapes, or blink rates. As a result, the operator can easily recognize each of the guidance routes GR1and GR2.