Patent Description:
A forklift is for an operator to board and perform operations of carrying and unloading a cargo. Hence, many years of experience are required for the operator to operate the forklift quickly and accurately. On the other hand, since the number of skilled operators is limited, the number of skilled operators may be insufficient in a locality with a small population. Hence, it is required that a skilled operator remotely operate a local forklift from an area where there are many skilled operators.

Further, when handling a tank filled with a gas that adversely affects the human body or an explosive that explodes if dropped, it is preferred that the operator operates the forklift remotely away from the warehouse where the cargo is handled.

Therefore, a remote control system for remotely operating a forklift is known (see, for example, Patent Literature <NUM>). In the remote control system, an operation apparatus is provided in a remote base station away from the warehouse where the cargo is handled by the forklift, and the operator operates the forklift from the remote base station using the operation apparatus.

The forklift has a control device, a wireless communication device, a camera, a sensor, and a driving device for operating its traveling and the raising/lowering of its fork, etc. The operation apparatus in the base station includes a handle, a lever, a pedal, a control unit and a display unit, etc. For example, the display unit is provided with two display monitors, one of which displays an image of the front of the forklift taken by a camera mounted on the forklift, and the other of which displays the information detected by the sensor and so on.

When carrying and unloading with a forklift, the operator operates the forklift using the handle or the lever, etc., while checking the object to be handled in front of the forklift based on the image on the display unit. On the other hand, depending on the direction of the forklift, the object to be handled may not be visible from the display unit, so that there is a problem that operating the forklift is difficult. Further, if there is an obstacle between the forklift and the object, operating the forklift is also difficult, as it is required to approach the object in a manner of avoiding the obstacle while checking the positions of both the object and the obstacle.

Patent literature <CIT> relates to an industrial vehicle remote operation system having the features of preamble of claim <NUM>, the system including: an industrial vehicle including a camera; a remote operation device for remotely controlling the industrial vehicle; and a display unit including a display screen for enabling a camera picture imaged by the camera to be displayed. The industrial vehicle remote operation system includes: a target designation unit for designating a target displayed in the camera picture; a world coordinate acquisition unit for acquiring world coordinates of the target designated by the target designation unit; a world coordinate change unit for following movement of the industrial vehicle to change the world coordinates of the target; and a guide line drawing unit that on the camera picture, draws a guide line between the world coordinates of the industrial vehicle and the world coordinates of the target changed by the world coordinate change unit.

Patent literature <CIT> relates to a cargo handling system including a control device and an unmanned work vehicle. The control device includes a communication unit and a travel route determination unit for outputting first route information about a travel route of the unmanned work vehicle. The unmanned work vehicle includes: a recognition unit for acquiring current location information of a vehicle; a detection unit for acquiring obstacle information; a memory unit; a learning model unit for performing a machine learning so as to create second route information about a movable route of the vehicle; a processing unit for acquiring the second route information from the learning model unit by inputting the first route information and the obstacle information to the learning model unit; and a travel control unit for controlling the travel of the vehicle based on the second route information.

Patent literature <CIT> relates to an industrial vehicle travel assistance device that includes a camera, a travel guide generation part, a display part, and a switching part. The switching part is to switch from relative position display to absolute position display. In the relative position display, images captured by the camera are displayed by the display part with a travel guide superimposed onto the images such that the travel guide maintains the relative positional relationship thereof with an industrial vehicle. In the absolute position display, images captured by the camera are displayed by the display part with the travel guide superimposed onto the images such that the travel guide does not maintain the relative positional relationship thereof with the industrial vehicle and is fixed in an absolute position.

Patent literature <CIT> relates to a vehicle management device having a first communication circuit for communicating with each vehicle and a processing circuit for determining a traveling path for each vehicle and transmitting an instruction indicating the traveling path to each vehicle via the first communication circuit. Each vehicle has a second communication circuit, an obstacle sensor, and a controller for causing the vehicle to move according to the instruction received via the second communication circuit. When the obstacle sensor has detected an obstacle on the path, the controller notifies the presence of the obstacle to the outside via the second communication circuit. Upon receiving the notification indicating the presence of the obstacle from any one the plurality of vehicles, the processing circuit of the management device instructs the display to indicate the presence of the obstacle.

Accordingly, this invention provides a remote control system that allows a forklift to be operated intuitively and easily even if an obstacle is present between the forklift and the object, while the forklift is operated remotely.

The remote control system is for remotely operating a forklift, and includes a camera provided on the forklift, a display unit displaying images taken by the camera, a forklift coordinates acquisition section acquiring the coordinates of the forklift, an object coordinates acquisition acquiring the coordinates of a designated object, a judgement section judging whether or not an obstacle is present between the forklift and the object, and a guidance route display section displaying a guidance route connecting the forklift and the object on the display unit. The guidance route display section is configured to display a guidance route straightly connecting the forklift and the object if the judgement section judges that the obstacle is not present between the forklift and the object, or display a guidance route connecting the forklift and the object in a manner of avoiding the obstacle if the judgement section judges that the obstacle is present between the forklift and the object.

The judgement section forms a virtual route straightly connecting the forklift and the object and having a width corresponding to the width of the forklift, and judges that the obstacle is not present between the forklift and the object if the entirety or a portion of the obstacle is not present inside the virtual route, or that the obstacle is present between the forklift and the object if the entirety or a portion of the obstacle is present inside the virtual route, and the guidance route display section displays the guidance route in a manner that the entirety or a portion of the obstacle is not present inside the guidance route.

In an embodiment, the guidance route display section displays the guidance route having a width corresponding to the width of the forklift.

In an embodiment, the guidance route display section displays first and second guidance routes as the above guidance route, wherein the first guidance route avoids the right side of the obstacle and the second guidance route avoids the left side of the obstacle.

In an embodiment, the first guidance route and the second guidance route have different colors, shapes or blink rates.

With the remote control system of this invention, while the forklift is operated remotely, the forklift can be operated intuitively and easily even if an obstacle is present between the forklift and the object.

The remote control system according to an embodiment of this invention is described below with reference to the drawings.

Referring to <FIG>, the remote control system includes forklifts <NUM> that travel and do cargo handling work in the facility <NUM>. The remote control system is provided with a base station <NUM> provided at a remote location away from the facility <NUM>. The base station <NUM> is provided with an operation apparatus <NUM>. An operator can remotely operate a predetermined forklift <NUM> by using the operation apparatus <NUM>.

Although the facility <NUM> is a warehouse in this embodiment, the facility in this invention may alternatively be a factory or an outdoor work place, etc. In addition, though the forklifts <NUM> are reach forklifts in this embodiment, they may be counterbalance forklifts or the like alternatively. When an operator operates a forklift <NUM> from a remote base station <NUM>, for example, a tank filled with a gas that adversely affects the human body or an explosive that explodes if dropped can be loaded and unloaded in the facility <NUM>.

Referring to <FIG>, the forklift <NUM> includes a camera <NUM>, an obstacle sensor <NUM>, a laser scanner <NUM>, a driving device <NUM>, a control unit <NUM>, and a wireless communication unit <NUM>, etc. Referring to <FIG>, the control unit <NUM> includes an image processing section <NUM>, a forklift coordinates acquisition section <NUM>, a driving control section <NUM> and an obstacle coordinates acquisition section <NUM>, etc. The control unit <NUM> is composed of a CPU (as a central processing device), an input/output interface, ROM and RAM, etc., and stores a program for processing information. The camera <NUM>, the obstacle sensor <NUM>, the laser scanner <NUM>, the driving device <NUM> and the wireless communication unit <NUM> are connected to the control unit <NUM>.

The driving device <NUM> includes a traveling motor for driving the driving wheels <NUM> provided at the rear part of the body of the forklift <NUM>, a plurality of hydraulic cylinders for raising, lowering, tilting, advancing and retreating the fork <NUM> provided at the front part of the body of the forklift <NUM>, and so on. An operation signal from the operation apparatus <NUM> provided in the base station <NUM> is sent to the control unit <NUM> via the wireless communication unit <NUM>, this operation signal is processed by the driving control section <NUM>, and the driving device <NUM> of the traveling motor and the hydraulic cylinders is driven based on the operation of the operation apparatus <NUM>. As a result, the driving device <NUM> of the traveling motor, the hydraulic cylinders and so on is driven in conjunction with the operation of the operation apparatus <NUM> by the operator, and the forklift <NUM> can be operated.

The camera <NUM> is arranged at a position of the line of sight of an operator who operates on board the forklift <NUM>, and takes pictures of the front of the forklift <NUM> from the position of the line of sight of such operator. The camera <NUM> includes, for example, a CCD image sensor or a CMOS image sensor. The image taken by the camera <NUM> is processed by the image processing section <NUM> of the control unit <NUM> and displayed on the display unit <NUM> (<FIG>) provided in the operation apparatus <NUM>. Thereby, the operator who remotely controls the forklift <NUM> using the operating apparatus <NUM> at the base station <NUM> is able to confirm, by means of the display unit <NUM>, the front of the forklift <NUM> from the same line of sight as when he or she is on board the forklift <NUM> to operate the forklift <NUM>.

The forklift <NUM> is provided with a laser scanner <NUM>, and a plurality of reflectors <NUM> are installed in the facility <NUM>. The laser scanner <NUM> transmits/receives laser to/from the reflectors <NUM> while rotating the laser horizontally by <NUM>°. As a result, the forklift <NUM> can recognize the plurality of reflectors <NUM> arranged along the traveling path in the facility <NUM> using the laser scanner <NUM>. Here, the reflectors <NUM> are fixed to the walls in the facility <NUM>, and their position information stored in the map of the forklift coordinates acquisition section <NUM> of the control unit <NUM>. With the recognition of the plurality of reflectors <NUM> by the forklift <NUM> using the laser scanner <NUM>, the forklift coordinates acquisition section <NUM> can measure and acquire the position coordinates of the forklift <NUM> based on the principle of triangulation.

The forklift <NUM> is provided with an obstacle sensor <NUM>, which includes, for example, an optical sensor. The obstacle sensor <NUM> is capable of detecting an obstacle present in a predetermined area in front of the forklift <NUM> and also measure the distance and the direction from the forklift <NUM> to the obstacle. The obstacle coordinates acquisition section <NUM> of the control unit <NUM> can measure and acquire the position coordinates of the obstacle based on the detection signals of the obstacle sensor <NUM> and the position coordinates of the forklift <NUM>.

Referring to <FIG>, the operation apparatus <NUM> includes a display unit <NUM>, an operation unit <NUM>, an object designation unit <NUM>, a control unit <NUM>, and a wireless communication unit <NUM>, etc. The control unit <NUM> includes an image processing section <NUM>, a judgement section <NUM>, a guidance route display section <NUM>, and an object coordinates acquisition section <NUM>, etc. The control unit <NUM> is composed of a CPU (a central processing device), an input/output interface, ROM, RAM and so on, and stores a program for processing information. The display unit <NUM>, the operation unit <NUM>, the object designation unit <NUM> and the wireless communication unit <NUM> are connected to the control unit <NUM>.

The display unit <NUM> may include, for example, two display monitors, wherein one display monitor displays an image of the front of the forklift <NUM> taken by the camera <NUM> mounted on the forklift <NUM>, and the other display monitor displays information detected by various sensors and information necessary for operation, etc..

The operation unit <NUM> includes a handle, a lever and a pedal, etc., and the operator can operate the forklift in the same manner as when actually boarding the forklift and operating. The operation signal from the operation unit <NUM> is transmitted by the wireless communication unit <NUM> via the control unit <NUM> and received by the wireless communication unit <NUM> of the forklift <NUM>. Then, the operator can remotely operate the driving device <NUM> and so on of the forklift <NUM>, as described above, by operating the handle, the lever and so on of the operation unit <NUM>.

The object designation unit <NUM> is, for example, a mouse. As described later, after the operator uses the mouse as the object designating unit <NUM> to designate an object <NUM> with the pointer <NUM> displayed on the display unit <NUM>, the guidance route display section <NUM> displays, on the display unit <NUM>, a guidance route GR or guidance routes GR1 and GR2 connecting the designated object <NUM> and the forklift <NUM> (<FIG>, <FIG> and <FIG>). When the forklift <NUM> travels to the object <NUM>, the operator can virtually travel along the guidance route GR, GR1 or GR2 displayed on the display unit <NUM>, so that the forklift <NUM> can be operated intuitively and easily.

The first embodiment of the remote control system will be described below.

Referring to <FIG>, the forklift coordinates acquisition section <NUM> constantly measures and acquires the absolute coordinates (Fx, Fy) of the forklift <NUM> (step S1). An image taken by the camera <NUM> is always displayed on the display unit <NUM>. The operator remotely controls the forklift <NUM> by the operation unit <NUM> based on the image displayed on the display unit <NUM>.

Referring to <FIG>, after the objects <NUM> to be handled is displayed on the display unit <NUM>, the operator uses the mouse as the object designation unit <NUM> and the pointer <NUM> displayed on the display unit <NUM> to designate an object <NUM> (step S2). Specifically, after the operator puts the pointer <NUM> on the object <NUM> and clicks the mouse, the object <NUM> is designated.

The image processing section <NUM> identifies the boundary with the floor surface, the wall surface or the like to recognize the shape of the designated object <NUM> (step S3). Then, the object coordinates acquisition section <NUM> acquires the camera coordinates of the object <NUM> displayed on the display unit <NUM>, and the absolute coordinates (Ox, Oy) of the object <NUM> are measured and acquired based on the absolute coordinates (Fx, Fy) of the forklift <NUM> (step S4).

Referring to <FIG>, the judgement section <NUM> connects the absolute coordinates (Fx, Fy) of the forklift <NUM> and the absolute coordinates (Ox, Oy) of the object <NUM> to form a virtual route VR having a width corresponding to the width W of the forklift <NUM> (step S5). Then, the obstacle sensor <NUM> measures and acquires the coordinate position of the obstacle <NUM>, and the image processing section <NUM> identifies the boundary with the floor surface, the wall surface or the like, so that the shape of the obstacle <NUM> existing other than the object <NUM> is recognized (step S6). After that, the judgement section <NUM> judges whether or not the obstacle <NUM> is present between the forklift <NUM> and the object <NUM> (step S7). The presence or absence of the obstacle <NUM> is judged as follows.

The judgement section <NUM> judges that the obstacle <NUM> is not present between the forklift <NUM> and the object <NUM> if the entirety or a portion of the obstacle <NUM> is not present inside the formed virtual route VR (step S7). Because a travel route having a width corresponding to the width W of the forklift <NUM> is formed when the forklift <NUM> actually travels, with the virtual route VR having the width W, the actual travel route of the forklift <NUM> can be conceived virtually and intuitively.

Referring to <FIG> and <FIG>, if the judgement section <NUM> judges that the obstacle <NUM> is not present between the forklift <NUM> and the object <NUM>, a guidance route GR straightly connecting the forklift <NUM> and the object <NUM> is formed (step S8). The guidance route GR has a width corresponding to the width W of the forklift <NUM>, so that if it is judged that the obstacle <NUM> is not present, the guidance route GR coincides with the virtual route VR. Because a travel route having a width corresponding to the width W of the forklift <NUM> is formed when the forklift <NUM> actually travels, with the guidance route GR having the width W, the actual travel route of the forklift <NUM> can be conceived virtually and intuitively.

Referring shown in <FIG>, the guidance route display section <NUM> converts the formed guidance route GR into camera coordinates and displays it on the display unit <NUM>. Then, as the forklift <NUM> moves, the absolute coordinates (Fx, Fy) of the forklift <NUM> are changed, and the guidance route GR is displayed on the display unit <NUM> following the absolute coordinates (Fx, Fy) of the forklift <NUM> that are changed at any time.

On the other hand, the judgement section <NUM> judges that the obstacle <NUM> is present between the forklift <NUM> and the object <NUM> if the entirety or a portion of the obstacle <NUM> is present inside the formed virtual route VR (step S7). If the judgement section <NUM> judges that the obstacle <NUM> is present, then the guidance route GR connecting the forklift <NUM> and the object <NUM> is formed in a manner of avoiding the obstacle <NUM> (step S9). Referring to <FIG> and <FIG>, the guidance route GR has a width corresponding to the width W of the forklift <NUM>, and is formed by curves not overlapping with the recognized obstacle <NUM>. For example, a guidance route GR can be formed by connecting both end points of the forklift <NUM> with points separated from the obstacle <NUM> by predetermined distances and both end points of the object <NUM> by spline curves.

Referring to <FIG>, the guidance route display section <NUM> converts the formed guidance route GR into camera coordinates and displays it on the display unit <NUM>. Then, as the forklift <NUM> moves, the absolute coordinates (Fx, Fy) of the forklift <NUM> are changed, and the guidance route is displayed on the display unit <NUM> following the absolute coordinates (Fx, Fy) of the forklift <NUM> that are changed at any time.

A second embodiment of the remote control system will be described below. For the same configuration as the first embodiment, detailed description thereof is omitted to avoid duplicate explanations.

Referring to <FIG>, the forklift coordinates acquisition section <NUM> constantly measures and acquires the absolute coordinates (Fx, Fy) of the forklift <NUM> (step S1). Then, the operator designates an object <NUM> by the object designation unit <NUM> (step S2).

The image processing section <NUM> recognizes the shape of the designated object <NUM> (step S3). Then, the object coordinates acquisition section <NUM> measures and acquires the absolute coordinates (Ox, Oy) of the object <NUM> (step S4).

The judgement section <NUM> forms a virtual route VR connecting the absolute coordinates (Fx, Fy) of the forklift <NUM> and the absolute coordinates (Ox, Oy) of the object <NUM> (step S5). Then, the image processing section <NUM> recognizes the shape of the obstacle <NUM> (step S6). After that, the judgement section <NUM> judges whether or not the obstacle <NUM> is present between the forklift <NUM> and the object <NUM> (step S7). If the judgement section <NUM> judges that the obstacle <NUM> is not present, a guidance route GR straightly connecting the forklift <NUM> and the object <NUM> is formed as in the first embodiment (step S8).

On the other hand, if the judgement section <NUM> judges that the obstacle <NUM> is present, a curved guidance route GR connecting the forklift <NUM> and the object <NUM> is formed in a manner of avoiding the obstacle <NUM> (step S9). In the second embodiment, as shown in <FIG>, two guidance routes are formed at this time, including a first guidance route GR1 avoiding the right side of the obstacle <NUM>, and a second guidance route GR2 avoiding the left side of the obstacle <NUM>. As a result, the operator can intuitively determine to drive on the right side or the left side of the obstacle <NUM> while checking the conditions of the road surface and the surroundings and so on.

Referring to <FIG>, the guidance route display section <NUM> converts the formed first and second guidance routes GR1 and GR2 into camera coordinates and displays them on the display unit <NUM>. Then, as the forklift <NUM> moves, the absolute coordinates (Fx, Fy) of the forklift <NUM> are changed, and the first and second guidance routes GR1 and GR2 are displayed on the display unit <NUM> following the absolute coordinates (Fx, Fy) of the forklift <NUM> that are changed at any time. The first and second guidance routes GR1 and GR2 are displayed in different colors, shapes or blink rates to allow the operator to easily recognize each of the guidance routes GR1 and GR2.

Although the preferred embodiments of this invention have been described above, the configuration of this invention is not limited to them. For example, the configuration may be modified as follows.

Effects of this invention are described below.

The remote control system according to this invention includes a camera <NUM> provided on the forklift <NUM>, a display unit <NUM> displaying the images taken by the camera <NUM>, and a forklift coordinates acquisition section <NUM> acquiring the coordinates of the forklift <NUM>. Further, the remote control system designates an object <NUM> displayed on the display unit <NUM>, and also includes an object coordinates acquisition section <NUM> acquiring the coordinates of the object <NUM>, a judgement section <NUM> judging whether or not the obstacle <NUM> is present between the forklift <NUM> and the object <NUM>, and a guidance route display section <NUM> displaying on the display unit <NUM> a guidance route GR connecting the forklift <NUM> and the object <NUM>. The guidance route display section <NUM> is configured to display a guidance route GR straightly connecting the forklift <NUM> and the object <NUM> if the judgement section <NUM> judges that the obstacle <NUM> is not present between the forklift <NUM> and the object <NUM>, or display a guidance route GR connecting the forklift <NUM> and the object <NUM> in a manner of avoiding the obstacle <NUM> if the judgement section <NUM> judges that the obstacle <NUM> is present between the forklift <NUM> and the object <NUM>.

As described above, even when the obstacle <NUM> is present between the forklift <NUM> and the object <NUM>, a guidance route GR connecting the forklift <NUM> and the object <NUM> in a manner of avoiding the obstacle <NUM> is displayed on the display unit <NUM>. As a result, when the forklift <NUM> travels to the object <NUM>, the operator can virtually travel along the guidance route GR displayed on the display unit <NUM>, so that the forklift <NUM> can be operated intuitively and easily.

It is preferred that the judgement section <NUM> connects the forklift <NUM> and the object <NUM> to form a virtual route VR having a width corresponding to the width W of the forklift <NUM> and then judges that the obstacle <NUM> is not present if the entirety or a portion of the obstacle <NUM> is not present inside the virtual route VR, or judges that the obstacle <NUM> is present if the entirety or a portion of the obstacle <NUM> is present inside the virtual route VR, and the guidance route display section <NUM> displays the guidance route GR in a manner that the entirety or a portion of the obstacle <NUM> is not present inside the guidance route GR.

Because a travel route having a width corresponding to the width W of the forklift <NUM> is formed when the forklift <NUM> actually travels, with the virtual route VR having the width W, an actual travel route of the forklift <NUM> can be conceived virtually and intuitively.

Further, it is desirable that the guidance route display section <NUM> displays the guidance route GR having a width corresponding to the width W of the forklift <NUM>.

When the forklift <NUM> actually travels, a travel route having a width corresponding to the width W of the forklift <NUM> is formed. Since the guidance route GR has a width W, the operator can virtually and intuitively conceive an actual travel route of the forklift <NUM>, so that the forklift <NUM> can easily travel.

Furthermore, it is preferred that the guidance route display section <NUM> displays a first guidance route GR1 that avoids the right side of the obstacle <NUM> and a second guidance route GR2 that avoids the left side of the obstacle <NUM>.

The operator can intuitively determine to drive on the right side or the left side of the obstacle <NUM> according to the conditions of the road surface and the surroundings, and so on.

Claim 1:
A remote control system for remotely operating a forklift (<NUM>), comprising:
a camera (<NUM>), provided on the forklift (<NUM>);
a display unit (<NUM>), displaying images taken by the camera (<NUM>);
a forklift coordinates acquisition section (<NUM>), acquiring coordinates of the forklift (<NUM>);
an object coordinates acquisition section (<NUM>), acquiring coordinates of a designated object (<NUM>);
a judgement section (<NUM>), judging whether or not an obstacle (<NUM>) is present between the forklift (<NUM>) and the object (<NUM>); and
a guidance route display section (<NUM>), configured to display on the display unit (<NUM>) a guidance route (GR) straightly connecting the forklift (<NUM>) and the object (<NUM>) if the judgement section (<NUM>) judges that the obstacle (<NUM>) is not present between the forklift (<NUM>) and the object (<NUM>), or display on the display unit (<NUM>) a guidance route (GR) connecting the forklift (<NUM>) and the object (<NUM>) in a manner of avoiding the obstacle (<NUM>) if the judgement section (<NUM>) judges that the obstacle (<NUM>) is present between the forklift (<NUM>) and the object (<NUM>),
wherein the remote control system is characterized in that:
the judgement section (<NUM>) forms a virtual route (VR) straightly connecting the forklift (<NUM>) and the object (<NUM>) and having a width corresponding to a width (W) of the forklift (<NUM>), and judges
that the obstacle (<NUM>) is not present between the forklift (<NUM>) and the object (<NUM>) if an entirety or a portion of the obstacle (<NUM>) is not present inside the virtual route (VR), or
that the obstacle (<NUM>) is present between the forklift (<NUM>) and the object (<NUM>) if the entirety or a portion of the obstacle (<NUM>) is present inside the virtual route (VR), and
the guidance route display section (<NUM>) displays the guidance route (GR) in a manner that that entirety or a portion of the obstacle (<NUM>) is not present inside the guidance route (GR).