Patent Description:
Conventionally, an unmanned transport vehicle (i.e., automated guided vehicle, AGV) is used for transporting a weight object in a predetermined field such as a factory, warehouse, etc. In the conventional unmanned transport system, a flow line (or path, route) on which the unmanned transport vehicle should travel is designated with a tape or the like. The unmanned transport vehicle travels along this tape while detecting the tape, whereby traveling on the designated flow line to transport the weight object to a destination location in the field.

In the field, there may be a case where the destination location is need to be changed. In this case, the flow line is also need to be changed. However, when changing the flow line, the tape installed on the field is need to be re-installed, and then, a working burden is heavy.

In particular, in a case where the unmanned transport vehicle is a forklift configured to transport the weight object put on a pallet together with the pallet, since intervals between stored weight objects are changed when sizes of the pallets are changed, shifts in the destination locations occur, and such changing in the destination location is frequently performed. Therefore, it becomes extremely large that the burden of re-installing the tape and the like which serve as the flow line each time the sizes of the pallets are changed.

As the unmanned transport vehicle, there is an autonomous traveling transport vehicle configured not to travel along the designated flow line, but to travel so as to aim at the destination location while performing an autonomous travel by grasping a surrounding environment (obstruction, etc.) using sensing. However, it is dangerous that the unmanned transport vehicle being transporting the weight object can freely travel on a path in a field where a human is also passing through, such as a factory or warehouse.

The prior art document <CIT> discloses systems and methods for monitoring the movements of autonomous vehicles in warehouses and factories and taking actions to avoid collisions.

The prior art document <CIT> discloses a robot which includes a communication unit configured to communicate with external camera modules acquiring external images including the robot that is being driven, a drive-information acquiring unit configured to acquire driving related information at the time of driving the robot, a driving unit configured to drive the robot, and a control unit configured to control the driving unit using external information including the external images received from the external camera modules and the driving related information.

The prior art document <CIT> discloses systems and methods for closed-loop feedback control of controllable devices using motion capture systems.

The present invention aims to provide a new unmanned transport system for transporting a weight object to a destination location on the premise that it travels on a designated flow line.

The present invention provides a solution to the above mentioned problems according to the independent claim. Preferred embodiments are provided by the dependent claims. The embodiments and/or examples of the following description which are not covered by the claims, are provided for illustrative purpose only and are only intended to assist the reader in understanding the present invention. However, such embodiments and/or examples which are not covered by the claims do not form part of the present invention that is solely defined by the claims. An unmanned transport system according to an aspect of the present invention is an unmanned transport system configured to transport a weight object by traveling on a designated virtual flow line to a designated destination location, and having a configuration comprising: an unmanned transport vehicle configured to travel unmanned according to a control signal; one or more cameras configured to image a field on which the unmanned transport vehicle is to travel from above the field to generate a still image or video; and an operation control means configured to generate the control signal for controlling such that the unmanned transport vehicle travels on the virtual flow line based on the still image or video, and wherein the operation control means comprises: a detecting portion configured to detect the unmanned transport vehicle from the still image or video; a position grasping portion configured to grasp a position of the unmanned transport vehicle in the field without being based on a sensor equipped with the unmanned transport vehicle using a result of a detection by the detecting portion; and a control portion configured to generate the control signal based on the position of the unmanned transport vehicle which the position grasping portion grasps.

In the above unmanned transport system, the unmanned transport vehicle may be a forklift vehicle configured to lift and transport the weight object placed on a pallet together with the pallet.

In the above unmanned transport system, the field may be indoors, and the one or more cameras may be mounted on a ceiling or wall and configured to image the field from above.

In the above unmanned transport system, the field may be outdoors, and the one or more cameras may be mounted on an outdoor structure and configured to image the field from above.

In the above unmanned transport system, the operation control means may further comprise an orientation grasping portion configured to grasp an orientation of the unmanned transport vehicle based on the still image or video, and the control portion may be configured to generate the control signal based also on the orientation grasped by the orientation grasping portion.

In the above unmanned transport system, the unmanned transport vehicle may be configured to perform an autonomous operation for loading or unloading of the weight object at the destination location.

In the above unmanned transport system, the detecting portion of the operation control means may be configured to further detect an obstruction from the still image or video, and the control portion of the operation control means may be configured to further generate the control signal based also on a result of a detection by the detecting portion.

In the above unmanned transport system, the unmanned transport vehicle may include a manipulating portion which a user is to manipulate in order to start traveling on the flow line, and the control portion of the operation control means may be configured to start generating the control signal in accordance with a manipulation to the manipulating portion.

In the above unmanned transport system, the unmanned transport system may comprise a plurality of unmanned transport vehicles whose destination locations and virtual flow lines are designated respectively, the detecting portion of the operation control means may be configured to detect the plurality of unmanned transport vehicles from the still image or video, the position grasping portion of the operation control means may be configured to grasp respective positions of the plurality of unmanned transport vehicles in the field, and the control portion of the operation control means may be configured to generate the control signal for each of the plurality of unmanned transport vehicles based on the plurality of positions grasped by the position grasping portion.

The above unmanned transport system may comprise a plurality of cameras whose fields of view are different form each other, and the position grasping portion may be configured to grasp the position of the unmanned transport vehicle in the field by integrating a plurality of still images or videos generated by the plurality of cameras.

In the above unmanned transport system, the detecting portion may be configured to detect the unmanned transport vehicle by using a neural network.

In the above unmanned transport system, the unmanned transport vehicle may include a predetermined mark on a top surface thereof, and the detecting portion may be configured to detect the unmanned transport vehicle by detecting the predetermined mark.

In the above unmanned transport system, the operation control means may be configured to store a map of the field in which the flow line is defined, and the position grasping portion may be configured to grasp the position of the unmanned transport vehicle by mapping the unmanned transport vehicle detected by the detecting portion on the map.

The above unmanned transport system may comprise a predetermined mark at a predetermined position in the field, the detecting portion may be configured to detect the predetermined mark, and the position grasping portion may be configured to grasp the position of the unmanned transport vehicle based on the predetermined mark which the detecting portion detects.

In the above unmanned transport system, the detecting portion of the operation control means may be configured to further detect the weight object being transported by the unmanned transport vehicle.

In the above unmanned transport system, the unmanned transport vehicle may include: an in-vehicle detecting portion for detecting a surrounding obstruction; and an emergency stop processing portion configured to perform an emergency stop processing of the unmanned transport vehicle based on a detection of the obstruction by the in-vehicle detecting portion.

The above unmanned transport system may further comprise a derailment judging means configured to judge whether or not the unmanned transport vehicle deviates from the flow line, and the operation control means may be configured to generate the control signal so as to stop the unmanned transport vehicle or return the unmanned transport vehicle to the flow line when it is judged in the derailment judging means that the unmanned transport vehicle deviates from the flow line.

In the above unmanned transport system, the derailment judging means may include: an in-vehicle camera equipped with the unmanned transport vehicle and configured to image surroundings around the unmanned transport vehicle to generate the still image or video; and a judging portion configured to judge whether or not the unmanned transport vehicle deviates from the flow line based on the still image or video generated by the in-vehicle camera.

In the above unmanned transport system, the derailment judging means may include: a receiving instrument configured to receive a positioning signal; and a judging portion configured to judge whether or not the unmanned transport vehicle deviates from the flow line based on the positioning signal received by the receiving instrument.

Embodiments according to the present invention is described below with reference to drawings. The embodiments described below show examples when implementing the present invention, and do not limit the present invention to specific configurations described below. In implementing the present invention, specific configurations according to the embodiments may be adopted appropriately.

<FIG> is a plan view showing a field to which an unmanned transport system is applied, according to an embodiment of the present invention, and <FIG> is a side view showing the field to which the unmanned transport system is applied, according to the embodiment of the present invention. As a field <NUM> to which the unmanned transport system (i.e., automated guided system) according the present embodiment is applied, a factory, warehouse, or the like is expected. In the present invention, an unmanned transport vehicle (i.e., automated guided vehicle, AGV) <NUM> is a forklift vehicle configured to lift and transport a weight object <NUM> placed on a pallet <NUM> together with the pallet <NUM>.

The unmanned transport vehicle <NUM> transports the weight object, traveling on a designated virtual flow line, toward a designated destination location, in the field <NUM>. In the field <NUM>, for example, manufacturing equipment <NUM> is installed, and a worker <NUM> may being walking. The manufacturing equipment <NUM> and worker <NUM> may be obstructions as the unmanned transport vehicle <NUM> travels within the field <NUM>. The weight object <NUM> and pallet <NUM> placed in the field <NUM> can also become obstructions, and in a case where a plurality of unmanned transport vehicles <NUM> travels in the field <NUM>, other unmanned transport vehicle <NUM> can also become obstruction.

The field <NUM> is inside a building (indoors), and a plurality of overlooking cameras <NUM> are mounted on a ceiling <NUM>. The plurality of overlooking cameras <NUM> images the field <NUM> from above, respectively. The overlooking cameras <NUM> may be installed at a wall, pillar, pole, or other high point of the building. In the field <NUM>, a plurality of marks <NUM> is also provided at positions visible from the overlooking cameras <NUM>.

For each of the plurality of overlooking cameras <NUM>, areas of the field <NUM> which the overlooking cameras <NUM> can image are partially overlapped (referring to <FIG>), whereby the imaged areas of the plurality of overlooking cameras <NUM> covers the entire field <NUM>. It is not necessarily required to image the entire field <NUM>, and some areas which are not imaged by any of the overlooking cameras <NUM> are included in the field <NUM>. That is, areas of the field <NUM> where the unmanned transport vehicle <NUM> is not scheduled to travel may not be included in the imaged area of any of the overlooking cameras <NUM>.

The unmanned transport vehicle <NUM> loads at a loading position and unloads at a destination location, and then, transports the weight object <NUM> to the destination location with the loading position as a starting location. The unmanned transport vehicle <NUM> which completed the transporting returns to the loading position in a state where it is not being carrying the weight object <NUM>, for the next transport. In this case, the unmanned transport vehicle <NUM> moves with the unloading position as the starting location and the loading position as the destination location (a non-loading returning travel). In the unmanned transport system according to the present embodiment, such travel for the transporting of the weight object <NUM> and the non-loading returning travel of the unmanned transport vehicle <NUM> is performed unmanned.

A virtual flow line <NUM> is set in the field <NUM>. It is noted that this flow line <NUM> is virtual and is not provided as a physical entity in the field <NUM>. In a case of a conventional unmanned transport system in which a flow line as a physical entity (hereinafter referred to as a "physical flow line") is set in the field <NUM>, an unmanned transport vehicle <NUM> can follow the physical flow line and travel to a destination location by traveling such that it does not deviate from the physical flow line while detecting the physical flow line.

However, in such conventional unmanned transport system, when the travel route of the unmanned transport vehicle <NUM> is changed, the physical flow line is need to be changed, and then, its working burden is heavy. Therefore, in the unmanned transport vehicle system <NUM> according to the present embodiment, the virtual flow line <NUM> is set without using the physical flow line, and the unmanned transport vehicle <NUM> is controlled such that the unmanned transport vehicle <NUM> can travel on this virtual flow line <NUM>.

<FIG> is a perspective view showing the unmanned transport vehicle in the unmanned transport system according to the embodiment of the present invention. As shown in <FIG>, the unmanned transport vehicle <NUM> includes a fork <NUM> at its front end, and the fork <NUM> is configured to be movable up and down. An in-vehicle camera <NUM> capable of imaging the front is mounted in the unmanned transport vehicle <NUM> in order to image surroundings of the unmanned transport vehicle <NUM>. Moreover, a mark <NUM> is provided on the upper surface of the unmanned transport vehicle <NUM>.

Moreover, individual identification information is given to each of the unmanned transport vehicles <NUM>, and this identification information is expressed on the upper surface of the unmanned transport vehicle <NUM> as an identification information notation <NUM> such that it can be imaged by the overlooking cameras <NUM> above. In the example in <FIG>, the two unmanned transport vehicles <NUM> are given the identification information "<NUM>" and "<NUM>", respectively, and the identification information notations <NUM> representing the identification information are marked on the top surfaces of the unmanned transport vehicles <NUM>. The identification information notation <NUM> may be a code generated by encoding the identification information (e.g., bar code, two-dimensional code, etc.).

<FIG> is a block diagram showing a configuration of the unmanned transport system according to the embodiment of the present invention. As shown in <FIG>, the unmanned transport system <NUM> according to the present embodiment includes the plurality of overlooking cameras <NUM>, an operation control portion <NUM>, and the unmanned transport vehicle <NUM>. The plurality of overlooking cameras <NUM> are connected to the operation control device <NUM> by wired cables, respectively. The overlooking camera <NUM> images the field <NUM> on which the unmanned transport vehicle <NUM> travels, from above the field <NUM>, and then generates a video. The overlooking camera <NUM> transmits the generated video stream to the operation control device <NUM> in real time, and the operation control device <NUM> receives the video streams from the plurality of overlooking cameras <NUM>.

The overlooking camera <NUM> may be wirelessly connected to the operation control device <NUM>, and transmit and receive the video stream wirelessly. The overlooking camera <NUM> may also continuously perform still imaging to generate continuous still images, and transmit the generated still images to the operation control device <NUM> in real time. Unique identification information is assigned to each of the plurality of overlooking cameras <NUM>. The operation control device <NUM> identifies which the overlooking cameras <NUM> the still image or video is from by using the identification information of the overlooking camera <NUM>.

The operation control device <NUM> generates a control signal for controlling such that the unmanned transport vehicle <NUM> travels on the virtual flow line <NUM> based on the still image or video transmitted from the overlooking camera <NUM>, and transmits it to the unmanned transport vehicle <NUM>. The operation control device <NUM> may be provided within or outside the field <NUM>, or it may be provided in the cloud and communicate with the overlooking camera <NUM> and the unmanned transport vehicle <NUM> via the Internet. Furthermore, the operation control device <NUM> may be provided in the overlooking camera <NUM> or the unmanned transport vehicle <NUM>. In this way, the operation control device <NUM> may be located anywhere as long as it can transmit data (the video stream, still image, control signal, etc.) between the overlooking camera <NUM> and the unmanned transport vehicle <NUM>.

The operation control device <NUM> includes a detecting portion <NUM>, a position grasping portion <NUM>, an orientation grasping portion <NUM>, a control portion <NUM>, and a storage portion <NUM>. The detecting portion <NUM> detects the unmanned transport vehicle <NUM> from the still image or video. Since the operation control device <NUM> acquires the still images or videos from the plurality of overlooking cameras <NUM>, the detecting portion <NUM> detects the unmanned transport vehicle <NUM> from each of the still images or videos obtained from the plurality of overlooking cameras <NUM>.

The detecting portion <NUM> inputs the still image or video into a learned neural network, whereby detecting the unmanned transport vehicle <NUM> from the still image or video. In the case where the operation control device <NUM> acquires the video from the overlooking camera <NUM>, the detecting portion <NUM> may extract frame images from the video at a predetermined interval and detect the unmanned transport vehicle <NUM> from those frame images (still images).

Since the predetermined mark <NUM> is provided on the upper surface of the unmanned transport vehicle <NUM>, the detecting portion <NUM> may detect the unmanned transport vehicle <NUM> from the still image or video with this mark <NUM> as a clue. That is, the detecting portion <NUM> may detect the mark <NUM> on the unmanned transport vehicle <NUM> as the unmanned transport vehicle <NUM>.

The detecting portion <NUM> further detects the identification information notation <NUM> of the unmanned transport vehicle <NUM>. As shown in the example in <FIG>, in the case where the identification information is expressed in characters (including numbers), the detecting portion <NUM> performs character recognition on the detected identification information notation <NUM>, whereby identifying the identification information of the detected unmanned transport vehicle <NUM>. In the case where the identification information is expressed as the code information, the detecting portion <NUM> decodes the detected identification information notation <NUM> whereby identifying the identification information of the detected unmanned transport vehicle <NUM>. The mark <NUM> may also double as the identification information notation <NUM>.

The position grasping portion <NUM> grasps a position of the unmanned transport vehicle <NUM> in the field <NUM> by using a result of the detection by the detecting portion <NUM>. Although the unmanned transport vehicle <NUM> is equipped with an in-vehicle camera <NUM>, the position grasping portion <NUM> grasps the position of the unmanned transport vehicle <NUM> in the field <NUM> based on the detection result from the still image or video imaged by the overlooking camera <NUM>, without being based on the in-vehicle camera <NUM> or other in-vehicle sensor equipped with the unmanned transport vehicle <NUM>.

The storage portion <NUM> stores a map of the field <NUM>. In this map, the flow line along which the unmanned transport vehicle <NUM> should travel is defined. The position grasping portion <NUM> maps on this map the unmanned transport vehicle <NUM> detected by the detecting portion <NUM>, whereby grasping the position of the unmanned transport vehicle <NUM>. For this mapping, a predetermined mark <NUM> is set at predetermined positions in the field <NUM> and imaged by the overlooking camera <NUM>. The detecting portion <NUM> detects the mark <NUM>, and the position grasping portion <NUM> grasps the position of the unmanned transport vehicle <NUM> based on the mark <NUM> detected by the detecting portion <NUM>.

The position grasping portion <NUM> may integrate the still images or videos of the plurality of areas which are different from each other and acquired from the plurality of overlooking cameras <NUM>, in accordance with the identification information of the overlooking cameras <NUM>, to generate a still image or video of the entire field <NUM>, and grasp the position of the unmanned transport vehicle <NUM> in the field <NUM> using such integrated image.

The orientation grasping portion <NUM> grasps an orientation of the unmanned transport vehicle <NUM> based on the still image or video. The orientation grasping portion <NUM> may input the still image or video into a learned neural network, whereby detecting the orientation of the unmanned transport vehicle <NUM> from the still image or video. The orientation grasping portion <NUM> may also grasp an orientation of the mark <NUM> of the unmanned transport vehicle <NUM> detected by the detecting portion <NUM>, whereby grasping the orientation of the unmanned transport vehicle <NUM>.

The control portion <NUM> generates the control signal to control the travelling of the unmanned transport vehicle <NUM> based on the position of the unmanned transport vehicle <NUM> which the position grasping portion <NUM> grasps and the orientation of the unmanned transport vehicle <NUM> which the orientation grasping portion <NUM> grasps. Specifically, the control portion <NUM> generates the control signal including an instruction of a forward moving, stopping, or steering based on the position and orientation of the unmanned transport vehicle <NUM> such that the unmanned transport vehicle <NUM> travels toward the destination location on the flow line <NUM> defined in the map. The generated control signal is transmitted in real time to the corresponding unmanned transport vehicle <NUM>.

The unmanned transport vehicle <NUM> receives the control signal from the operation control device <NUM> and travels in accordance with the control signal. The unmanned transport vehicle <NUM> includes a travel control portion <NUM>, a lift control portion <NUM>, an in-vehicle detecting portion <NUM>, an emergency stop processing portion <NUM>, a judging portion <NUM>, an autonomous operation processing portion <NUM>, a manipulating portion <NUM>, a travel driving portion <NUM>, a lift driving portion <NUM>, an in-vehicle camera <NUM>, and a receiving instrument <NUM>.

The travel driving portion <NUM> includes a power source, a power transmission mechanism, a steering mechanism, a speed change mechanism, a braking mechanism, wheels, and the like for traveling the unmanned transport vehicle <NUM>. The lift driving portion <NUM> includes a drive source and power transmission mechanism for moving the fork <NUM> up and down, a child lock mechanism for fixing the fork <NUM> in a predetermined position, and the like. The travel driving portion <NUM> is driven in accordance with controlling by the travel control portion <NUM>. The lift driving portion <NUM> is driven in accordance with controlling by the lift control portion <NUM>.

The travel control portion <NUM> drives the travel driving portion <NUM> in accordance with the control signal received from the operation control device <NUM>, whereby performing controlling in order that the unmanned transport vehicle <NUM> travels along the set virtual flow line <NUM>. In the unmanned transport vehicle system <NUM> according to the present embodiment, what is necessary for the unmanned transport vehicle <NUM> to travel along the virtual flow line <NUM> is the travel control portion <NUM> and travel driving portion <NUM>, and the other elements (e.g., the in-vehicle detecting portion <NUM> and in-vehicle camera <NUM>) are additional elements.

The in-vehicle detecting portion <NUM> is used for an emergency stop of the unmanned transport vehicle <NUM>. The in-vehicle detecting portion <NUM> may be, for example, an infrared sensor or LiDAR. When there is an obstruction in front of a traveling direction of the unmanned transport vehicle <NUM>, the in-vehicle detecting portion <NUM> detects it. Alternatively, when there is an obstruction around in all directions of the unmanned transport vehicle <NUM> not limiting to in front of the traveling direction, the in-vehicle detecting portion <NUM> may detect it.

The emergency stop processing portion <NUM> performs emergency stop processing of the unmanned transport vehicle <NUM> when the in-vehicle detecting portion <NUM> detects the obstruction. As the emergency stop processing, the emergency stop processing portion <NUM> specifically controls the traveling drive portion <NUM> to stop generation of power from the power source and to brake. At this time, since abrupt braking may cause the pallet <NUM> to come loose from the forks <NUM> and the heavy load <NUM> to detach from the unmanned transport vehicle <NUM>, the braking is performed with a strength so as not to cause such accident.

The obstruction may be detected by the operation control device <NUM>, and the control signal may be generated by the control portion <NUM>, taking into account the presence of this obstruction. In this case, the detecting portion <NUM> detects the obstruction from the still image or video obtained from the overlooking camera <NUM>. The control portion <NUM> generates the control signal based also on the result of the detection of the obstruction by the detecting portion <NUM>.

At this time, in the present embodiment, controlling such that the unmanned transport vehicle <NUM> travels deviating from the set flow line <NUM> to avoid the obstruction are not performed, and when the unmanned transport vehicle <NUM> cannot travel on the flow line <NUM> due to the obstruction, it is stopped on the place. This is because the unmanned transport vehicle <NUM> according to the present embodiment transports the weight object <NUM>, and it is undesirable from a safety perspective that the unmanned transport vehicle <NUM> travel in an arbitrary space, deviating from the flow line <NUM> which was set in advance. At this time, the unmanned transport vehicle <NUM> may activate an alarm not shown and sound an alarm in order to inform surroundings around the unmanned transport vehicle <NUM> that it cannot move forward on the flow line <NUM> due to the obstruction.

As is above, in the unmanned transport system <NUM> according to the present embodiment, the unmanned transport vehicle <NUM> can travel on the set virtual flow line <NUM> by controlling the travel of the unmanned transport vehicle <NUM> based on the still image or video obtained by the overlooking camera <NUM>, and if there is an obstruction in a travel path of the unmanned transport vehicle <NUM>, the unmanned transport vehicle <NUM> itself detects it and performs the emergency stop.

Even through the unmanned transport vehicle <NUM> is in an emergency stop state, the control portion <NUM> may make the unmanned transport vehicle <NUM> travel again when recognizing that the event which caused the unmanned transport vehicle <NUM> the emergency stop state is resolved. As in the past, in a case where a human determines that the event which caused the unmanned transport vehicle <NUM> the emergency stop state is resolved and processing to resume travelling from the emergency stop state depends on a manual operation, its reliability become very low. The control portion <NUM> may recognize that the event which caused the emergency stop state is resolved when the obstruction is no longer detected by the in-vehicle detecting portion <NUM> or in-vehicle camera <NUM> of the unmanned transport vehicle <NUM>, or it may recognize that the event which caused the emergency stop state is resolved based on the still image video of the overlooking camera <NUM>.

As is described above, it is generally expected that the unmanned transport vehicle <NUM> travels or stops while being always on the flow line <NUM>. However, it is also expected that the unmanned transport vehicle <NUM> may deviate (derail) from the flow line <NUM> for some reason. Therefore, the unmanned transport system <NUM> according to the present embodiment has a function for determining a derailment of the unmanned transport vehicle <NUM>.

The control portion <NUM> judges whether or not the position grasped by the position grasping portion <NUM> deviates from the flow line <NUM> defined in the map stored in the storage portion <NUM>. When judging that the position deviates from the flow line <NUM>, the control portion <NUM> generates the control signal to stop the unmanned transport vehicle <NUM> and return the unmanned transport vehicle <NUM> to the flow line <NUM>.

In this way, the operation control device <NUM> may grasp the position of the unmanned transport vehicle <NUM> based on the still image or video from the overlooking camera <NUM> and judge the derailment, but in addition to or instead of this, it may include a derailment judging system not using the still image or video from the overlooking camera <NUM>. In this case, the control portion <NUM> also generates the control signal to stop the unmanned transport vehicle <NUM> and return the unmanned transport vehicle <NUM> to the flow line <NUM> in accordance with judging by the derailment judging system that the unmanned transport vehicle <NUM> deviates from the flow line <NUM>.

A first derailment judging system is formed of the in-vehicle camera <NUM> and the judging portion <NUM>. The in-vehicle camera <NUM> is equipped with the unmanned transport vehicle <NUM> and images around the unmanned transport vehicle <NUM> to generate a still image or video. The judging portion <NUM> judges whether or not the unmanned transport vehicle <NUM> deviates from the flow line <NUM> based on the still image or video generated by the in-vehicle camera <NUM>. Specifically, the judging portion <NUM> judges whether or not the image of the in-vehicle camera <NUM> is an image which should be imaged when the unmanned transport vehicle <NUM> is in the flow line <NUM>, whereby judging whether or not the unmanned transport vehicle <NUM> deviates from the flow line <NUM>.

A second derailment judging system is formed of the receiving instrument <NUM> and the judging portion <NUM>. The receiving instrument <NUM> receives a positioning signal propagating within the field <NUM>. The positioning signal is generated within the field <NUM>. Various conventional indoor positioning techniques were proposed and employed, and any conventional indoor positioning technique may be employed as the second derailment judging system.

For example, the receiving instrument <NUM> may receive a Wi-Fi radio wave as the positioning signal. In this case, a plurality of base stations transmitting the Wi-Fi radio waves are installed in the field <NUM>. The receiving instrument <NUM> receives the Wi-Fi radio waves from the plurality of base stations, and the judging portion <NUM> grasps the position of the unmanned transport vehicle <NUM> from differences in strength of the Wi-Fi radio waves which are from the plurality of base stations and received by the receiving instrument <NUM>, and maps it in the map in which the flow line <NUM> is defined, thereby judging whether or not the unmanned transport vehicle <NUM> deviates from the flow line <NUM> defined in the map.

Moreover, for example, the receiving instrument <NUM> may receive a positioning signal from a beacon installed in the field <NUM>. In this case, a plurality of beacons is installed in the field <NUM>. The judging portion <NUM> grasps the position of the unmanned transport vehicle <NUM> within the field <NUM> based on strength of the positioning signals received from the plurality of beacons, respectively, and maps it in the map in which the flow line <NUM> is defined, whereby judging whether or not the unmanned transport vehicle <NUM> deviates from the flow line <NUM> defined in the map.

The derailment judging system may use any other conventional positioning technique, such as positioning using magnetic sensor, positioning using IMES (Indoor MEssaging System) based on the same principle as GPS, positioning using PDR (Pedestrian Dead Reckoning), positioning using visible light, positioning using ultrasonic, etc..

As is above, in the unmanned transport system <NUM> according to the present embodiment, while it is principle to make the unmanned transport vehicle <NUM> travel on the virtual flow line <NUM>, it is possible to control the travel of the unmanned transport vehicle <NUM> to return to the flow line <NUM> by detecting the derailment even though it derailed due to some reasons.

When the unmanned transport vehicle <NUM> transports the weight object <NUM>, the weight object <NUM> is loaded onto the unmanned transport vehicle <NUM> at the starting location and the weight object <NUM> is unloaded from the unmanned transport vehicle <NUM> at the destination location. When the unmanned transport vehicle <NUM> turns from the unloading location to the loading location, the weight object <NUM> is unloaded from the unmanned transport vehicle <NUM> at the starting location and the weight object <NUM> is loaded onto the unmanned transport vehicle <NUM> at the destination location.

The unmanned transport vehicle <NUM> may perform autonomously these loading and unloading at the stating location and destination location. The autonomous operation processing portion <NUM> of the unmanned transport vehicle <NUM> performs an autonomous operation for loading and unloading of the weight object <NUM> at the starting location or destination location. For this purpose, when performing the non-loading returning travel to arrive at the starting location, the autonomous operation processing portion <NUM> detects the weight object <NUM> to be loaded from an image of the in-vehicle camera <NUM> by analyzing the image. The autonomous operation processing portion <NUM> controls the travel driving portion <NUM> and the lift driving portion <NUM> according to the detected weight object <NUM> to perform the loading.

When performing the transport operation to reach at the destination location, the autonomous operation processing portion <NUM> detects a position at which it should perform the unloading from an image of the in-vehicle camera <NUM> by analyzing the image. The autonomous operation processing portion <NUM> controls the travel driving portion <NUM> and the lift driving portion <NUM> according to the detected position where it should perform the unloading to perform the unloading.

The unmanned transport vehicle <NUM> includes the manipulating portion <NUM> for a user to operate to start the traveling on the flow line <NUM>. When an operation to start the traveling is performed by the user to the manipulating portion <NUM>, an operation signal is sent from the unmanned transport vehicle <NUM> to the operation control device <NUM>. The control portion <NUM> which received the operation signal starts generating the control signal. In the case where the loading and unloading are performed by the autonomous operation as described above or a case where the loading and unloading are not performed by the autonomous operation and the loading and unloading are performed by a manual manipulation, the control portion <NUM> may start the travelling control in accordance with the operation signal. The manipulating portion <NUM> may be a terminal independent of the unmanned transport vehicle <NUM>, for example, a tablet terminal. In the case where the loading and unloading are performed by the autonomous operation, the traveling may be started automatically when the loading or unloading is completed without depending on the operation signal from the manipulating portion <NUM>.

As is described above, the plurality of unmanned transport vehicles <NUM> to which the different destination locations and different flow lines are designated, respectively, are placed in the field <NUM>, and the operation control device <NUM> controls the plurality of unmanned transport vehicles <NUM> simultaneously. For this purpose, the detecting portion <NUM> detects the plurality of unmanned transport vehicles <NUM> from the still image or video of the overlooking camera <NUM>. The position grasping portion <NUM> grasps the respective positions of the plurality of unmanned transport vehicles <NUM> in the field <NUM>. The control portion <NUM> generates the control signal for each of the plurality of unmanned transport vehicles <NUM> based on the plurality of positions grasped by the position grasping portion <NUM>.

In this way, it is possible to control simultaneously the plurality of unmanned transport vehicles <NUM> in the single field <NUM>. By considering other unmanned transport vehicle <NUM> as the obstruction, when there is other unmanned transport vehicle <NUM> in front of the travelling direction of one unmanned transport vehicle <NUM>, the other unmanned transport vehicle <NUM> may be detected as the obstruction and the one unmanned transport vehicle <NUM> can be make stopped emergency. In this way, it possible to set the virtual flow lines <NUM> set for the plurality of unmanned transport vehicles <NUM> so as to intersect each other.

In the above embodiment, when controlling the plurality of unmanned transport vehicles <NUM> simultaneously, by providing the identification information notations <NUM> on the upper surfaces of the unmanned transport vehicles <NUM>, the operation control device <NUM> detects the identification information notation <NUM> from the still image or video of the overlooking camera <NUM>, grasps the position of each unmanned transport vehicle <NUM>, and generates the control signal for each of the unmanned transport vehicles <NUM>. That is, the identification information notations <NUM> are used for controlling each of the unmanned transport vehicles <NUM> individually. Alternatively, a configuration in which the unmanned transport vehicle <NUM> does not include the identification information notation <NUM> may be.

In this case, the unmanned transport vehicle <NUM> transmits a in-vehicle camera image of the in-vehicle camera <NUM> to the operation control device <NUM>. The control portion <NUM> of the operation control device <NUM> analyzes the in-vehicle camera image, whereby recognizing which the unmanned transport vehicle <NUM>'s position the image is from. In this way, the operation control device <NUM> can recognize that the unmanned transport vehicle <NUM> detected from the image of the overlooking camera <NUM> and the unmanned transport vehicle <NUM> which send the in-vehicle camera image are the same unmanned transport vehicle <NUM>. Then, it become possible for the operation control device <NUM> to send the control signal for each of the unmanned transport vehicles <NUM> detected from the images of the overlooking camera <NUM> to the corresponding unmanned transport vehicle <NUM>.

The detecting portion <NUM> described above may not only detect the unmanned transport vehicle <NUM> from the still image or video, but also the weight object being transported by the unmanned transport vehicle <NUM>. Whereby, it is possible to check whether the unmanned transport vehicle <NUM> is transporting the weight object <NUM>, that is, whether it is in the transporting or the non-loading returning traveling, and whether the weight object <NUM> being transported falls from the unmanned transport vehicle <NUM>.

As is explained above, by the unmanned transport system <NUM> according to the present embodiment, since the position of the unmanned transport vehicle <NUM> is grasped using the overlooking camera <NUM> and the travelling of the unmanned transport vehicle <NUM> is controlled referring to the map, it is possible for the unmanned transport vehicle <NUM> to travel along the virtual flow line <NUM> from the starting location to the destination location without using the physical flow line. This is effective, in particular, in the case where it is necessary to change the flow line. This is because it is not necessary to physically change the flow line in the field <NUM>, but only necessary to change the flow line on the map, when changing the flow line. Here, a situation to change the flow line will be explained.

<FIG> and <FIG> are views showing an example of the change in the flow line according to the embodiment of the present invention. In an example in <FIG>, relatively small pallets <NUM> (weight objects <NUM>) are transport targets, and in an example in <FIG>, relatively large pallets <NUM> (weight objects <NUM>) are transport targets, and so when the weight objects <NUM> are stored side by side at the destination location, a distance between adjacent weight objects <NUM>, that is, a distance between the flow lines <NUM> toward the storage location of the adjacent weight objects (destination location) are different from each other. In the example in <FIG>, the distance between the flow lines 210a toward the storage location which is the destination location is relatively short, and in the example in <FIG>, the distance between the flow lines 210b toward the storage location which is the destination location is relatively long.

In a state in which the weight object <NUM> is being transported with such field <NUM> and weight object <NUM> being target, when a size of the pallets <NUM> (weight objects <NUM>) is changed, the distance between the flow lines <NUM> is also need to be changed. For example, when the size of the pallets <NUM> (weight objects <NUM>) is increased, the distance between the flow lines <NUM> is also need to be increased, and the flow lines is need to be changed from the flow lines 210a shown in <FIG> to the flow lines 210b shown in <FIG>. In such a case, by the present embodiment, since both flow lines 210a and flow lines 210b are not physically provided in the actual field <NUM>, but are defined on the map stored in the storage portion <NUM>, it is possible to easily change.

<FIG> is a side view showing another example of a field to which the unmanned transport system is applied, according to the embodiment of the present invention. In the above embodiment, the example in which the field <NUM> is inside the building and the overlooking camera <NUM> is mounted on the ceiling <NUM> is explained, but the unmanned transport system <NUM> according to the present embodiment can also be used outdoors. In this case, however, since there is no ceiling, the overlooking camera <NUM> is mounted on a high location of a pole <NUM>, an exterior wall of the building, or other tall structure to image the field <NUM>.

The control portion <NUM> may control a speed of the unmanned transport vehicle <NUM> based on the still image or video of the overlooking camera <NUM>. The detecting portion <NUM> detects the obstructions (including the worker <NUM>, manufacturing equipment <NUM>, stored weight object <NUM>, other unmanned transport vehicle <NUM>, etc.) together with the unmanned transport vehicle <NUM> which is a control target, from the still image or video of the overlooking camera <NUM>. When there are no obstructions within a predetermined range around the unmanned transport vehicle <NUM> which is the control target, the control portion <NUM> generates the control signal to make it travel with its travel speed increased. Alternatively, since the unmanned transport vehicle <NUM> travels on the flow line <NUM>, the control portion <NUM> may judge the presence or absence of the obstructions within a predetermined range along the flow line <NUM> from the unmanned transport vehicle <NUM>.

In this way, it is possible to shorten a cycle time of the transport working by increasing the speed of the unmanned transport vehicle <NUM> while confirming that surroundings around the unmanned transport vehicle <NUM> are safe. This is effective in a factory to improve production efficiency. As the controlling of the speed, for example, controlling may be performed such that the unmanned transport vehicle <NUM> travels at a relatively low speed when moving forward with the fork <NUM> in front of it and at a relatively high speed when moving backward toward the opposite side of the fork <NUM>.

In the case controlling so as to change the speed in accordance with the presence or absence of the surrounding obstruction in this way, controlling based on the still image or video from the overlooking camera <NUM> is also effective. That is, in a case confirming safety by detecting the surrounding obstruction by a sensor mounted on the unmanned transport vehicle <NUM> (in-vehicle camera, LiDAR, etc.), since there may be a case where it is not possible to detect the obstruction due to a blind spot of the sensor, or a case where it is not possible to detect the obstruction relatively far away, it is not possible to ensure safety. Then, it is possible to set a necessary range around the unmanned transport vehicle <NUM> and check whether there is an obstruction within that range, by using the overlooking camera <NUM>.

Regardless of whether there is an obstruction around the unmanned transport vehicle <NUM>, a speed may be set for each section in the flow line defined on the map, and the control signal may be generated in accordance with the speed set on the map based on the position of the unmanned transport vehicle <NUM>. For example, a relatively high speed may be set for a section which does not intersect with the flow line <NUM> of other unmanned transport vehicle <NUM>, and a relatively low speed may be set for a section which intersects with the flow line <NUM> of other unmanned transport vehicle <NUM> or cross an area where the worker <NUM> is passing through.

The above operation control device <NUM> may be realized by a computer, and the detecting portion <NUM>, position grasping portion <NUM>, orientation grasping portion <NUM>, and control portion <NUM> may be functions realized by executing one or more operation control programs according to the present embodiment with a processor. The travel control portion <NUM>, lift control portion <NUM>, emergency stop processing portion <NUM>, judging portion <NUM>, and autonomous operation processing portion <NUM> of the unmanned transport vehicle <NUM> may be functions realized by executing one or more operation control programs according to the present embodiment with a processor. The travel control portion <NUM>, lift control portion <NUM>, emergency stop processing portion <NUM>, judging portion <NUM>, and autonomous operation processing portion <NUM> do not necessarily have to be provided in the unmanned transport vehicle <NUM>, and, for example, they may be provided in a service provider in the cloud, and the unmanned transport vehicle <NUM> may communicate with the service provider.

Claim 1:
An unmanned transport system (<NUM>) configured to transport a weight object (<NUM>) by traveling on a designated virtual flow line (<NUM>) to a designated destination location, the unmanned transport system (<NUM>) comprising:
an unmanned transport vehicle (<NUM>) configured to travel unmanned according to a control signal;
one or more cameras (<NUM>) configured to image a field (<NUM>) on which the unmanned transport vehicle (<NUM>) is to travel from above the field (<NUM>) to generate a still image or video; and
an operation control means including a storage portion (<NUM>) wherein the virtual flow line (<NUM>) is defined in a map of the field (<NUM>) stored in the storage portion (<NUM>) without using a physical flow line provided as a physical entity in the field (<NUM>), and configured to generate the control signal for controlling such that the unmanned transport vehicle (<NUM>) travels on the virtual flow line (<NUM>) based on the still image or video, and
wherein the operation control means comprises:
a detecting portion (<NUM>) configured to detect the unmanned transport vehicle (<NUM>) from the still image or video;
a position grasping portion (<NUM>) configured to grasp a position of the unmanned transport vehicle (<NUM>) in the field (<NUM>) without being based on a sensor equipped with the unmanned transport vehicle (<NUM>) using a result of a detection by the detecting portion (<NUM>); and
a control portion (<NUM>, <NUM>, <NUM>, <NUM>) configured to generate the control signal based on the position of the unmanned transport vehicle (<NUM>) which the position grasping portion (<NUM>) grasps.