Patent Description:
The members <NUM>, <NUM> may typically be made of metal, e.g. extruded aluminium profiles.

Each prior art container handling vehicle <NUM>,<NUM> also comprises a lifting device (not shown) for vertical transportation of storage containers <NUM>, e.g. raising a storage container <NUM> from, and lowering a storage container <NUM> into, a storage column <NUM>. The lifting device comprises one or more gripping / engaging devices which are adapted to engage a storage container <NUM>, and which gripping / engaging devices can be lowered from the vehicle <NUM>,<NUM> so that the position of the gripping / engaging devices with respect to the vehicle <NUM>,<NUM> can be adjusted in a third direction Z which is orthogonal the first direction X and the second direction Y. Parts of the gripping device of the container handling vehicle <NUM> are shown in <FIG> indicated with reference number <NUM>. The gripping device of the container handling device <NUM> is located within the vehicle body 301a in <FIG>.

The storage volume of the framework structure <NUM> has often been referred to as a grid <NUM>, where the possible storage positions within this grid are referred to as storage cells. Each storage column may be identified by a position in an X- and Y-direction, while each storage cell may be identified by a container number in the X-, Y and Z-direction.

Such a vehicle is described in detail in e.g. NO <NUM>.

The central cavity container handling vehicles <NUM> shown in <FIG> may have a footprint that covers an area with dimensions in the X and Y directions which is generally equal to the lateral extent of a storage column <NUM>, e.g. as is described in <CIT>.

The term 'lateral' used herein may mean 'horizontal'.

The access station may typically be a picking or a stocking station where product items are removed from or positioned into the storage containers <NUM>. In a picking or a stocking station, the storage containers <NUM> are normally not removed from the automated storage and retrieval system <NUM> but are returned into the framework structure <NUM> again once accessed. A port can also be used for transferring storage containers to another storage facility (e.g. to another framework structure or to another automated storage and retrieval system), to a transport vehicle (e.g. a train or a lorry), or to a production facility.

Alternatively, or in addition, the automated storage and retrieval system <NUM> may have container handling vehicles <NUM>,<NUM> specifically dedicated to the task of temporarily removing storage containers <NUM> from a storage column <NUM>. Once the target storage container <NUM> has been removed from the storage column <NUM>, the temporarily removed storage containers <NUM> can be repositioned into the original storage column <NUM>. However, the removed storage containers <NUM> may alternatively be relocated to other storage columns <NUM>.

When a storage container <NUM> is to be stored in one of the columns <NUM>, one of the container handling vehicles <NUM>,<NUM> is instructed to pick up the storage container <NUM> from the pick-up port column <NUM> and transport it to a location above the storage column <NUM> where it is to be stored. After any storage containers <NUM> positioned at or above the target position within the stack <NUM> have been removed, the container handling vehicle <NUM>,<NUM> positions the storage container <NUM> at the desired position. The removed storage containers <NUM> may then be lowered back into the storage column <NUM> or relocated to other storage columns <NUM>.

A problem with the storage grid is that over time dust and dirt will collect in the tracks the container handling vehicle is traveling on. This can damage the vehicle and lead to unnecessary cost and risk. Further, the tracks and even the framework of the underlying storage unit can experience wear and tear over time. As it is now there is no way of surveiling possible faults. A robot would have to stop due to an error over the problem area in order for the problem to be detected. This is a costly and time-consuming way of dealing with the problem.

Another problem with the present situation is that it is no easy way of checking the feet of the storage facility. If there is an alignment problem in a column it can lead to the gripper with or without a container will collide with the column on the way up or down and this will over time cause damage to the equipment. This requires shut down of the storage facility which again is costly and time-consuming.

<CIT>, which was identified as prior art according to the Article <NUM>(<NUM>) EPC during the EPO's examination procedure, discloses a system and a method for controlling movement of a plurality of container handling vehicles on a rail system, where each container handling vehicle comprises a local controller adapted to control movements of the container handling vehicle, and wherein a central operational controller in communication with the local controller in each container handling vehicle is adapted to receiving data relating to a subsection of the rail system, the data comprising a container handling vehicle movement threshold for the subsection. Also disclosed is instructing a container handling vehicle to follow a path which takes in at least a part of the subsection, and instructing the container handling vehicle to reduce speed and/or acceleration such that the movement of the container handling vehicle within the subsection is below the container handling vehicle movement threshold of the subsection.

<CIT>, which was identified as the "closest prior art" during the EPO's examination procedure, discloses, according to its abstract, a robotic service device for use on a robotic picking system grid, wherein the robotic service device is capable of driving to any location on the grid in order to perform maintenance operations or cleaning. Additionally, the service device may be used to rescue robotic load handling devices operational in the picking system. The robotic service device may comprise a releasable docking mechanism to enable it to dock and latch on to malfunctioning load handling devices. The service device is provided with cleaning means and may be provided with camera means to enable the condition of the grid and other robotic devices to be monitored.

In one aspect, the invention is related to an automated storage and retrieval system according to claim <NUM>.

Further, the at least one camera can be mounted on the side of the container handling vehicle in the travelling directions of the container handling vehicle such that it is pointing to observe an area of the rail system which is at a level below a height of the at least one camera on the container handling vehicle.

Also, the container handling vehicle can have a camera on all sides of the container handling vehicle such that it is pointing to observe an area of the rail system which is at a level below a height of the at least one camera on the container handling vehicle.

Also, the central computer system, cloud storage and/or image analysis and processing system can be configured to process the information gathered by the at least one sensor.

The at least one camera on each side of the container handling vehicle in the travelling directions of the container handling vehicle can be pointed at an angle of between <NUM>°-<NUM>° pointing downwards from a horizontal level.

The at least one camera of the container handling vehicle is mounted on top of the container handling vehicle can be pointed at an angle of between <NUM>°-<NUM>° pointing downwards from a horizontal.

The at least one sensor is in the form of a pendulum and/or a level, and/or an accelerometer and/or a sound detecting device.

The at least one sensor can be mounted on a lifting platform of the container handling vehicle.

There can be at least one sensor connected to each of the wheels on the container handling vehicle.

The at least one sensor can be an accelerometer detecting if the wheels are spinning.

The container handling vehicle can have a sensor for monitoring the tilt and movement of the vehicle.

The at least one sensor can be a sound detecting device on the lifting platform to detect if the container handling vehicle hits an obstacle in a cell during lifting and lowering of containers into a cell of the storage system.

At least one camera can be mounted on the lifting platform of the container handling vehicle.

In a second aspect, the invention concerns a method for condition-based maintenance of an automated storage and retrieval system according to claim <NUM>.

Wherein taking an image can comprise taking an image straight downwards along the sides of the vehicle when the container handling vehicle is stationary above a vertical storage column.

Taking an image can comprise taking an image of the storage grid on each side of the container handling vehicle when the container handling vehicle is stationary above a vertical storage column.

Taking an image can comprise using a camera mounted at an angle of between <NUM>°-<NUM>° pointing downwards from a horizontal.

Detecting anomalies in the tracks on the grid can comprise using machine learning to analyse the uploaded images.

Taking an image can comprise using at least one additional camera mounted on the lifting platform of the container handling vehicle.

Also detecting anomalies in the tracks on the grid using machine learning to analyse the uploaded images, including using at least one additional camera mounted on the lifting platform of the container handling vehicle.

Using the present invention as stated in the set of claims solves the problems mentioned above.

According to a preferred embodiment of the present invention the monitoring of the condition of the storage and retrieval system can be accomplished by at least one container handling vehicle traveling from column to column and performing measurements.

In an alternative embodiment of the present invention the container handling vehicles operating on the storage and retrieval system can monitor the conditions of the storage and retrieval system while working.

In yet another embodiment of the present invention, at least one container handling vehicle can travel from column to column and perform measurements at fixed intervals to detect loose framework or destroyed grid feet or any other measurements that cannot be done during normal operations. Checking for dust, dirt or debris in the tracks on the storage and retrieval system that can be done by a container handling vehicle during ordinary operations.

<FIG> is a perspective view of a framework structure of a prior art automated storage and retrieval system.

<FIG> are perspective drawings of a prior art container handling vehicles having a centrally arranged cavity for carrying storage containers therein a cantilever solution for carrying storage containers underneath a container handling vehicle wherein the containers are stored on top of the vehicle.

<FIG> is a side view of an embodiment of the present invention wherein a container handling vehicle with a central cavity solution has cameras mounted to the sides of the vehicle, and the cameras are pointing downwards.

There is a camera mounted on each side of the container handling vehicle. Each camera is pointing downwards in order to take an image of the tracks framing the column(s) the container handling vehicle is positioned over. This solution gives the closest images of the tracks, so it is possible to establish if there is something in the tracks and optionally what the thing is. If you know what an object on the tracks actually is it is much easier to establish if this is something that requires immediate attention, or if it can be handled later at a slower time of day, when the repairs can be done without the entire storage and retrieval system having to be shut down.

<FIG> is a side view of an embodiment of the present invention wherein a container handling vehicle with a central cavity solution has cameras mounted to the sides of the vehicle pointing outwards.

In this solution there is a camera at each side of the container handling vehicle. The camera is pointing at an angle downwards. The angle the camera is pointing could be from <NUM>°-<NUM>° down from a horizontal line, meaning straight down to almost horizontal.

In a further solution the camera could be movable so that you could have one camera on each side that covers <NUM>° form straight down to straight up. This solution allows one container handling vehicle to get several images from each side of the container handling vehicle. There are several benefits with this solution. It allows the central computer system or the cloud service or the image analysis and processing system to get more images of each column. This allows the analysis tool to establish what movement of the framework or tracks or grid are due to the weight of the container handling vehicle. So, it is easier to establish how bad a situation actually is, if it is possible to get several images from different angles, from different sides and with or without weight on the tracks right above the column.

<FIG> is a perspective view of a container handling vehicle with a central cavity solution, where the wheel sets of the container handling vehicle has a pendulum function. The pendulum function allows two sets of wheels to move at an angle. This adds a measuring instrument on the robot's pendulum, and if the robot pendulum reaches a maximum angle, there is a danger that the unevenness of the cell is higher than the robot can take up, with a potential to lead to collisions.

Another way of detecting unevenness of the tracks would be to add at least one electronic level. This will indicate how many degrees the tracks are out of line and if it is something that e.g. the pendulum of the container handling vehicle can take up or if it is something that requires immediate attention and locking down of parts or all of the storage and retrieval system.

<FIG> is a side view of an embodiment of the present invention comprising a container handling vehicle with a cantilever solution with cameras pointing downwards.

There is a camera mounted on each side of the container handling vehicle. Each camera is pointing downwards in order to take an image of the tracks framing the column(s) the container handling vehicle is positioned over. This solution gives the closest images of the tracks, so it is possible to establish if there is something in the tracks and alternatively what the thing is. If you know what an object on the tracks actually is it is much easier to establish if this is something that requires immediate attention, or if it can be handled later at a slower time of day, when the repairs can be done without the entire storage and retrieval system having to be shut down.

<FIG> is a side view of an embodiment of the present invention comprising a container handling vehicle with a cantilever solution with cameras pointing outwards.

In this solution there is a camera at each side of the container handling vehicle. The camera is pointing at an angle downwards. The angle the camera is pointing in could be from <NUM>-<NUM>° down from a horizontal line.

<FIG> is a side view of an embodiment of the present invention wherein a container handling vehicle with a cantilever solution wherein the lifting platform has cameras pointing in outwards mounted to it.

There is a camera pointing outwards to all sides that allows for the present invention to take images when the lifting platform of the container handling vehicles are lowered into the columns. With this solution it is further possible to take images of the grid feet by lowering the lifting platform all the way down in the column. Further there can be lights on the lifting frame since it can be dark down the columns.

It is also possible to use other types of equipment than cameras to check the condition down the columns. One such example could be a Lidar. This would be a good way of checking the height position of the tracks, and the levels of framework downwards.

<FIG> is a side view of an embodiment of the present invention where a container handling vehicle with a cantilever solution has a camera mounted to the lifting platform pointing downwards.

There is one camera that can be moved around covering the entire area under the lifting platform. This allows for the present invention to take images when the lifting platform of the container handling vehicles are lowered into the columns. With this solution it is further possible to take images of the grid feet by lowering the lifting platform all the way down in the column. Further there can be lights on the lifting frame since it can be dark down the columns.

Although the drawings in <FIG> show a container handling vehicle with a cantilever solution, this technical feature with attaching one or more cameras to the lifting platform of the container handling vehicle, could just as well be applied to the central cavity solution.

<FIG> is a side view of an embodiment of the present invention where a container handling vehicle with a top carrying solution has cameras mounted to the sides pointing downwards.

<FIG> is a side view of an embodiment of the present invention where a container handling vehicle with a top carrying solution has cameras mounted to the sides pointing outwards.

In this solution there is a camera at each side of the container handling vehicle. The camera is pointing at an angle downwards. The angle the camera is pointing in could be from <NUM>-<NUM>° down from a horizontal line.

In a further solution the camera could be movable so that you could have one camera on each side that covers <NUM>° from straight down to straight up. This solution allows one container handling vehicle to get several images from each side of the container handling vehicle. There are several benefits with this solution. It allows the central computer system or the cloud service or the image analysis and processing system to get more images of each column. This allows the analysis tool to establish what movement of the framework or tracks or grid are due to the weight of the container handling vehicle. So, it is easier to establish how bad a situation actually is, if it is possible to get several images from different angles, from different sides and with or without weight on the tracks right above the column.

<FIG> is a side view of an embodiment of the present invention where a container handling vehicle with a central cavity solution having a top mounted camera. This camera can take images in all directions by moving the camera around. So, it is a balance of whether it is worth the extra investment when it comes to the number of cameras or if it is more expensive to shut down the storage and retrieval system for a longer period. <FIG> is a side view of an embodiment of the present invention where a container handling vehicle with a cantilever solution having a top mounted camera. This is the same technical features as presented in <FIG>, only with the top mounted camera mounted on a container handling vehicle with a cantilever solution instead of a central cavity solution.

An additional sensor to use in order to detect problems with unevenness in the framework of the storage and retrieval system is a motion sensor. The motion sensor can detect motion in every direction. The motion is an indication that the tracks or the framework of the storage and retrieval system is wrong and the gathered information can be sent to a central computer system or the cloud service or the image analysis and processing system to analyse where in the storage and retrieval system there is a problem. The motion sensor can give information that the images cannot, like if there are a part of the grid that gives way when there are weight on it. If the framework gives way when the container handling vehicle drives over it the motion sensor can give information like how much does it give way and to which direction. With this information stored it is possible to find out where in the framework there might be a problem.

During analysis there is a benefit when you have measurements of neighbouring columns. It allows the analysis program to get information about the extent and location of the problem. Therefore, in a preferred solution of the present invention if there is identified that there is a problem with one of the columns, images and measurements of neighbouring columns can be used together with information from the column in question in order to get an estimate of the extent of the problem. If the problem is a recent development or if it has gradually become worse over time. Adding a sound detecting device in the container handling vehicles gripper makes it possible to detect if the container handling vehicle hits an obstacle in a cell during lifting and lowering of containers into a cell of the storage system. A sound recorder can separate undesired noise from the background noise of the operation of a container handling vehicle.

Cameras, lidar, sound and all the other types of sensors can be fitted in a container that can be either grabbed by the lifting platform of the container handling vehicle or placed on top of the container handling vehicle according to <FIG>.

Also, the container handling vehicle can be fitted with an accelerometer in order to detect if the wheels slip on the tracks.

In the preceding description, various aspects of the delivery vehicle and the automated storage and retrieval system according to the invention have been described with reference to the illustrative embodiment. For purposes of explanation, specific numbers, systems and configurations were set forth in order to provide a thorough understanding of the system and its workings. However, this description is not intended to be construed in a limiting sense. Various modifications and variations of the illustrative embodiment, as well as other embodiments of the system, which are apparent to persons skilled in the art to which the disclosed subject matter pertains, are deemed to lie within the scope of the present invention, which is defined by the appended claims.

Claim 1:
An automated storage and retrieval system with a system (<NUM>) for condition-based monitoring of it comprising a framework structure (<NUM>) forming a three-dimensional storage grid structure (<NUM>) for storing storage containers (<NUM>) for storing items, where the grid structure (<NUM>) forms vertical storage columns (<NUM>) each having a horizontal area defined by the size of an access opening (<NUM>) of the vertical storage columns (<NUM>) and where the framework structure comprises a rail system (<NUM>) arranged above the storage columns (<NUM>), the rail system comprising a plurality of rails (<NUM>, <NUM>) extending in an X-direction and a Y-direction to form a grid, the rails (<NUM>, <NUM>) defining a perimeter of each access opening (<NUM>) on top of each storage column (<NUM>), the rail system (<NUM>) providing available routes in the X-direction or the Y-direction for container handling vehicles (<NUM>, <NUM>) handling and transferring the storage containers (<NUM>) to and from the storage columns (<NUM>), characterized in that at least one container handling vehicle (<NUM>, <NUM>) has at least one camera (<NUM>) and at least one sensor, directed to monitor a section of the rails (<NUM>, <NUM>) adjacent of the container handling vehicle (<NUM>, <NUM>), and that the at least one camera (<NUM>) and the at least one sensor are part of the monitoring system (<NUM>) set up to report a condition of the rails (<NUM>, <NUM>) of the grid system and upload the information to one or all of a central computer system, a cloud system and/or an image analysis and processing system, and wherein the at least one sensor comprises a pendulum (<NUM>) and/or a level and/or an accelerometer and/or a sound detecting device (<NUM>).