Patent Description:
With the development of the level of economy, express delivery has become a mainstream of transportation. However, with the increasing demands of the people, more contradictions have appeared in the express delivery industry. At present, the main contradiction in the express delivery industry is that the service capacity and development level can not meet the economic and social development and the ever-increasing service demands of the people. Traditionally, the volume calculation of a package usually needs tools such as a tape measure for manual measurement. However, if the measurement process of the package is done manually, the efficiency is relatively low, the measurement tools are outdated, and measurement results are easy to cause disputes between both sides of the trade. With the increasing requirements of customers for logistics efficiency and cost settlement, this approach is increasingly unable to meet the requirements of modern logistics for high efficiency and accuracy.

In related technologies, the automatic calculation of the package volume by a flow volume calculation algorithm can free staff from heavy volumetric measuring work, so that the labor force is saved, the working efficiency of logistics enterprises is improved, thereby playing an important role in promoting the modernization of the logistics enterprises. However, the flow volume calculation algorithm needs to know the "flow" speed of the package in advance, that is, the movement speed of the package on a conveyor belt. If the speeds of all conveyor belts in an express company are consistent, it is only necessary to measure the speed of the conveyor belt and set speed variables in advance, so that the algorithm can run normally in theory; but, if the conveyor belt has various speeds, for example, the speed of the conveyor belt can be set manually or automatically adjusted according to the situations, such as time and the amounts of packages, it is difficult for a program to accurately calculate the package volume. However, the loading speed of loading staff is limited, especially during busy business periods such as "Double Eleven", packages are often piled up and parked on the conveyor belt, and the loading staff often load packages in a "stop-and-go" way of the conveyor belt. At this time, the speed is often in an unstable state in which it will be high or low abruptly, or the problem of speed change caused by voltage instability makes the algorithm unable to calculate the accurate package volume.

<CIT> describes an apparatus for determining the dimensions of a package while being moved by a transport through a scanning zone. Sensors with different respective fields of view are disposed about a scanning zone and generate corresponding frames of <NUM>-D images where some of the points represent the transport and package. A computing apparatus translates the points in the images into a transport coordinate system with a common point of origin on the transport so that the package does not appear to be in motion from frame to frame. The computing apparatus merges the translated points in different frames into a combined point cloud image and deletes points representing the transport resulting in a final point cloud image of the package. The computing apparatus determines the dimensions of the package based on the location of points representing the package in the final point cloud image.

<CIT>describes a method for estimating dimensions of an approximately cuboidal object from a 3D image of the object acquired by an image sensor of the vision system processor. An identification module, associated with the vision system processor, automatically identifies a 3D region in the 3D image that contains the cuboidal object. A selection module, associated with the vision system processor, automatically selects 3D image data from the 3D image that corresponds to approximate faces or boundaries of the cuboidal object. An analysis module statistically analyzes, and generates statistics for, the selected 3D image data that correspond to approximate cuboidal object faces or boundaries. A refinement module chooses statistics that correspond to improved cuboidal dimensions from among cuboidal object length, width and height which are provided as dimensions for the object.

<CIT>describes a device having TOF depth image sensors which are arranged for sensing detectable cargo from two different directions. The sequence of individual images in first resolution is produced from image sensors, such that an industrial truck and detectable cargo are sensed from different viewing angles. A volume-determining apparatus determines sequences of individual images for performing super resolution process to create three-dimensional model in a second resolution which is higher than first resolution to determine volume of detectable cargo.

<CIT>describes a volume measurement method and system based on a depth camera. The method includes the steps of acquiring depth images with objects to be measured from the depth camera; extracting the objects to be measured from the depth images according to depth information to obtain target areas of the objects to be measured; converting two-dimensional image coordinates of pixels in the target areas of the objects to be measured to three-dimensional coordinates under a three-dimensional camera coordinate system by the aid of pre-calibrated parameters of the depth camera; calculating heights and lengths and widths of the objects to be measured according to the three-dimensional coordinates of the objects to be measured under the three-dimensional camera coordinate system, and calculating volumes of the objects to be measured.

In view of the aforementioned problem, an object of the present application is to provide a volumetric measuring method and apparatus based on a time-of-flight depth camera.

In order to achieve the above object, the present application adopts following technical solutions.

A volumetric measuring method based on a time-of-flight depth camera provided by an embodiment of the present application, comprises:.

In some embodiments, the "acquiring depth images returned by the time-of-flight depth camera" comprises:.

In some embodiments, the "denoising the first depth image and the second depth image" comprises:.

In some embodiments, the "traversing each pixel point in the first feature area at multiple traversal speeds" comprises:.

In some embodiments, the "traversing each pixel point in the first feature area at multiple traversal speeds" comprises following specific steps:.

In some embodiments, the "matching a first depth value of a current pixel point with a second depth value of a target pixel point corresponding to the current pixel point in the second feature area to acquire multiple total similarity values" comprises:.

In some embodiments, the weighting process comprises:.

In some embodiments, the first threshold is smaller than the second threshold; the first weight value is smaller than the second weight value, and the second weight value is smaller than the third weight value.

A volumetric measuring apparatus based on a time-of-flight depth camera provided by an embodiment of the present application, comprises:.

In some embodiments, the matching module comprises:.

A computer device provided by an embodiment of the present application, comprises a processor and a memory connected with the processor;.

The present application provides a volumetric measuring method and apparatus based on a TOF depth camera, which is achieved by: acquiring depth images returned by the TOF depth camera; denoising the first depth image and the second depth image; selecting a first feature area of the first depth image and a second feature area of the second depth image that have been denoised respectively, traversing each pixel point in the first feature area at multiple traversal speeds, and matching a first depth value of the current pixel point with a second depth value of a target pixel point corresponding to the current pixel point in the second feature area to acquire multiple total similarity values; determining a traversal speed corresponding to a maximum total similarity value among the multiple total similarity values as an instantaneous speed at a current moment; calculating a package volume according to the instantaneous speed. The present application effectively solves the problem that an accurate package volume cannot be obtained when the speed of the conveyor belt is not fixed due to turning-on, turning-off, unstable voltage, manual intervention and the like, and can make the package volume calculation more accurate, and the method is accurate and effective in measurement, and high in feasibility.

The present application achieves the effects that there is no need to measure or set the speed of the conveyor belt in advance and an instantaneous speed can be accurately acquired when the speed of the conveyor belt is not fixed so as to calculate the package volume on the conveyor belt, and package volume calculation can be more accurate, so that the requirements of modern logistics for high efficiency and accuracy are met, and a great significance is brought to the process optimization and decision-making process of logistics enterprises.

In order to explain the embodiments of the present application, the following will make a brief introduction to the figures that are used in the description of the embodiments. Obviously, the figures in the following description are only some of embodiments of the present application, and for those of ordinary skill in the art, other figures can also be derived from these figures without creative labor.

In order to make objects, technical solutions and advantages of the present application clearer, technical solutions of the present application will be described in detail below. Apparently, the described embodiments are merely some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

Next, a specific volumetric measuring method and apparatus based on a time-of-flight (referred as TOF in the present application for simplify) depth camera provided in an embodiment of the present application will be introduced with the accompanying drawings below.

The TOF depth camera obtains a distance between a target object and the TOF depth camera by continuously sending light pulses to the target object, then receiving light returned from the target object with a sensor, and detecting time of flight of the light pulses. The present application can be realized by employing the currently existing TOF depth camera.

As shown in <FIG>, a volumetric measuring method based on a TOF depth camera provided in an embodiment of the present application comprises:.

It can be understood that a data processing control module in the TOF depth camera controls turning-on and turning-off of the TOF depth camera, and converts the depth flows acquired by the TOF depth camera after shooting the package on the conveyor belt into the depth images. The method of converting the depth flows into the depth images mentioned above is common knowledge to a person skilled in the art, and is omitted here.

S102, denoising the first depth image and the second depth image;
optionally, the "denoising the first depth image and the second depth image" comprises:.

S103, selecting a first feature area of the first depth image and a second feature area of the second depth image that have been denoised respectively, traversing each pixel point in the first feature area at multiple traversal speeds v, and matching a first depth value of a current pixel point with a second depth value of a target pixel point corresponding to the current pixel point in the second feature area to acquire multiple total similarity values;
specifically, selecting an area of interest of the denoised first depth image as the first feature area; selecting an area of interest of the denoised second depth image as the second feature area.

Where, the area of interest is selected according to routine operation of a person skilled in the art; generally speaking, it is preferable to select an area near an object entry position on a target object as the area of interest, so that a situation that the area of interest from falling out of the field of view of the TOF depth camera in the second depth image can be prevented. Selecting the second feature area of the second depth image actually means that the second feature area is obtained by speed traversal on the first feature area of the first depth image.

It can be understood that a preset initial speed v<NUM> and a traversal accuracy r are preset, that is, each traversal increases v by r, and traversal at multiple traversal speeds is performed; for each speed v, after traversing the first feature area of the first depth image, a total similarity value will be obtained. Therefore, for multiple traversal speeds, multiple total similarity values can be obtained.

Where, the preset initial speed v<NUM> and the traversal accuracy r are determined according to a speed of the conveyor belt and a required accuracy, and after traversing multiple traversal accuracies r, a maximum traversal speed vmax is equal to the speed of the conveyor belt. For example, when the speed of the conveyor belt is <NUM>/s, the following may be set: the preset initial speed v<NUM>=<NUM>/s, the traversal accuracy r=<NUM>/s, each pixel point traverses five speeds, and the maximum traversal speed vmax= the speed of the conveyor belt = <NUM>/s.

S104, determining a traversal speed corresponding to a maximum total similarity value among the multiple total similarity values as an instantaneous speed at a current moment;.

After traversing all the traversal speeds, several total similarity values will be obtained, and the traversal speed corresponding to the maximum total similarity value is taken as the instantaneous speed at the current moment, the instantaneous speed actually being an average speed in the time between two adjacent frames currently.

S105, calculating a package volume according to the instantaneous speed.

After the instantaneous speed is determined, the package volume is calculated by a flow volume calculation algorithm. Where, after the instantaneous speed is determined, the instantaneous speed is taken as a fixed speed, and the package volume is calculated by the flow volume calculation algorithm based on the fixed speed, the flow volume calculation algorithm has been disclosed in the related art (<CIT>) and will be omitted herein.

When adopting the volumetric measuring method based on the TOF depth camera provided by the embodiment of the present application, there is no need to set the speed of the conveyor belt in advance, which makes the algorithm more general; and the problem that the speed of the conveyor belt changes due to turning-on, turning-off, unstable voltage, manual intervention, etc. can be solved, so that the flow volume calculation algorithm is no longer limited by the constant speed, and the volume calculation is more accurate. The requirements of modern logistics for high efficiency and accuracy are met, and a great significance is brought to the process optimization and decision-making process of logistics enterprises.

Optionally, the "traversing each pixel point in the first feature area at multiple traversal speeds" comprises:.

Specifically, the "traversing each pixel point in the first feature area at multiple traversal speeds" comprises the following specific steps:.

Through the above steps, the target pixel point pp corresponding to the current pixel point p is found in the second depth image and subsequent depth value matching is carried out.

The working principle of the volumetric measuring method based on the TOF depth camera provided by the present application is as follows: acquiring two adjacent depth images returned, installing the TOF depth camera directly above the conveyor belt, with a shooting direction being perpendicular to the conveyor belt, acquiring depth flows by shooting by the TOF depth camera, and converting the acquired depth flows into the depth images; denoising the depth images, that is, performing morphological processing on the depth images, and denoising the depth images through image erosion and filtering; selecting a first feature area of the denoised first depth image and a second feature area of the denoised second depth image respectively, traversing each pixel point in the first feature area at multiple traversal speeds, and matching a first depth value of a current pixel point with a second depth value of a target pixel point corresponding to the current pixel point in the second feature area to acquire multiple total similarity values; determining a traversal speed corresponding to the maximum total similarity value among the total similarity values as an instantaneous speed at the current moment, so as to determine an optimal instantaneous speed; and finally, calculating a package volume by the flow volume calculation algorithm with the optimal instantaneous speed.

Optionally, the "matching a first depth value of a current pixel point with a second depth value of a target pixel point corresponding to the current pixel point in the second feature area to acquire multiple total similarity values" in S <NUM>, comprises:.

It can be understood that a value of the preset threshold is determined according to an error acceptable by a person skilled in the art, and the value may be different in different situations, which will not be limited in the present application.

According to the method provided by the present application, a matching degree of speed is measured by adding the single pixel similarity value to the total similarity value, the total similarity value indicates a similarity between a simulated speed and a real speed, and the simulated speed corresponding to the maximum total similarity value will be regarded as a current real speed. The first depth value of the current pixel point p and the second depth value of the target pixel point pp are compared, if the absolute value of the difference is greater than or equal to the first preset threshold δ<NUM>, the current pixel point matching fails; when the current pixel point matching fails, if the first height of the first position represented by the current pixel point is greater than the second height of the second position represented by the target pixel point, the single pixel similarity value of the current pixel point is obtained; it should be noted that in this case, the single pixel similarity value is set as a negative value when weighting and added to the total similarity value, that is, the single pixel similarity value of the current pixel point that fails to match will play a role in reducing the total similarity value when added to the total similarity value. When the package moves, it can be known from the calculation formula of Δn that a pixel point corresponding to a high position moves faster in the area of interest than that corresponding to a low position, resulting in that the pixel point corresponding to the high position in the image may cover the pixel point corresponding to the original low position in the next frame image. Therefore, when the current pixel point matching fails, if the first height of the first position represented by the first depth value of the current pixel point p is less than the second height of the second position represented by the second depth value of the target pixel point pp, no reduction treatment in the total similarity value is performed for this situation. If the absolute value of the difference is less than the first preset threshold δ<NUM>, the current pixel point is matched successfully, and after the difference is weighted, the single pixel similarity value is obtained and added to the total similarity value.

Optionally, the first weight value q<NUM> is smaller than the second weight value q<NUM>, and the second weight value q<NUM> is smaller than the third weight value q<NUM>, that is, q<NUM><q<NUM><q<NUM>; it can be understood that when both of the current pixel point and the target pixel point are on the edges of the package, the eight pixel points around the current pixel point and the eight pixel points around the target pixel point are compared correspondingly, and when the absolute value of the difference between any pair of pixel points is less than the second preset threshold, it means that this pair of pixel points is successfully matched; the number of the successfully matched pairs among the eight pairs of pixel points is counted, the more the successfully matched pairs, the greater the weight value set correspondingly.

Specifically, as shown in <FIG>, specific steps of the weighting process are as follows:.

It can be understood that <NUM>≤ β<NUM><β<NUM>, where β<NUM> can be a multiple of β<NUM>, and the number of successfully matched pairs can be judged by setting β<NUM>, β<NUM>, so as to set corresponding weight values.

As shown in <FIG>, an embodiment of the present application provides a volumetric measuring apparatus based on the TOF depth camera, comprises:.

Optionally, as shown in <FIG>, the acquisition module <NUM> comprises:.

Optionally, the processing module <NUM> comprises:.

Optionally, as shown in <FIG>, the matching module <NUM> comprises:.

Optionally, as shown in <FIG>, the matching module <NUM> further comprises:.

As shown in <FIG>, another embodiment of the present application provides a volumetric measuring apparatus based on the TOF depth camera, which comprises a processor <NUM> and a memory <NUM> connected with the processor <NUM>, wherein the processor <NUM> is configured to execute following program modules stored in the memory <NUM>:.

The working principle of the volumetric measuring apparatus based on the TOF depth camera provided by the embodiment of the present application is as follows: the acquisition module <NUM> acquires the depth images returned by the TOF depth camera, wherein the depth images comprises the first depth image and the second depth image which are adjacent; the processing module <NUM> denoises the first depth image and the second depth image; the matching module <NUM> is used for selecting the first feature area of the denoised first depth image and the second feature area of the denoised second depth image respectively, traversing each pixel point in the first feature area at multiple traversal speeds, and matching the first depth value of the current pixel point with the second depth value of the target pixel point corresponding to the current pixel point in the second feature area to acquire multiple total similarity values; the determination module <NUM> determines the traversal speed corresponding to the maximum total similarity value among the multiple the total similarity values as the instantaneous speed at the current moment; the calculation module <NUM> calculates the package volume according to the instantaneous speed.

As shown in <FIG>, an embodiment of the present application provides a computer device <NUM>, which comprises a processor <NUM> and a memory <NUM> connected with the processor <NUM>;
the memory <NUM> is configured to store a computer program which is to execute the following steps:.

An embodiment of the present application provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the following steps:.

To sum up, the present application provides a volumetric measuring method and apparatus based on a TOF depth camera, comprising: acquiring depth images returned by the TOF depth camera; denoising the first depth image and the second depth image; selecting a first feature area of the first depth image and a second feature area of the second depth image that have been denoised respectively, traversing each pixel point in the first feature area at multiple traversal speeds, and matching a first depth value of the current pixel point with a second depth value of a target pixel point corresponding to the current pixel point in the second feature area to acquire multiple total similarity values; determining a traversal speed corresponding to a maximum total similarity value among the multiple total similarity values as an instantaneous speed at a current moment; calculating a package volume according to the instantaneous speed. The present application effectively solves the problem that an accurate package volume cannot be obtained when the speed of the conveyor belt is not fixed due to turning-on, turning-off, unstable voltage, manual intervention and the like, and can make the package volume calculation more accurate, and the method is accurate and effective in measurement, and high in feasibility.

It can be understood that the aforementioned method embodiment corresponds to the above-mentioned apparatus embodiment, and the corresponding specific contents can refer to each other, and are omitted here.

It will be appreciated by those skilled in the art that the embodiment of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk memory, optical memory, etc.) having computer usable program codes embodied therein.

The present application is described with reference to the flowcharts and/or block diagrams of the method, the device (system), and the computer program products according to embodiments of the present application. It should be understood that each flow and/or block in the flowcharts and/or block diagrams, as well as combinations of flows and/or blocks in the flowcharts and/or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a means for implementing the functions specified in one or more flows of the flowcharts and/or one or more blocks of the block diagrams.

These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce an article of manufacture including instruction methods that implement the functions specified in one or more flows of the flowcharts and/or one or more blocks of the block diagrams.

Claim 1:
A volumetric measuring method based on a time-of-flight depth camera, comprising:
acquiring depth images returned by the time-of-flight depth camera; the depth images comprise a first depth image and a second depth image which are adjacent to each other;
denoising the first depth image and the second depth image;
selecting a first feature area of the first depth image and a second feature area of the second depth image that have been denoised respectively, traversing each pixel point in the first feature area at multiple traversal speeds v, and matching a first depth value of a current pixel point with a second depth value of a target pixel point corresponding to the current pixel point in the second feature area to acquire multiple total similarity values;
determining a traversal speed corresponding to a maximum total similarity value among the multiple total similarity values as an instantaneous speed at a current moment;
calculating a package volume according to the instantaneous speed.