Patent Description:
Patent Literature (PTL) <NUM> discloses a three-dimensional shape measuring device that obtains a three-dimensional shape using a three-dimensional laser scanner.

Patent Application <CIT> relates to a measurement system for managing space in a physical distribution warehouse by extracting a shelf area and a baggage placement area from shape measurement data. In particular, a laser distance sensor is used to obtain shape data of the warehouse. Moreover CAD data of the shelves are matched with the shelves within the warehouse shape data to obtain their position and orientation. Then, baggage placement area shape data is obtained alongside with location data of shelf positions and the package information of packages in the shelves. An identification of the packages together with the detected location are provided to a warehouse management system. By dividing a calculated baggage amount by the baggage placement area volume (obtained from package installation area data and package installation area shape data), a bag occupancy rate at the corresponding stage of the corresponding shelf is calculated.

There are no sufficient discussions about examples of application of measured three-dimensional shapes. For example, there are no sufficient discussions about calculation of a filling rate that indicates how many measurement targets are stored in a prescribed storage space.

The present disclosure provides a filling rate measurement method capable of calculating a filling rate of a measurement target, and the like.

It should be noted that the present disclosure may be implemented to a program that causes a computer to execute the steps included in the above-described filing rate measurement method. Furthermore, the present disclosure may be implemented to a non-transitory computer-readable recording medium, such as a Compact Disc-Read Only Memory (CD-ROM), on which the program is recorded. The present disclosure may be implemented to information, data, or signals indicating the program. The program, the information, the data, and the signals may be distributed via a communication network, such as the Internet.

According to the present disclosure, a filling rate measurement method capable of calculating a filling rate of a measurement target, and the like can be provided.

There is a demand for measuring a filling rate of a measurement target such as baggage with respect to a storage space to improve an efficiency of use of the storage space in a distribution site. Further, since measurement targets are to be stored in many storages such as containers in a distribution site, there is a demand for measuring as many filling rates in a short time as possible. However, there are no sufficient discussions about a method for measuring a filling rate easily.

Therefore, the present disclosure provides a filling rate measurement method for easily calculating as many filling rates of storages in a short time as possible by applying a technique of generating a three-dimensional model to a storage in which a measurement target is stored.

In accordance with an aspect of the present disclosure, a filling rate measurement method includes: obtaining a space three-dimensional model generated by measuring a first storage having an opening and a first storage space in which a measurement target is to be stored, the measuring being performed through the opening using a range sensor facing the first storage; obtaining a storage three-dimensional model that is a three-dimensional model of the first storage in which the measurement target is not stored; extracting a target portion corresponding to the measurement target from the space three-dimensional model using the space three-dimensional model obtained and the storage three-dimensional model obtained; estimating a target three-dimensional model using the target portion extracted, the target three-dimensional model being a three-dimensional model of the measurement target in the first storage space; and calculating a first filling rate of the measurement target with respect to the first storage space, using the storage three-dimensional model and the target three-dimensional model.

According to this aspect, a target three-dimensional model of a measurement target is estimated using a target portion that is extracted using a space three-dimensional model resulting from measuring a first storage in a state where the measurement target is stored and a storage three-dimensional model of the first storage in which the measurement target is not stored. Therefore, the first filling rate of the measurement target with respect to the first storage space can be calculated easily only by measuring the first storage in a state where the measurement target is stored.

Furthermore, it is possible that the estimating of the target three-dimensional model is performed, based on a first three-dimensional coordinate system based on a shape of a part of the first storage.

Therefore, a processing amount of estimation of the target three-dimensional model can be reduced.

Furthermore, according to the invention, the filling rate measurement method further includes: calculating the first three-dimensional coordinate system based on only the shape of the part of the first storage.

This enables the shape of only the part of the first storage, which is easy to extract on an image, to be used for calculation of the first three-dimensional coordinate system. Therefore, a processing speed of the estimation of the target three-dimensional model can be improved, and a precision of calculating the first three-dimensional coordinate system can be improved.

Furthermore, it is possible that the shape of the part of the first storage is a shape of the opening.

Therefore, the coordinate system based on the shape of the opening can be calculated easily, and the target three-dimensional model can be estimated based on the calculated coordinate system.

Furthermore, according to an example, which is not covered by the appended set of claims, it is possible that the estimating of the target three-dimensional model is performed, based on a first three-dimensional coordinate system based on a position of a marker provided to the first storage.

Therefore, the coordinate system based on the marker can be calculated easily, and the target three-dimensional model can be estimated based on the calculated coordinate system.

Furthermore, it is possible that the estimating of the target three-dimensional model is performed by estimating a shape of a second portion of the measurement target which does not face the range sensor in a direction from the range sensor toward the measurement target, based on a shape of a first portion of the measurement target which faces the range sensor in the direction.

Therefore, even in a case where the second portion through which the range sensor does not face the measurement target is present, the target three-dimensional model can be estimated.

Furthermore, it is possible that the first storage further includes a cover part including a through hole, the cover part being opened and closed, and covering the opening when the cover part is in a closed state, the first portion faces, in the direction, the through hole of the cover part in the closed state, the second portion is hidden in the direction by the cover part in the closed state, the filling rate measurement method further includes determining whether the cover part is in an open state or the closed state, when the cover part is in the open state, the extracting and the estimating of the target three-dimensional model are performed to estimate the target three-dimensional model, and when the cover part is in the closed state, the second portion is estimated based on the first portion, and the target three-dimensional model is estimated using the first portion, the second portion estimated, and the storage three-dimensional model.

According to this, even in a case where the measurement target is stored in the first storage provided with the cover part that opens and closes the opening, the method for estimating the target three-dimensional model is switched according to the open/closed state of the cover part, and thus the target three-dimensional model can be estimated appropriately.

Furthermore, it is possible that the direction is horizontal.

This eliminates a need to adjust a position of the range sensor so that measurement can be performed in a direction in which the cover part having through holes is not present, and thus a flexibility of placing the range sensor is high. Therefore, a result of measurement for estimating the target three-dimensional model by the range sensor can be obtained even when the position of the range sensor is not adjusted completely.

Furthermore, it is possible that the calculating of the first filling rate is performed by calculating, as the first filling rate, a proportion of a volume of the measurement target stored in the first storage space to a capacity of an available space for storing the measurement target in the first storage space.

Therefore, the first filling rate for appropriately determining how many measurement targets can be stored in a vacant space of the first storage space can be calculated.

Furthermore, it is possible that the first storage and an additional first storage are stored in a second storage space included in a second storage, and the filling rate measurement method further includes calculating a second filling rate of the first storage and the additional first storage with respect to the second storage space.

This enables the second filling rate in a case where one or more first storages are stored in the second storage space to be calculated appropriately.

Furthermore, it is possible that the storage three-dimensional model is a three-dimensional model measured by the range sensor and an additional range sensor.

Therefore, a storage three-dimensional model with little occlusion can be generated.

Furthermore, it is possible that the range sensor includes at least two cameras for generating the space three-dimensional model and is fixed to a position above the first storage.

In a case where the range sensor is fixed above the first storage in this manner, objects imaged by the two cameras of the range sensor are limited to the ground or a mount (bottom face) of the first storage, and there is no movable object other than the first storage, which makes it easy to separate the measurement target from a background.

In accordance with another aspect of the present disclosure, an information processing device includes: a processor; and a memory, wherein, using the memory, the processor: obtains a space three-dimensional model generated by measuring a first storage having an opening and a first storage space in which a measurement target is to be stored, the measuring being performed through the opening using a range sensor facing the first storage; obtains a storage three-dimensional model that is a three-dimensional model of the first storage in which the measurement target is not stored; extracts a target portion corresponding to the measurement target from the space three-dimensional model using the space three-dimensional model obtained and the storage three-dimensional model obtained; estimates a target three-dimensional model using the target portion extracted, the target three-dimensional model being a three-dimensional model of the measurement target in the first storage space; and calculates a first filling rate of the measurement target with respect to the first storage space, using the storage three-dimensional model and the target three-dimensional model.

Hereinafter, exemplary embodiments of the filing rate measurement method and the like according to the present disclosure will be described in detail with reference to the accompanying Drawings. The following embodiments are examples of the present disclosure. The numerical values, shapes, materials, elements, arrangement and connection configuration of the elements, steps, the order of the steps, etc., described in the following embodiments are merely examples, and are not intended to limit the present disclosure.

It should be noted that the respective figures are schematic diagrams and are not necessarily precise illustrations. Additionally, components that are essentially the same share like reference signs in the figures. Accordingly, overlapping explanations thereof are omitted or simplified.

With reference to <FIG>, an outline of a filling rate measurement method according to an embodiment will be described.

<FIG> is a diagram for describing the outline of the filling rate measurement method according to the embodiment.

In the filling rate measurement method, as illustrated in <FIG>, baggage <NUM> stored in rack <NUM> that includes storage space <NUM> is measured with range sensor <NUM>. Then, using results of measurement obtained, a filling rate of baggage <NUM> with respect to storage space <NUM> is calculated. Rack <NUM> is formed with opening 102a through which baggage <NUM> is put into or taken out from storage space <NUM>. Range sensor <NUM> is disposed at a location facing opening 102a of rack <NUM> in an orientation that allows range sensor <NUM> to measure rack <NUM> having opening 102a and measures measurement region R1, which contains an inside of storage space <NUM>, through opening 102a.

Rack <NUM> has, for example, a box shape as illustrated in <FIG>. The rack need not have a box shape as long as the rack has a configuration in which the rack includes a placement surface on which baggage <NUM> is placed and includes, over the placement surface, storage space <NUM> where baggage <NUM> is stored. Rack <NUM> is an example of a first storage. Storage space <NUM> is an example of a first storage space. Although storage space <NUM> is configured to be an internal space included in rack <NUM>, storage space <NUM> is not limited to the internal space and may be a space in a storehouse where measurement targets such as baggage <NUM> are stored. Baggage <NUM> is an example of the measurement targets. The measurement targets are not limited to baggage <NUM> and may be goods. That is, the measurement targets may be any bodies as long as they are transportable.

<FIG> is a block diagram illustrating a characteristic configuration of a three-dimensional measurement system according to the embodiment. <FIG> is a diagram for describing a first example of a configuration of the range sensor. <FIG> is a diagram for describing a second example of the configuration of the range sensor. <FIG> is a diagram for describing a third example of the configuration of the range sensor.

As illustrated in <FIG>, three-dimensional measurement system <NUM> includes range sensor <NUM> and information processing device <NUM>. Three-dimensional measurement system <NUM> may include range sensors <NUM> or may include one range sensor <NUM>.

Range sensor <NUM> measures a three-dimensional space including storage space <NUM> of rack <NUM> via opening 102a of rack <NUM>, thus obtaining results of measurement including rack <NUM> and storage space <NUM> of rack <NUM>. Specifically, range sensor <NUM> generates a space three-dimensional model represented as a group of three-dimensional points that indicate three-dimensional positions of measurement points on rack <NUM> or baggage <NUM> (hereinafter, referred to as measurement target) (on a surface of the measurement target). The group of the three-dimensional points is called three-dimensional point cloud. Three-dimensional positions indicated by three-dimensional points in a three-dimensional point cloud are each represented as, for example, a set of coordinates of three-value information consisting of an X component, a Y component, and a Z component in a three-dimensional coordinate space formed by XYZ axes. It should be noted that the three-dimensional model may include not only sets of three-dimensional coordinates but also color information items each indicating a color of a point or shape information items each representing a point and a surface shape around the point. The color information items may be each represented in, for example, an RGB color space or another color space such as HSV, HLS, and YUV.

A concrete example of range sensor <NUM> will be described with reference to <FIG>.

As illustrated in <FIG>, range sensor <NUM> in the first example emits electromagnetic waves and obtains reflected waves that are the electromagnetic waves reflected at a measurement target, thus generating a space three-dimensional model. Specifically, range sensor <NUM> measures a time taken by an emitted electromagnetic wave to be reflected at the measurement target and return to range sensor <NUM> from the emission and calculates a distance between range sensor <NUM> and point P1 on a surface of the measurement target using the measured time and a wavelength of the electromagnetic wave used for the measurement. Range sensor <NUM> emits electromagnetic waves from a reference point of range sensor <NUM> in predetermined radial directions. For example, range sensor <NUM> may emit electromagnetic waves in horizontal directions at first angular intervals and emit electromagnetic waves in vertical directions at second angular intervals. Therefore, by detecting a distance between range sensor <NUM> and the measurement target in each of directions from range sensor <NUM>, range sensor <NUM> can calculate sets of three-dimensional coordinates of points on the measurement target. Range sensor <NUM> thus can calculate position information items indicating three-dimensional positions on the measurement target and can generate a space three-dimensional model including the position information items. The position information items may be a three-dimensional point cloud including three-dimensional points that indicate the three-dimensional positions.

As illustrated in <FIG>, range sensor <NUM> in the first example is a three-dimensional laser measuring instrument including laser emitter <NUM> that emits laser light beams as the electromagnetic waves and laser receiver <NUM> that receives reflected light beams that are the emitted laser light beams reflected at a measurement target. Range sensor <NUM> scans the measurement target with laser light by rotating or swinging a unit including laser emitter <NUM> and laser receiver <NUM> about two different axes or by means of a movable mirror that swings about two axes (micro electro mechanical systems (MEMS) mirror) placed in a route of a laser beam emitted or to be received. This enables range sensor <NUM> to generate a high-precision, high-density three-dimensional model of the measurement target.

Although a three-dimensional laser measuring instrument that measures a distance from a measurement target by emitting laser light beams is exemplified as range sensor <NUM>, range sensor <NUM> is not limited to this; range sensor <NUM> may be a millimeter-wave radar measuring instrument, which measured a distance from a measurement target by emitting millimeter waves.

Range sensor <NUM> may generate a three-dimensional model including color information. First color information items are color information items that are generated from images captured by range sensor <NUM> and indicate colors of first three-dimensional points included in a first three-dimensional point cloud.

Specifically, range sensor <NUM> may include a camera built therein that images a measurement target present around range sensor <NUM>. The camera built in range sensor <NUM> images a region including an emission range of laser light beams emitted by range sensor <NUM>, thus generating images. An imaging range imaged by the camera is associated in advance with the emission range. Specifically, directions in which laser light beams are emitted by range sensor <NUM> are associated in advance with pixels in an image captured by the camera, and range sensor <NUM> sets, as color information items indicating colors of three-dimensional points included in a three-dimensional point cloud, pixel values in the image associated with directions of the three-dimensional points.

As illustrated in <FIG>, range sensor 210A in the second example is a range sensor based on a structured light method. Range sensor 210A includes infrared pattern emitter 211A and infrared camera 212A. Infrared pattern emitter 211A projects infrared pattern 213A, which is predetermined, onto a surface of a measurement target. Infrared camera 212A images the measurement target onto which infrared pattern 213A is projected, thereby obtaining an infrared image. Range sensor 210A searches infrared pattern 213A included in the obtained infrared image and calculates a distance from infrared pattern emitter 211A or infrared camera 212A to point P1 in the infrared pattern on the measurement target in real space based on a triangle formed by connecting three positions including a position of point P1 on the measurement target, a position of infrared pattern emitter 211A, and a position of infrared camera 212A. This enables range sensor 210A to obtain a three-dimensional position of a measurement point on the measurement target.

Range sensor 210A can obtain a high-density three-dimensional model by moving a unit of range sensor 210A including infrared pattern emitter 211A and infrared camera 212A or by making the infrared pattern emitted by infrared pattern emitter 211A have a fine texture.

Further, using a visible light range of color information that can be obtained by infrared camera 212A, range sensor 210A may generate a three-dimensional model including color information items by associating the obtained visible light range with three-dimensional points with consideration given to a position or an orientation of infrared pattern emitter 211A or infrared camera 212A. Alternatively, range sensor 210A may have a configuration further including a visible light camera for adding color information.

As illustrated in <FIG>, range sensor 210B in the third example is a range sensor that measures three-dimensional points by stereo camera measurement. Range sensor 210B is a stereo camera that includes two cameras 211B and 212B. By imaging a measurement target with two cameras 211B and 212B at a synchronized timing, range sensor 210B obtains stereo images with parallax. Using the obtained stereo images (two images), range sensor 210B performs a matching process for a feature point on the two images, thus obtaining alignment information of the two images with pixel precision or sub-pixel precision. Based on a triangle formed by connecting a matched position of point P1 on a measurement target in real space and positions of two cameras 211B and 212B, range sensor 210B calculates a distance from any one of two cameras 211B and 212B to the matched position on the measurement target (i.e., point P1). This enables range sensor 210B to obtain a three-dimensional position of a measurement point on the measurement target.

Range sensor 210B can obtain a high-precision three-dimensional model by moving a unit of range sensor 210B including two cameras 211B and 212B or by increasing the number of cameras provided in range sensor 210B to three or more, imaging the same measurement target and performing the matching process.

Alternatively, using visible light cameras as cameras 211B and 212B included in range sensor 210B can make it easy to add color information to the obtained three-dimensional model.

It should be noted that the present embodiment will be described with an example in which information processing device <NUM> includes range sensor <NUM> in the first example, but information processing device <NUM> may have a configuration including range sensor 210A in the second example or range sensor 210B in the third example in place of range sensor <NUM> in the first example.

Two cameras 211B and 212B are capable of capturing monochrome images including visible light images or infrared images. In this case, the matching process on the two images by three-dimensional measurement system <NUM> may be performed using, for example, Simultaneous Localization And Mapping (SLAM) or Structure from Motion (SfM). Further, using information indicating positions and orientations of cameras 211B and 212B obtained by performing this process, a point cloud density of a measurement space model may be increased by Multi View Stereo (MVS).

Referring back to <FIG>, a configuration of information processing device <NUM> will be described.

Information processing device <NUM> includes obtainer <NUM>, coordinate system calculator <NUM>, model generator <NUM>, filling rate calculator <NUM>, and storage <NUM>.

Obtainer <NUM> obtains a space three-dimensional model and an image generated by range sensor <NUM>. Specifically, obtainer <NUM> may obtain a space three-dimensional model and an image from range sensor <NUM>. The space three-dimensional model and the image obtained by obtainer <NUM> may be stored in storage <NUM>.

Coordinate system calculator <NUM> calculates a positional relation between range sensor <NUM> and rack <NUM> using the space three-dimensional model and the image. Coordinate system calculator <NUM> thereby calculates a measurement coordinate system based only on the shape of the part of rack <NUM>. Specifically, as the shape of the part based on which the measurement coordinate system is calculated, coordinate system calculator <NUM> calculates the measurement coordinate system based on a shape of opening 102a of rack <NUM>. In a case where the shape of opening 102a is rectangular as illustrated in the embodiment, the shape of opening 102a based on which the measurement coordinate system is calculated may be a corner of the shape of opening 102a or may be a side of the shape of opening 102a.

It should be noted that the measurement coordinate system is a three-dimensional orthogonal coordinate system and is an example of a first three-dimensional coordinate system. By calculating the measurement coordinate system, a relative position and a relative orientation of range sensor <NUM> based on rack <NUM> can be determined. That is, this enables a sensor coordinate system of range sensor <NUM> to be aligned with the measurement coordinate system, thus enabling calibration between rack <NUM> and range sensor <NUM>. It should be noted that the sensor coordinate system is a three-dimensional orthogonal coordinate system.

It should be noted that, in the present embodiment, rack <NUM> having a rectangular-parallelepiped shape includes opening 102a at one face of rack <NUM>, but rack <NUM> is not limited to this. The rack may have a configuration in which openings are provided at faces of the rectangular-parallelepiped shape such as a configuration with openings at two faces including a front face and a rear face, and a configuration with openings at two faces including a front face and a top face. In a case where the rack includes openings, prescribed reference positions described later may be set to one of the openings. The prescribed reference positions may be set in a space where neither three-dimensional point nor voxel of a storage three-dimensional model being the three-dimensional model of rack <NUM> is present.

Here, coordinate system calculator <NUM> in the first example will be described with reference to <FIG> and <FIG>.

<FIG> is a block diagram illustrating a configuration of the coordinate system calculator in the first example. <FIG> is a diagram for describing a method for calculating a measurement coordinate system by the coordinate system calculator in the first example.

Coordinate system calculator <NUM> calculates the measurement coordinate system. The measurement coordinate system is a three-dimensional coordinate system that serves as a reference for a space three-dimensional model. For example, range sensor <NUM> is placed at an origin of the measurement coordinate system and placed in an orientation in which range sensor <NUM> directly faces opening 102a of rack <NUM>. At this time, the measurement coordinate system may be such that an upward direction of range sensor <NUM> is set as an X axis, a rightward direction is set as a Y axis, and a frontward direction is set as a Z axis. Coordinate system calculator <NUM> includes assister <NUM> and calculator <NUM>.

As illustrated in (a) of <FIG>, assister <NUM> successively obtains images <NUM>, which are results of measurement by range sensor <NUM> obtained by obtainer <NUM>, in real time, and superimposes adjustment markers <NUM> on each of images <NUM> successively obtained. Assister <NUM> successively outputs superimposed images <NUM> in each of which adjustment marker <NUM> is superimposed on image <NUM>, to a display device not illustrated. The display device successively displays superimposed images <NUM> output from information processing device <NUM>. It should be noted that assister <NUM> and the display device may be integrated together in range sensor <NUM>.

Adjustment markers <NUM> are markers for assisting a user in moving range sensor <NUM> such that a position and an orientation of range sensor <NUM> with respect to rack <NUM> become a specific position and a specific orientation. The user can dispose range sensor <NUM> such that range sensor <NUM> takes the specific position and the specific orientation with respect to rack <NUM> by changing the position and the orientation of range sensor <NUM> while watching superimposed images <NUM> displayed on the display device such that adjustment markers <NUM> match the prescribed reference positions on rack <NUM>. The prescribed reference positions on rack <NUM> are, for example, positions of four corners of quadrilateral opening 102a of rack <NUM>.

When range sensor <NUM> is disposed at the specific position and in the specific orientation with respect to rack <NUM>, superimposed images <NUM> in which four adjustment markers <NUM> are superimposed at four positions corresponding to the positions of the four corners of opening 102a of rack <NUM> are generated. For example, by moving range sensor <NUM> such that adjustment markers <NUM> move in directions of arrows illustrated in (a) of <FIG>, the user can align four adjustment markers <NUM> with the positions of the four corners of opening 102a as illustrated in (b) of <FIG>.

Although assister <NUM> is configured to superimpose adjustment markers <NUM> on image <NUM>, adjustment markers may be superimposed on a space three-dimensional model, and the space three-dimensional model on which the adjustment markers are superimposed may be displayed on the display device.

As illustrated in (c) of <FIG>, calculator <NUM> calculates rotation matrix <NUM> and translation vector <NUM> that indicate a positional relation between rack <NUM> and range sensor <NUM> at a time when four adjustment markers <NUM> are aligned with the positions of the four corners of opening 102a. Calculator <NUM> converts sensor coordinate system <NUM> of range sensor <NUM> using rotation matrix <NUM> and translation vector <NUM> calculated, thus calculating measurement coordinate system <NUM>, of which an origin is a given corner (one of the four corners) of opening 102a. When four adjustment markers <NUM> are aligned with the positions of the four corners of opening 102a, the user may make an input into an input device not illustrated. By obtaining a time when the input from the input device, information processing device <NUM> may determine a time when four adjustment markers <NUM> are aligned with the positions of the four corners of opening 102a. Further, by analyzing image <NUM>, information processing device <NUM> may determine whether four adjustment markers <NUM> have been aligned with the positions of the four corners of opening 102a.

Next, coordinate system calculator 222A in the second example will be described with reference to <FIG> and <FIG>.

<FIG> is a block diagram illustrating a configuration of the coordinate system calculator in the second example. <FIG> is a diagram for describing a method for calculating a measurement coordinate system by the coordinate system calculator in the second example.

Coordinate system calculator 222A includes detector <NUM>, extractor <NUM>, and calculator <NUM>.

Using space three-dimensional model <NUM>, which is a result of measurement illustrated in (a) of <FIG> from range sensor <NUM> obtained by obtainer <NUM>, and storage three-dimensional model <NUM> illustrated in (b) of <FIG>, detector <NUM> detects rack region <NUM> corresponding to rack <NUM> as illustrated in (c) of <FIG>. Storage three-dimensional model <NUM> is a three-dimensional model of rack <NUM> where no baggage <NUM> is stored, and storage three-dimensional model <NUM> is a three-dimensional model that is generated in advance using results of measurement, by range sensor <NUM>, on rack <NUM> at the time when no baggage <NUM> is stored. Storage three-dimensional model <NUM> is generated by model generator <NUM> described later and is stored in storage <NUM>. Storage three-dimensional model <NUM> may include position information <NUM> that indicates positions of four corners of opening 102a of rack <NUM>.

As illustrated in (d) of <FIG>, using position information <NUM> in storage three-dimensional model <NUM>, extractor <NUM> extracts four opening endpoints <NUM>, which are positions of four corners of opening <NUM> in rack region <NUM>. A shape of opening <NUM> defined by four opening endpoints <NUM> is an example of a shape of a part based on which a measurement coordinate system is calculated.

As illustrated in (e) of <FIG>, calculator <NUM> calculates rotation matrix <NUM> and translation vector <NUM> that indicate a positional relation between range sensor <NUM> and rack <NUM> based on the shape of four opening endpoints <NUM> as viewed from range sensor <NUM>. Calculator <NUM> converts sensor coordinate system <NUM> of range sensor <NUM> using rotation matrix <NUM> and translation vector <NUM>, thus calculating measurement coordinate system <NUM>. Specifically, when rotation matrix <NUM> is denoted by R, and translation vector <NUM> is denoted by T, calculator <NUM> can convert three-dimensional point x in sensor coordinate system <NUM> into three-dimensional point X in measurement coordinate system <NUM> by Equation <NUM> shown below. Calculator <NUM> thus can calculate measurement coordinate system <NUM>.

Next, coordinate system calculator 222B in the third example will be described with reference to <FIG> and <FIG>.

<FIG> is a block diagram illustrating a configuration of the coordinate system calculator in the third example. <FIG> is a diagram for describing a method for calculating a measurement coordinate system by the coordinate system calculator in the third example.

Coordinate system calculator 222B includes detector <NUM>, extractor <NUM>, and calculator <NUM>. In the third example, which is not covered by the appended set of claims, marker <NUM> is disposed at a specific position on rack <NUM> (e.g., a position on its top face), and coordinate system calculator 222B determines measurement coordinate system <NUM> based on a position of marker <NUM>. That is, measurement coordinate system <NUM> in this case is a coordinate system based on the position of marker <NUM> placed on rack <NUM>.

Marker <NUM> has, for example, a checkered pattern. Marker <NUM> is not limited to a checkered pattern as long as marker <NUM> is an alignment mark (registration mark) having a prescribed shape.

From image <NUM> illustrated in (a) of <FIG>, which is a result of measurement by range sensor <NUM> obtained by obtainer <NUM>, detector <NUM> detects marker region <NUM> corresponding to marker <NUM> placed on rack <NUM> as illustrated in (c) of <FIG>.

From marker region <NUM> in image <NUM>, extractor <NUM> extracts pattern contour <NUM>, which is a contour of the checkered pattern, as illustrated in (d) of <FIG>.

Based on a shape of extracted pattern contour <NUM>, calculator <NUM> calculates rotation matrix <NUM> and translation vector <NUM> that indicate a positional relation between range sensor <NUM> and marker <NUM>. Using rotation matrix <NUM> and translation vector <NUM>, and a positional relation between storage three-dimensional model <NUM> and marker <NUM> illustrated in (b) of <FIG>, calculator <NUM> calculates a three-dimensional positional relation between range sensor <NUM> and rack <NUM> and calculates measurement coordinate system <NUM> by converting sensor coordinate system <NUM> using the calculated three-dimensional positional relation. It should be noted that the positional relation between storage three-dimensional model <NUM> and marker <NUM> may be measured in advance or may be generated in advance based on design information of rack <NUM> on which marker <NUM> is disposed.

Referring back to <FIG>, model generator <NUM> will be described.

Model generator <NUM> generates a storage three-dimensional model, which is a three-dimensional model of rack <NUM> where no baggage <NUM> is stored. Model generator <NUM> obtains a result of measurement by range sensor <NUM> on rack <NUM> where no baggage <NUM> is stored, thus generating the storage three-dimensional model. A specific process by model generator <NUM> will be described later. The generated storage three-dimensional model is stored in storage <NUM>.

Here, model generator <NUM> will be described specifically with reference to <FIG>.

<FIG> is a block diagram illustrating an example of a configuration of the model generator. <FIG> is a flowchart of a process of calculating a capacity of a storage space by the model generator.

Model generator <NUM> includes detector <NUM>, generator <NUM>, and capacity calculator <NUM>.

Detector <NUM> detects a rack region corresponding to rack <NUM> from a space three-dimensional model measured by range sensor <NUM> (S101). In a case where three-dimensional measurement system <NUM> includes range sensors <NUM>, detector <NUM> performs the process of step S101 on each of range sensors <NUM>. Detector <NUM> thus detects rack regions corresponding to range sensors <NUM>.

In a case where three-dimensional measurement system <NUM> includes range sensors <NUM>, generator <NUM> integrates the rack regions together, thus generating a storage three-dimensional model (S102). Specifically, generator <NUM> may perform alignment of a three-dimensional point cloud by Iterative Closest Point (ICP) to integrate the rack regions together or may calculate a relative positional relation among range sensors <NUM> in advance and integrate the rack regions together based on the calculated relative positional relation. The relative positional relation may be calculated by Structure from Motion (SfM) using images obtained by range sensors <NUM> as multi-viewpoint images. Range sensors <NUM> may be placed based on a design drawing in which the relative positional relation is determined.

The storage three-dimensional model of rack <NUM> may be generated by using results of measurement measured at positions to which one range sensor <NUM> is moved, rather than using range sensors <NUM>, and by integrating rack regions obtained from the results of measurement.

Without using the results of measurement by range sensor <NUM>, the storage three-dimensional model may be generated based on 3DCAD data at a time when rack <NUM> is designed or may be generated based on dimension measurement data of rack <NUM> or on equipment specification data of rack <NUM> published from its manufacturer.

In a case where three-dimensional measurement system <NUM> does not include range sensors <NUM> but includes only one range sensor <NUM>, and one result of measurement measured at one position is used, model generator <NUM> need not include generator <NUM>. That is, model generator <NUM> need not perform step S102.

Capacity calculator <NUM> calculates a capacity of storage space <NUM> of rack <NUM> using the storage three-dimensional model (S103).

Referring back to <FIG>, filling rate calculator <NUM> will be described.

Filling rate calculator <NUM> calculates a filling rate of baggage <NUM> with respect to storage space <NUM> of rack <NUM>. For example, filling rate calculator <NUM> may calculate, as the filling rate, a proportion of a volume of baggage <NUM> to the capacity of storage space <NUM> using a space three-dimensional model obtained by range sensor <NUM>, an image, and measurement coordinate system <NUM>.

Here, filling rate calculator <NUM> will be described specifically with reference to <FIG> and <FIG>.

<FIG> is a block diagram illustrating an example of a configuration of the filling rate calculator. <FIG> is a diagram for describing an example of a method for calculating the filling rate by the filling rate calculator. <FIG> illustrates an example of a case where range sensor <NUM> directly faces opening 102a of rack <NUM>. Range sensor <NUM> is disposed on a Z-axis negative direction side on which opening 102a of rack <NUM> is formed, and range sensor <NUM> measures storage space <NUM> of rack <NUM> via opening 102a of rack <NUM>. That is, range sensor <NUM> is disposed above rack <NUM> in a vertical direction. This example is an example of a case where measurement coordinate system <NUM> is measured by coordinate system calculator <NUM> in the first example. That is, in this case, sensor coordinate system <NUM> matches measurement coordinate system <NUM>.

Filling rate calculator <NUM> includes extractor <NUM>, estimator <NUM>, and calculator <NUM>.

Using space three-dimensional model <NUM> and a storage three-dimensional model, extractor <NUM> extracts baggage region <NUM>, which is a portion of the space three-dimensional model corresponding to baggage <NUM>. Specifically, extractor <NUM> converts a data structure of space three-dimensional model <NUM> illustrated in (a) of <FIG>, which is a result of measurement by range sensor <NUM> obtained by obtainer <NUM>, into voxel data, thus generating voxel data <NUM> illustrated in (b) of <FIG>. Using voxel data <NUM> generated and storage three-dimensional model <NUM> illustrated in (c) of <FIG>, which is a storage three-dimensional model converted into voxels, extractor <NUM> subtracts storage three-dimensional model <NUM> from voxel data <NUM>, thus extracting baggage region <NUM> in voxel data <NUM> illustrated in (d) of <FIG>, which is a region resulting from measuring baggage <NUM>. Baggage region <NUM> is an example of a target portion, which is a portion corresponding to a measurement target.

Using baggage region <NUM> extracted, estimator <NUM> estimates baggage model <NUM>, which is a three-dimensional model of baggage <NUM> in storage space <NUM>. Specifically, using baggage region <NUM>, estimator <NUM> interpolates baggage region <NUM> toward a region in which baggage <NUM> is hidden with respect to range sensor <NUM> in a Z-axis direction, in which range sensor <NUM> and rack <NUM> are arranged, that is, toward a Z-axis positive direction side. For example, for each of voxels constituting baggage region <NUM>, estimator <NUM> determines whether the voxel is a voxel that is disposed on the Z-axis negative direction side of a farthest voxel, which is disposed farthest on the Z-axis positive direction side among the voxels. When the voxel is disposed on the Z-axis negative direction side of the farthest voxel, in a case where there are no voxels disposed on the Z-axis positive direction side of the voxel, estimator <NUM> interpolates voxels up to the same position as a position of the farthest voxel in the Z-axis direction. Estimator <NUM> thus estimates baggage model <NUM> as illustrated in (e) of <FIG>.

Using the storage three-dimensional model and baggage model <NUM>, calculator <NUM> calculates a first filling rate of baggage <NUM> with respect to storage space <NUM>. Specifically, calculator <NUM> counts the number of voxels constituting baggage model <NUM> and multiplies a predetermined voxel size by the counted number, thus calculating the volume of baggage <NUM>. Calculator <NUM> calculates, as the first filling rate, a proportion of the calculated volume of baggage <NUM> with respect to the capacity of storage space <NUM> of rack <NUM> calculated by model generator <NUM>.

Range sensor <NUM> need not directly face opening 102a of rack <NUM>. <FIG> is a diagram for describing another example of the method for calculating the filling rate by the filling rate calculator. <FIG> illustrates an example of a case where range sensor <NUM> is disposed inclined with respect to opening 102a of rack <NUM>. This example is an example of a case where measurement coordinate system <NUM> is measured by coordinate system calculator 222A in the second example or coordinate system calculator 222B in the third example. That is, in this case, sensor coordinate system <NUM> differs from measurement coordinate system <NUM>.

A coordinate system used in the case in the example illustrated in <FIG> is measurement coordinate system <NUM>. Using baggage region <NUM>, estimator <NUM> interpolates baggage region <NUM> toward a region in which baggage <NUM> is hidden with respect to range sensor <NUM> in a Z-axis direction of measurement coordinate system <NUM>, in which range sensor <NUM> and rack <NUM> are arranged, that is, toward the Z-axis positive direction side.

The rest of processing by filling rate calculator <NUM> is the same as in the case illustrated in <FIG>, and thus description thereof will be omitted.

It should be noted that a combination of the space three-dimensional model and the image used for the calculation of the measurement coordinate system by coordinate system calculator <NUM> and the calculation of the filling rate by filling rate calculator <NUM> may be results of measurement performed by range sensor <NUM> at the same time or may be results of measurement performed at different times.

Range sensor <NUM> and information processing device <NUM> may be connected to each other via a communication network so as to be communicated with each other. The communication network may be a public telecommunication network such as the Internet or a private telecommunication network. Thus, the space three-dimensional model and the image obtained by range sensor <NUM> are transmitted from range sensor <NUM> to information processing device <NUM> via the communication network.

Information processing device <NUM> may obtain the space three-dimensional model and the image from range sensor <NUM> not via the communication network. For example, the space three-dimensional model and the image may be stored once from range sensor <NUM> in an external storage device such as a hard disk drive (HDD) and a solid state drive (SSD), and information processing device <NUM> may obtain the space three-dimensional model and the image from the external storage device. Alternatively, the external storage device may be a cloud server.

For example, information processing device <NUM> includes at least a computer system that includes a control program, a processing circuit that executes the control program, such as a processor and a logic circuit, and a recording device that stores the control program such as an internal memory or an accessible external memory. Functions by processing units of information processing device <NUM> may be implemented in a form of software or may be implemented in a form of hardware.

Next, operation of information processing device <NUM> will be described.

<FIG> is a flowchart of a filling rate measurement method performed by the information processing device.

Information processing device <NUM> obtains a space three-dimensional model from range sensor <NUM> (S111). At this time, information processing device <NUM> may further obtain an image of a measurement target from range sensor <NUM>.

Information processing device <NUM> obtains a storage three-dimensional model stored in storage <NUM> (S112).

Information processing device <NUM> calculates a measurement coordinate system based on a shape of opening 102a of rack <NUM> (S113). Step S113 is a process by coordinate system calculator <NUM>.

Using voxel data <NUM> of space three-dimensional model <NUM> and storage three-dimensional model <NUM> of the storage three-dimensional model, information processing device <NUM> extracts baggage region <NUM> that corresponds to baggage <NUM> in voxel data <NUM> (S114). Step S114 is a process by extractor <NUM> of filling rate calculator <NUM>.

Using baggage region <NUM> extracted, information processing device <NUM> estimates baggage model <NUM>, which is a three-dimensional model of baggage <NUM> in storage space <NUM> (S115). Step S115 is a process by estimator <NUM> of filling rate calculator <NUM>.

Using the storage three-dimensional model and baggage model <NUM>, information processing device <NUM> calculates a first filling rate of baggage <NUM> with respect to storage space <NUM> (S116). Step S116 is a process by calculator <NUM> of filling rate calculator <NUM>.

<FIG> is a flowchart of the process of calculating the measurement coordinate system by the coordinate system calculator in the first example (S113).

Coordinate system calculator <NUM> successively obtains images <NUM>, which are results of measurement by range sensor <NUM> obtained by obtainer <NUM>, in real time, and superimposes adjustment markers <NUM> on each of images <NUM> successively obtained (S121). Step S121 is a process by assister <NUM> of coordinate system calculator <NUM>.

Coordinate system calculator <NUM> obtains a position and orientation of range sensor <NUM> (S122). Step S122 is a process by assister <NUM> of coordinate system calculator <NUM>.

Using the position and the orientation of range sensor <NUM> at a time when four adjustment markers <NUM> are aligned with positions of four corners of opening 102a, coordinate system calculator <NUM> determines sensor coordinate system <NUM> of range sensor <NUM> and calculates measurement coordinate system <NUM> using determined sensor coordinate system <NUM> (S123). Step S123 is a process by calculator <NUM> of coordinate system calculator <NUM>.

<FIG> is a flowchart of the process of calculating the measurement coordinate system by the coordinate system calculator in the second example (S113).

Using space three-dimensional model <NUM>, which is a result of measurement by range sensor <NUM> obtained by obtainer <NUM>, and storage three-dimensional model <NUM>, coordinate system calculator 222A detects rack region <NUM> corresponding to rack <NUM> (S121A). Step S121A is a process by detector <NUM> of coordinate system calculator 222A.

Using position information <NUM> in storage three-dimensional model <NUM>, coordinate system calculator 222A extracts four opening endpoints <NUM>, which are positions of four corners of opening <NUM> in rack region <NUM> (S122A). Step S122A is a process by extractor <NUM> of coordinate system calculator 222A.

Coordinate system calculator 222A calculates rotation matrix <NUM> and translation vector <NUM> that indicate a positional relation between range sensor <NUM> and rack <NUM> based on a shape of four opening endpoints <NUM> as viewed from range sensor <NUM>. Coordinate system calculator 222A then converts sensor coordinate system <NUM> of range sensor <NUM> using rotation matrix <NUM> and translation vector <NUM>, thus calculating measurement coordinate system <NUM> (S123A). Step S123A is a process by calculator <NUM> of coordinate system calculator 222A.

<FIG> is a flowchart of the process of calculating the measurement coordinate system by the coordinate system calculator in the third example (S113), which is not covered by the appended set of claims.

Coordinate system calculator 222B detects marker region <NUM> from image <NUM>, which is a result of measurement by range sensor <NUM> obtained by obtainer <NUM> (S121B). Step S121B is a process by detector <NUM> of coordinate system calculator 222B.

From marker region <NUM> in image <NUM>, coordinate system calculator 222B extracts pattern contour <NUM> (S122B). Step S122B is a process by extractor <NUM> of coordinate system calculator 222B.

Based on a shape of extracted pattern contour <NUM>, coordinate system calculator 222B calculates rotation matrix <NUM> and translation vector <NUM> that indicate a positional relation between range sensor <NUM> and marker <NUM>. Using rotation matrix <NUM> and translation vector <NUM>, and a positional relation between storage three-dimensional model <NUM> and marker <NUM>, coordinate system calculator 222B then calculates a three-dimensional positional relation between range sensor <NUM> and rack <NUM> and calculates measurement coordinate system <NUM> by converting sensor coordinate system <NUM> using the calculated three-dimensional positional relation (S123B). Step S123B is a process by calculator <NUM> of coordinate system calculator 222B.

The filling rate calculated by information processing device <NUM> may be output from information processing device <NUM>. The filling rate may be displayed by a display device not illustrated included in information processing device <NUM> or may be transmitted to an external device different from information processing device <NUM>. For example, the calculated filling rate may be output to a baggage conveyance system and used for controlling the baggage conveyance system.

In the filling rate measurement method according to the present embodiment, baggage model <NUM> of baggage <NUM> is estimated using baggage region <NUM> that is extracted using the space three-dimensional model made by measuring rack <NUM> in a state where baggage <NUM> is stored and the storage three-dimensional model of rack <NUM> where no baggage <NUM> is stored. This enables the first filling rate of baggage <NUM> with respect to storage space <NUM> to be calculated easily only by measuring rack <NUM> in a state where baggage <NUM> is stored.

In addition, in the filling rate measurement method, baggage model <NUM> is estimated based on a three-dimensional coordinate system based only on a shape of a part of rack <NUM>. A shape of only a part of the first storage, which is easy to extract on an image, can be used for calculation of a measurement coordinate system. Therefore, a processing speed of the estimation of the baggage model can be improved, and a precision of calculating the measurement coordinate system can be improved.

Further, in the filling rate measurement method, the three-dimensional coordinate system is a three-dimensional orthogonal coordinate system having the Z axis, and baggage model <NUM> is estimated by interpolating the Z-axis positive direction side, which is opposite to a Z-axis negative direction of baggage region <NUM>. This enables an effective reduction in processing amount of the estimation of baggage model <NUM>.

Further, in the filling rate measurement method, the three-dimensional coordinate system is a coordinate system based on the shape of opening 102a of rack <NUM>. Therefore, the coordinate system based on the shape of opening 102a of rack <NUM> can be calculated easily, and baggage model <NUM> can be estimated based on the calculated coordinate system.

Further, in a filling rate measurement method, which is not covered by the appended set of claims, the three-dimensional coordinate system is a coordinate system based on marker <NUM> placed on rack <NUM>. Therefore, the coordinate system based on marker <NUM> can be calculated easily, and baggage model <NUM> can be estimated based on the calculated coordinate system.

Further, in the filling rate measurement method, range sensor 210B includes at least two cameras for generating a space three-dimensional model. Range sensor <NUM> including such range sensor 210B is fixed above the first storage.

In a case where range sensor <NUM> is fixed above the first storage in this manner, in a case where the first storage is movable such as a cage carriage described later, objects present within a measurement range of range sensor <NUM> are limited to the ground, a mount (bottom face) of the cage carriage, or the like, there is no movable object other than the cage carriage; therefore, a measurement target can be easily separated from a background from a result of measurement. It should be noted that the measurement range is an imaging range of a camera in a case where range sensor <NUM> includes the camera. In contrast, in a case where range sensor <NUM> is fixed at a position not above the first storage, a moving object other than the first storage tends to be present within the measurement range, and thus it is difficult to separate a measurement target from a background.

Information processing device <NUM> according to the embodiment described above is configured to calculate the proportion of the volume of baggage <NUM> stored in storage space <NUM> with respect to the capacity of storage space <NUM> as the filling rate, but the configuration is not limited to this.

<FIG> is a diagram for describing a method for calculating a filling rate.

In (a) and (b) of <FIG>, storage space <NUM> of rack <NUM> has a capacity that is capable of storing just <NUM> pieces of baggage <NUM>. As illustrated (a) of <FIG>, when eight pieces of baggage <NUM> are closely disposed, a vacancy of storage space <NUM> can store additional eight pieces of baggage <NUM>. In contrast, as illustrated (b) of <FIG>, when the pieces of baggage are disposed not closely, it is necessary to move the pieces of baggage <NUM> already stored so as to store additional eight pieces of baggage <NUM> in the rest of the space of storage space <NUM>. If pieces of baggage <NUM> are stored in the rest of the space of storage space <NUM> without moving the pieces of baggage <NUM> already stored, only six pieces of baggage <NUM> can be stored.

As seen from the above, although the numbers of pieces of baggage <NUM> storable in the rest of the space of storage space <NUM> are different between the case illustrated in (a) of <FIG> and the case illustrated in (b) of <FIG>, filling rates of both cases are calculated as the same filling rate, <NUM>%. It is therefore conceivable to calculate a filling rate with consideration given to a space in which baggage can be practically stored, according to a shape of the rest of the space of storage space <NUM>.

<FIG> is a block diagram illustrating an example of a configuration of a calculator of a filling rate calculator according to Variation <NUM>. <FIG> is a flowchart of a filling rate calculating process by the calculator of the filling rate calculator according to Variation <NUM>.

As illustrated in <FIG>, calculator <NUM> includes baggage volume calculator <NUM>, region divider <NUM>, intended baggage measurer <NUM>, region estimator <NUM>, and calculator <NUM>.

Baggage volume calculator <NUM> calculates a baggage volume, which is a volume of baggage <NUM>, from baggage model <NUM> (S131). Baggage volume calculator <NUM> calculates the volume of baggage <NUM> stored in storage space <NUM> by the same method as in the embodiment.

Next, region divider <NUM> divides storage space <NUM> in the space three-dimensional model <NUM> into occupied region <NUM> that is occupied by baggage <NUM> and vacant region <NUM> that is not occupied by baggage <NUM> (S132).

Next, intended baggage measurer <NUM> calculates a volume of one piece of baggage that is intended to be stored (S133). In a case where pieces of baggage intended to be stored are of types in shape and size as illustrated in (c) of <FIG>, intended baggage measurer <NUM> calculates a volume of one piece of baggage based on its type. For example, intended baggage measurer <NUM> calculates a volume of a piece of baggage 103a, a volume of a piece of baggage 103b, and a volume of a piece of baggage 103c.

Next, region estimator <NUM> estimates a disposing method that enables pieces of baggage <NUM> intended to be stored the most in vacant region <NUM> and estimates the number of pieces of baggage <NUM> intended to be stored in this case. That is, region estimator <NUM> estimates a maximum number of storable pieces of baggage <NUM> intended to be stored in vacant region <NUM>. Region estimator <NUM> calculates a capacity of vacant region <NUM> capable of storing baggage by multiplying the volume of one piece of baggage by the number of storable pieces of baggage (S134).

In a case where there are types of baggage, region estimator <NUM> may estimate the number of pieces of baggage that can be stored for each type or may estimate the numbers of pieces of baggage of the types in combination. In a case where pieces of baggage of the types are stored in combination, region estimator <NUM> calculates an integrated value of a capacity obtained by multiplying a volume of one piece of baggage of each type by the number of storable pieces of baggage of the type, as a capacity of vacant region <NUM> capable of storing baggage. For example, when estimating that number n1 of pieces of baggage 103a, number n2 of pieces of baggage 103b, and number n3 of pieces of baggage 103c are storable, region estimator <NUM> calculates an integrated value of a first volume resulting from multiplying a volume of a piece of baggage 103a by n1, a second volume resulting from multiplying a volume of a piece of baggage 103b by n2, and a third volume resulting from multiplying a volume of a piece of baggage 103c by n3, as the capacity of vacant region <NUM> capable of storing baggage. It should be noted that n1, n2, and n3 are each an integer larger than or equal to zero.

Calculator <NUM> calculates the filling rate by substituting the volume of baggage already stored and the capacity capable of storing baggage into Equation <NUM> shown below (S135).

As seen from the above, filling rate calculator <NUM> may calculate the proportion of the volume of baggage <NUM> stored in storage space <NUM> with respect to the capacity of an available space for storing baggage <NUM> in storage space <NUM>, as the filling rate.

This enables the calculation of the first filling rate for appropriately determining how many pieces of baggage <NUM> can be stored in a vacant space of storage space <NUM>.

Information processing device <NUM> according to the embodiment described above is configured to calculate the filling rate of baggage <NUM> with respect to storage space <NUM> of one rack <NUM>, but a filling rate of baggage <NUM> with respect to storage spaces <NUM> of two or more racks <NUM>.

<FIG> is a diagram illustrating an example of a case where two or more racks are stored in a storage space such as a platform of a truck. <FIG> is a table showing a relation between racks stored in the storage space on the platform and their filling rates.

As illustrated in <FIG>, in platform <NUM> including storage space <NUM>, cage carriages <NUM> are stored. Platform <NUM> may be a van-body type platform of a truck. Platform <NUM> is an example of a second storage. The second storage is not limited to platform <NUM> and may be a container or a storehouse.

Storage space <NUM> is an example of a second storage space. Storage space <NUM> has a capacity of a size that allows cage carriages <NUM> to be stored. In Variation <NUM>, storage space <NUM> is capable of storing six cage carriages <NUM>. Being capable of storing cage carriages <NUM>, storage space <NUM> is larger than storage spaces <NUM>.

Cage carriages <NUM> each have storage space <NUM> that is capable of storing pieces of baggage <NUM>. Cage carriage <NUM> is an example of the first storage. Storage space <NUM> is an example of the first storage space. In storage space <NUM>, rack <NUM> described in the embodiment may be stored.

The pieces of baggage <NUM> are not directly stored in platform <NUM> by stored in cage carriages <NUM>. Cage carriages <NUM> storing the pieces of baggage <NUM> are stored in platform <NUM>.

A configuration of calculator <NUM> of filling rate calculator <NUM> in this case will be described.

As illustrated in <FIG>, calculator <NUM> according to Variation <NUM> includes obtainer <NUM>, counter <NUM>, and calculator <NUM>.

Obtainer <NUM> obtains the number of cage carriages <NUM> that are storable in platform <NUM> (S141). In a case of Variation <NUM>, a maximum number of cage carriages <NUM> storable in platform <NUM> is six, and thus obtainer <NUM> obtains six.

Counter <NUM> counts the number of cage carriages <NUM> to be stored in platform <NUM> (S142). In a case where cage carriages <NUM> illustrated in <FIG> are stored in platform <NUM>, counter <NUM> takes three as the count of the number of cage carriages <NUM>.

Calculator <NUM> calculates a second filling rate, which is a filling rate of one or more cage carriages <NUM> with respect to platform <NUM> (S143). Specifically, calculator <NUM> may calculate, as the second filling rate, a proportion of the number of cage carriages <NUM> stored in platform <NUM> with respect to a maximum number of cage carriages <NUM> storable in platform <NUM>. For example, up to six cage carriages <NUM> are storable in platform <NUM>, and three cage carriages <NUM> out of six are stored in platform <NUM>, and thus calculator <NUM> calculates <NUM>% as the second filling rate.

It should be noted that calculator <NUM> may calculate a filling rate of baggage <NUM> with respect to each of one or more cage carriages <NUM> stored in platform <NUM> and calculate, using the calculated filling rate, a filling rate of baggage <NUM> with respect to second storage space. Specifically, calculator <NUM> may calculate an average of filling rates of baggage <NUM> with respect to cage carriages <NUM> as the filling rate of baggage <NUM> with respect to the second storage space. In this case, when there is a remaining available space for storing cage carriages <NUM> in storage space <NUM> of platform <NUM>, calculator <NUM> may calculate the average assuming that a filling rate of cage carriages <NUM> of the number of cage carriages <NUM> storable in the remaining space capable of storing cage carriages <NUM> is <NUM>%.

For example, in a case where filling rates of three cage carriages <NUM> illustrated in <FIG> are <NUM>%, <NUM>%, and <NUM>%, and six cage carriages <NUM> are storable in platform <NUM> at the maximum, filling rates of the six cage carriages <NUM> may be given as <NUM>%, <NUM>%, <NUM>%, <NUM>%, <NUM>%, and <NUM>%, and a result of determining their average, <NUM>%, may be calculated as the filling rate of baggage <NUM> with respect to the second storage space.

This enables the second filling rate in a case where one or more cage carriages <NUM> are stored in storage space <NUM> to be calculated appropriately.

Next, Variation <NUM> will be described.

<FIG> is a diagram for describing a configuration of a cage carriage according to Variation <NUM>.

In <FIG> is a diagram illustrating cage carriage <NUM> of which cover part <NUM> that is opened and closed is in a closed state. In <FIG> is a diagram illustrating cage carriage <NUM> of which cover part <NUM> is in an open state.

Cage carriage <NUM> according to Variation <NUM> includes cover part <NUM> that opens and closes opening 112a. Cover part <NUM> is a lattice-like or mesh-like cover having through holes 113a. Therefore, even when cover part <NUM> of cage carriage <NUM> is in the closed state, range sensor <NUM> can measure a three-dimensional shape of an inside of storage space <NUM> of cage carriage <NUM> via through holes 113a and opening 112a.

This is because electromagnetic waves emitted by range sensor <NUM> pass through through holes 113a and opening 112a. It should be noted that, in a case of range sensor 210A, an infrared pattern emitted by range sensor 210A passes through through holes 113a and opening 112a, and thus, even when cover part <NUM> of cage carriage <NUM> is in the closed state, the three-dimensional shape of the inside of storage space <NUM> of cage carriage <NUM> can be measured via through holes 113a and opening 112a. Further, in a case of range sensor 210B, two cameras 211B and 212B are capable of imaging the inside of storage space <NUM> via through holes 113a and opening 112a, and thus the three-dimensional shape of the inside of storage space <NUM> of cage carriage <NUM> can be measured.

Information processing device <NUM> therefore can determine whether baggage <NUM> is stored in storage space <NUM>. However, it is difficult to calculate a correct filling rate unless a method of calculating a filling rate is switched to another method between a case where cover part <NUM> is in the closed state and a case where cover part <NUM> is in the open state or a case where cover part <NUM> is not provided. Thus, filling rate calculator <NUM> according to Variation <NUM> calculates a filling rate by a first method when cover part <NUM> is in the open state and calculates a filling rate by a second method when cover part <NUM> is in the closed state.

<FIG> is a block diagram illustrating an example of a configuration of a filling rate calculator according to Variation <NUM>. <FIG> is a flowchart of a filling rate calculating process by the filling rate calculator according to Variation <NUM>.

As illustrated in <FIG>, filling rate calculator <NUM> according to Variation <NUM> includes detector <NUM>, switcher <NUM>, first filling rate calculator <NUM>, and second filling rate calculator <NUM>.

Detector <NUM> detects an open/closed state of cover part <NUM> using a space three-dimensional model (S151). Specifically, using the space three-dimensional model, detector <NUM> detects that cover part <NUM> is in the closed state when three-dimensional point clouds are present at positions inside and outside storage space <NUM> in a front-back direction of a region of opening 112a of cage carriage <NUM> (i.e., a direction in which range sensor <NUM> and cage carriage <NUM> are arranged). When a three-dimensional point cloud is present only inside storage space <NUM>, detector <NUM> detects that cover part <NUM> is in the open state.

Switcher <NUM> determines whether cover part <NUM> is in the open state or the closed state (S152), and switches between the following processes according to a result of the determination.

When cover part <NUM> is determined to be in the open state by switcher <NUM> (Open state in S152), first filling rate calculator <NUM> calculates a filling rate by the first method (S153). Specifically, first filling rate calculator <NUM> calculates a filling rate of cage carriage <NUM> by performing the same process as the process by filling rate calculator <NUM> in the embodiment.

When cover part <NUM> is determined to be in the closed state by switcher <NUM> (Closed state in S152), second filling rate calculator <NUM> calculates a filling rate by the second method (S154). The second method will be described in detail with reference to <FIG>.

<FIG> is a diagram for describing an example of the second method for calculating a filling rate.

As illustrated in (a) of <FIG>, consider a case where space three-dimensional model <NUM> is obtained.

In <FIG> is a diagram of region R2 in space three-dimensional model <NUM> in an enlarged manner. As illustrated in (b) of <FIG>, second filling rate calculator <NUM> classifies region R2 into a second portion where cover part <NUM> is detected and a first portion where baggage <NUM> is detected.

The first portion is a region including a three-dimensional point cloud on a back side of a region of opening 112a. In addition, the first portion is a portion through which range sensor <NUM> faces baggage <NUM> in a direction from range sensor <NUM> to baggage <NUM>. That is, the first portion is a portion that faces through holes 113a in cover part <NUM> in the closed state in the direction from range sensor <NUM> to baggage <NUM>. It should be noted that cover part <NUM> may have a configuration having one through hole 113a. Further, the direction from range sensor <NUM> to baggage <NUM> may be horizontal, for example.

The second portion is a region including a three-dimensional point cloud on a front side of a region of opening 112a of cage carriage <NUM> in the front-back direction. In addition, the second portion is a portion through which range sensor <NUM> does not face baggage <NUM> in a direction from range sensor <NUM> to baggage <NUM>. That is, the second portion is a portion that is hidden by cover part <NUM> in the closed state in the direction from range sensor <NUM> to baggage <NUM>.

Second filling rate calculator <NUM> converts the first portion and the second portion into voxels, thus generating voxel data <NUM> illustrated in (c) of <FIG>. In voxel data <NUM>, white regions not hatched are regions where the second portion has been converted into voxels, and dot-hatched regions are regions where the first portion has been converted into voxels.

On the white regions corresponding to regions of cover part <NUM>, second filling rate calculator <NUM> then estimates whether baggage <NUM> is present on the back side of cover part <NUM>. Specifically, in regions where the conversion into voxels has been carried out, second filling rate calculator <NUM> assigns a score based on a probability that the baggage is present to each of <NUM> voxels adjacent to a dot-hatched voxel, where baggage <NUM> is present. Then, as illustrated in (d) of <FIG>, additional scores are assigned to voxels illustrated as white regions adjacent to voxels where baggage <NUM> is present. Second filling rate calculator <NUM> performs this on all voxels where baggage <NUM> is present and determines that baggage <NUM> is present in voxels illustrated as white regions each of which has a total value of the scores being greater than or equal to a given threshold value. For example, when the given threshold value is assumed to be <NUM>, second filling rate calculator <NUM> determines that baggage <NUM> is present in all the regions, and thus, as illustrated in (e) of <FIG>, baggage model <NUM> into which a shape of a region concealed by cover part <NUM> is estimated can be calculated.

In this manner, information processing device <NUM> estimates a shape of the second portion through which range sensor <NUM> does not face a measurement target based on a shape of the first portion through which the range sensor faces baggage <NUM>, and thus, even in a case where the second portion is present, a target three-dimensional model can be estimated appropriately.

In a case where there is a rule that pieces of baggage <NUM> are to be closely disposed inside cage carriage <NUM>, second filling rate calculator <NUM> may extract, as illustrated in <FIG>, contour R3 of a region where one or more pieces of baggage <NUM> are disposed and may determine that pieces of baggage <NUM> are present inside extracted contour R3. Then, second filling rate calculator <NUM> may estimate a region of cover part <NUM> inside contour R3 using a three-dimensional point cloud in a region of through holes 113a of cover part <NUM>.

In a filling rate measurement method according to Variation <NUM>, cage carriage <NUM> further has through holes 113a and cover part <NUM> that opens and closes opening 112a. Further, in the filling rate measurement method, whether cover part <NUM> is in the open state or in the closed state is determined, and when cover part <NUM> is in the open state, baggage model <NUM> is estimated by extraction and estimation as filling rate calculator <NUM> in the embodiment does. When cover part <NUM> is in the closed state, filling rate calculator <NUM> estimates second portions hidden by cover part <NUM> based on first portions corresponding to through holes 113a of cover part <NUM> in voxel data <NUM> based on space three-dimensional model <NUM> and estimates baggage model <NUM> using the first portions, the estimated second portions, and storage three-dimensional model <NUM>.

According to this, even in a case where pieces of baggage <NUM> are stored in cage carriage <NUM> provided with cover part <NUM> that opens and closes opening 112a, the method for estimating baggage model <NUM> is switched between the first method and the second method according to the open/closed state of cover part <NUM>, and thus a target three-dimensional model can be estimated appropriately.

Further, in the filling rate measurement method according to Variation <NUM>, the direction from range sensor <NUM> to baggage <NUM> is horizontal. This eliminates a need to adjust a position of range sensor <NUM> so that measurement can be performed in a direction in which cover part <NUM> having through holes 113a is not present, and thus a flexibility of placing range sensor <NUM> is high. Therefore, a result of measurement for estimating a target three-dimensional model by range sensor <NUM> can be obtained even when the position of range sensor <NUM> is not adjusted completely.

<FIG> is a diagram for describing a method for generating a space three-dimensional model according to Variation <NUM>.

As illustrated in <FIG>, in a case where a space three-dimensional model is generated, three-dimensional measurement system <NUM> may integrate results of measurement by range sensors <NUM> together as in the processing by model generator <NUM>. In this case, three-dimensional measurement system <NUM> determines positions and orientations of range sensors <NUM> by performing calibration in advance and integrates obtained results of measurement together based on the determined positions and orientations of range sensors <NUM>, so that a space three-dimensional model including a three-dimensional point cloud with little occlusion can be generated.

As illustrated in <FIG>, in a case where a space three-dimensional model is generated, three-dimensional measurement system <NUM> may cause at least one of cage carriage <NUM> and one range sensor <NUM> to move in such a manner as to traverse measurement region R1 of one range sensor <NUM>, and results of measurement obtained by range sensor <NUM> at timings during the movement may be integrated together. In this case, a relative position and a relative orientation between cage carriage <NUM> and one range sensor <NUM> are calculated, and results of measurement are integrated together using the relative position and the relative orientation, so that a space three-dimensional model including a three-dimensional point cloud with little occlusion can be generated.

Although the filing rate measurement method and the like according to the present disclosure have been described based on the above embodiments, the present disclosure is not limited to the embodiments.

For example, in the above embodiments, each processing unit included in the information processing device is implemented to a CPU and a control program. For example, the constituent elements of each processing unit may be implemented to one or more electronic circuits. Each of the one or more electronic circuits may be a general-purpose circuit or a dedicated circuit. The one or more electronic circuits may include, for example, an Integrated Circuit (IC), a Large Scale Integration (LSI), and the like. The IC or LSI may be integrated to a single chip or integrated to a plurality of chips. Here, the terminology "LSI" or "IC" is used, but depending on the degree of integration, the circuit may also be referred to as a system LSI, a Very Large Scale Integration (VLSI), or an Ultra Large Scale Integration (ULSI). A Field Programmable Gate Array (FPGA) that is programed after manufacturing the LSI may be used for the same purpose.

It should be noted that general or specific aspects of the present disclosure may be implemented to a system, a device, a method, an integrated circuit, or a computer program. The general or specific aspects of the present disclosure may be implemented to a non-transitory computer-readable recording medium such as an optical disk, a Hard Disk Drive (HDD), or a semiconductor memory, on which the computer program is recorded. Furthermore, the general or specific aspects of the present disclosure may be implemented to any combination of the system, the device, the method, the integrated circuit, or the computer program.

In addition, the present disclosure may include embodiments obtained by making various modifications on the above embodiments which those skilled in the art will arrive at, or embodiments obtained by selectively combining the elements and functions disclosed in the above embodiments, without materially departing from the scope of the present disclosure.

Claim 1:
A filling rate measurement method comprising:
obtaining (S111) a space three-dimensional model generated (<NUM>; <NUM>) by measuring a first storage (<NUM>) having an opening (102a; 112a) and a first storage space (<NUM>) in which a measurement target (<NUM>) is to be stored, the measuring being performed through the opening (102a) using a range sensor (<NUM>) facing the first storage (<NUM>);
obtaining (S112) a storage three-dimensional model (<NUM>; <NUM>; <NUM>) that is a three-dimensional model of the first storage (<NUM>) in which the measurement target (<NUM>) is not stored;
extracting (S114) a target portion (<NUM>) corresponding to the measurement target (<NUM>) from the space three-dimensional model using the space three-dimensional model obtained and the storage three-dimensional model obtained;
calculating (S113) a first three-dimensional coordinate system based on only a shape of a part (<NUM>) of the first storage as viewed from the range sensor (<NUM>);
estimating (S115) a target three-dimensional model using the target portion extracted and the first three-dimensional coordinate system calculated, the target three-dimensional model being a three-dimensional model of the measurement target (<NUM>) in the first storage space (<NUM>); and
calculating (S116) a first filling rate of the measurement target (<NUM>) with respect to the first storage space (<NUM>), using the storage three-dimensional model and the target three-dimensional model.