Management device and management system

A management device includes a distance sensor, a sensor interface, and a processor. The distance sensor is configured to measure a distance between a predetermined position and a backmost side of a commodity line having commodities biased in an arrangement direction and supported from a foremost side of the commodity line. The sensor interface transmits and receives data to and from the distance sensor. The processor calculates a length of the commodity line based on the distance before a change in the distance and the distance after the change in the distance when the distance is changed, and determines whether the length of the commodity line matches a thickness of the commodity indicated by shelf label information. The processor calculates the increased or decreased number of commodities based on the determination.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2020-097462, filed on Jun. 4, 2020, the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

Embodiments described herein relate generally to a management device and a management system.

BACKGROUND

In stores, devices are provided to specify commodities (e.g., articles) extracted from shelves by users through image processing. Such devices count the number of extracted commodities by separating images of the commodities extracted from the shelves by the user through image processing.

In the related art, there is concern that the devices fail to separate images and fail to count the increased or decreased number of commodities.

DETAILED DESCRIPTION

At least one exemplary embodiment provides a management device and a management system capable of efficiently calculating an increase and a decrease in the number of commodities.

In general, according to at least one embodiment, a management device includes a sensor interface and a processor. The sensor interface transmits and receives data to and from a distance sensor that measures a distance between a predetermined position and a backmost side of a commodity line. The commodity line is formed by commodities biased in an arrangement direction and supported from a foremost side of the commodity line. The processor calculates a length of the commodity line based on a difference between the distance before a change in the distance and the distance after a change in the distance, each measured by the distance sensor when the distance is changed. The processor further determines whether a length of the commodity line matches a thickness of the commodity indicated by shelf label information. The processor calculates the increased or decreased number of commodities based on the length of the commodity line when the processor determines that the length of the commodity line matches the thickness of the commodity indicated by the shelf label information.

Hereinafter, at least one embodiment will be described with reference to the drawings. A management system according to the at least one embodiment calculates the number of commodities extracted from a shelf by a user in a store, for instance, or the number of commodities which are added by a user (an operator). The management system is used for a system that automatically adjusts commodities. The management system may calculate the increased or decreased number of commodities in a warehouse, a factory, or the like.

FIG.1is a diagram schematically illustrating a configuration example of a management system1.FIG.2is a diagram schematically illustrating a cross section of a shelf100included in the management system1according to the at least one embodiment. As illustrated inFIGS.1and2, the management system1includes the management device10, a camera20, electronic shelf labels30, a distance sensor40, and a shelf100.

The management device10is connected to the camera20, the electronic shelf labels30, and the distance sensor40. The management system1may be configured as illustrated inFIGS.1and2. However, the management device10may be in a variety of configurations or various components may be excluded from the management system1.

Here, the management system1calculates the increased or decreased numbers of commodities A, B, and C exhibited on the shelf100.

The management device10calculates the increased or decreased number of commodities exhibited on the shelf100based on a sensor signal from the distance sensor40. The management device10will be described later in detail.

The camera20captures images of the commodities A, B, and C. Here, the camera20captures images of the front side (face) of the commodities A, B, and C located in front. The camera20captures images of the commodities A to C under the control of the management device10. The camera20transmits the captured image to the management device10.

For example, the camera20is coupled to a shelf stage formed above the commodities A to C and is angled downward.

The electronic shelf labels30display various kinds of information under the control of the management device10. For example, the electronic shelf labels30display names, prices, and the like of the commodities. The electronic shelf labels30may display an error indicating that the exhibition of the commodities is wrong.

A support member101(e.g., a support mechanism) is formed at the shelf stage of the shelf100. The support member101is formed in a rectangular bar shape. The support member101is formed in a predetermined length on the front side of the shelf stage. The support member101is configured to be lower than the heights of the commodities A to C.

As will be described below, the commodities A to C are biased (e.g., urged, pressed, moved, forced, etc.) toward the front side. The support member101supports the biased commodities A to C from the front side. The support member101comes into contact with the lower ends of the commodities A to C to support the commodities A to C.

As illustrated inFIG.2, an biasing member102is formed in the shelf100. Here, the biasing member102that biases (e.g., urges) the commodities A will be described. The biasing member102biases the commodities A in a direction in which the commodities A are arranged (e.g., an arrangement direction). The biasing member102biases the commodities A in the front direction from the back side of the shelf stage.

The biasing member102includes a plate102aand a spring102b. The plate102ais installed to be movable in a depth direction (the arrangement direction of the commodities A) on the shelf stage of the shelf100.

The spring102bbiases the plate102ain the front direction from the back side of the plate102a. One end of the spring102bis fixed near a front side of the shelf stage and the other end thereof is fixed to the plate102a. The spring102bis fixed to the plate102aand is configured to be extended more than a natural length.

The biasing member102urges the plate102ain the front direction of the shelf stage by a tensile force of the spring102b. Therefore, the commodities A arranged between the support member101and the plate102aare biased to the front side by the biasing member102. The distance sensor40is installed on the back plate of the shelf100. The distance sensor40measures a distance between the back plate to the backmost side of the commodity line (e.g., the article line) in which the commodities A are formed. That is, the distance sensor40measures a distance to the back side of the commodity A disposed on the back-most side (e.g., the commodity A at the end of the line). Here, the distance sensor40measures a distance to the plate102a. The distance sensor40transmits the measured distance to the management device10.

For example, the distance sensor40measures a distance based on reflected light of light (visible light or invisible light) radiated from a light source. The distance sensor40may use a time-of-flight (ToF) scheme of measuring a distance to a measurement target based on a time until the radiated light is reflected from the measurement target and arrives at an optical sensor. The configuration of the distance sensor40is not limited to a specific configuration.

The biasing member102and the distance sensor40are formed in the shelf100for each commodity line. That is, the biasing member102and the distance sensor40are also formed in the line of the commodities B and the line of the commodities C.

The biasing member102may be a member independent from the shelf100. For example, the biasing member102may be placed on the shelf plate. The biasing member102is not limited to a specific configuration as long as the biasing member102is a mechanism that biases commodities toward the front side.

Next, the management device10will be described.FIG.3is a block diagram illustrating a configuration example of the management device10. As illustrated inFIG.3, the management device10includes a processor11, a ROM12, a RAM13, an NVM14, a communication unit15(e.g., a communicator), an operation unit16(e.g., an operator), a display unit17(e.g., a display interface), a camera interface18, a shelf label interface19, and a sensor interface110.

The processor11, the ROM12, the RAM13, the NVM14, the communication unit15, the operation unit16, the display unit17, the camera interface18, the shelf label interface19, and the sensor interface110are connected to each other via a data bus or the like. The management device10may be configured as illustrated inFIG.3. However, the management device10may be in a variety of configurations or certain components may be excluded from the management device10.

The processor11is configured to control an operation of the entire management device10. The processor11may include an internal cache and various interfaces. The processor11implements various processes by executing programs stored in advance in an internal memory, the ROM12, or the NVM14.

Some of the various functions implemented by allowing the processor11to execute programs may be implemented by hardware circuits. In this case, the processor11controls the functions implemented by the hardware circuits.

The ROM12is a nonvolatile memory in which a control program, control data, and the like are stored in advance. The control program and the control data stored in the ROM12are embedded in advance in accordance with the specification of the management device10.

The RAM13is a volatile memory. The RAM13temporarily stores data or the like which is being processed by the processor11. The RAM13stores various application programs in response to commands from the processor11. The RAM13may store data necessary to execute the application programs, execution results of the application programs, and the like.

The NVM14is a nonvolatile memory capable of writing and rewriting data. The NVM14is configured as, for example, a hard disk drive (HDD), a solid-state drive (SSD), or a flash memory. The NVM14stores a control program, applications, and various kinds of data in accordance with operational uses of the management device10.

The communication unit15is an interface that transmits and receives data to and from an external device. For example, the communication unit15is an interface that supports connection of a wired or wireless local area network (LAN).

The operation unit16receives inputs of various operations from the user. The operation unit16transmits a signal indicating an input operation to the processor11. The operation unit16may be configured as a touch panel. The display unit17displays image data from the processor11. For example, the display unit17is configured as a liquid crystal monitor. When the operation unit16is configured as a touch panel, the display unit17may be integrated with the operation unit16.

The camera interface18is an interface that transmits and receives data to and from the camera20. The camera interface18is connected to the camera20in a wired or wireless manner. For example, the camera interface18transmits a signal for causing the camera20to capture an image under the control of the processor11. The camera interface18transmits the image captured by the camera20to the processor11.

The shelf label interface19is an interface that transmits and receives data to and from the electronic shelf labels30. The shelf label interface19is connected to the electronic shelf labels30in a wired or wireless manner. For example, the shelf label interface19transmits a signal for displaying an image to the electronic shelf labels30under the control of the processor11.

The sensor interface110is an interface that transmits and receives data to and from the distance sensor40. The sensor interface110is connected to the distance sensor40in a wired or wireless manner. For example, the sensor interface110transmits a distance measured by the distance sensor40to the processor11.

The camera interface18, the shelf label interface19, and the sensor interface110may be integrated.

Next, functions of the management device10will be described. The functions of the management device10are attained by allowing the processor11to execute a program stored in an internal memory, the ROM12, or the NVM14.

First, the processor11has a function of acquiring the number of commodities exhibited on the shelf100based on a distance measured by the distance sensor40.

For example, when the user completes the exhibition of the commodities on the shelf100, the processor11acquires shelf label information. The shelf label information indicates commodities exhibited at each position of the shelf100.

When the shelf label information is acquired, the processor11acquires the thickness of the commodity exhibited at a predetermined position. Here, the thickness of the commodity is a distance in the depth direction when the commodity is exhibited on the shelf100.

The thickness of the commodity may be included in the shelf label information. In this case, the processor11acquires the thickness of the commodity with reference to the shelf label information. The processor11may acquire the thickness of the commodity from a database in which external dimensions of the commodity are stored. In this case, the database may be stored in the NVM14or the like, or may be stored in an external device.

When the thickness of the commodity is acquired, the processor11acquires a distance to the backmost side of the commodity line using the distance sensor40.

When the distance to the backmost side of the commodity line is acquired, the processor11calculates the number of commodities based on the acquired distance. For example, the processor11acquires the thickness of the commodity and the distance to the backmost side of the commodity line. When the thickness of the commodity and the distance are acquired, the processor11calculates a length of the commodity line (the distance between the foremost side and the backmost side of the commodities) formed by the commodities based on the depth of the shelf stage and the distance. When the length of the commodity line is acquired, the processor11calculates the number of commodities (the number of commodities forming the commodity line) based on the length of the commodity line and the thickness of the commodity. When the processor11calculates the number of commodities, the processor11stores the calculated number of commodities as an initial value of the commodities. The processor11stores the initial value of the commodities similarly at each position.

The processor11has a function of determining whether the distance measured by the distance sensor40is changed.

That is, based on the distance measured by the distance sensor40, the processor11determines whether the commodity is extracted or the commodity is added. For example, the processor11determines whether a difference between the distance measured by the distance sensor40and a moving average value or a previous measured value is greater than a predetermined threshold. When the processor11determines that the difference is greater than the predetermined threshold, the processor11determines that the distance has changed. Conversely, when the processor11determines that the difference is not greater than the predetermined threshold, the processor11determines that the distance has not changed.

The processor11has a function of determining whether a commodity indicated by the shelf label information is exhibited when the distance measured by the distance sensor40has changed.

When the distance measured by the distance sensor40has changed, the processor11calculates the length of the commodity line based on the distance after the change. The calculation of the length of the commodity line is described above.

When the processor11calculates the length of the commodity line, the processor11determines whether the thickness of the commodity indicated by the shelf label information matches the length of the commodity line. For example, the processor11determines whether the length of the commodity line is an integer multiple of the thickness of the commodity (or a difference between the length of the commodity line and the integer multiple of the thickness is equal to or less than a predetermined threshold). When the processor11determines that the length of the commodity line is the integer multiple of the thickness of the commodity, the processor11determines that the thickness of the commodity indicated by the shelf label information matches the length of the commodity line. When the processor11determines that the length of the commodity line is not the integer multiple of the thickness of the commodity, the processor11determines that the thickness of the commodity indicated by the shelf label information does not match the length of the commodity line.

When the processor11determines that the thickness of the commodity indicated by the shelf label information does not match the length of the commodity line, the processor11determines that the commodity indicated by the shelf label information is not exhibited.

FIG.4illustrates an example in which the thickness of a commodity indicated by the shelf label information does not match a length of the commodity line. In the example illustrated inFIG.4, the shelf label information is assumed to indicate the commodities A as the commodity exhibited at the position illustrated inFIG.4. As illustrated inFIG.4, the commodity B with the thickness different from the thickness of the commodity A is exhibited on the front side. In this case, the length of the commodity line is not the integer multiple of the thickness of the commodity A. Accordingly, the processor11determines that the thickness of the commodity indicated by the shelf label information does not match the length of the commodity line.

When the processor11determines that the thickness of the commodity indicated by the shelf label information matches the length of the commodity line, the processor11captures an image of the commodity on the foremost side using the camera20. When the image of the commodity on the foremost side is captured, the processor11specifies the commodity on the foremost side based on the image captured through image processing.

For example, the NVM14stores dictionary information for specifying a commodity in advance. The dictionary information may indicate a feature point, a feature amount, or the like of each commodity. The dictionary information may indicate a network for specifying a commodity. The configuration of the dictionary information is not limited to a specific configuration. The processor11specifies a commodity shown in the captured image using the dictionary information.

When the commodity on the foremost side is specified, the processor11determines that the commodity indicated by the shelf label information is identical with the specified commodity. When the processor11determines that both the commodities are identical to each other, the processor11determines that the commodity indicated by the shelf label information is exhibited. Conversely, when the processor11determines that both the commodities are not identical to each other, the processor11determines that the commodity indicated by the shelf label information is not exhibited.

The processor11has a function of counting the increased or decreased number of commodities when the processor11determines that the commodity indicated by the shelf label information is exhibited.

When the processor11determines that the commodity indicated by the shelf label information is exhibited, the processor11calculates a difference between the distance before the change and the distance after the change. When the processor11calculates the difference, the processor11calculates a value obtained by dividing the calculated difference by the thickness of the commodity as the increased or decreased number of commodities. Here, when the distance after the change is longer than the distance before the change, the processor11determines that the number of commodities on the shelf stage is decreased (the commodity is extracted). When the distance after the change is shorter than the distance before the change, the processor11determines that the number of commodities on the shelf stage is increased (the commodity is added).

When the processor11calculates the increased or decreased number of commodities, the processor11calculates the number of current commodities. For example, the processor11calculates the number of current commodities based on the initial value of the commodities and the increased or decreased number of commodities. When the processor11calculates the number of current commodities, the processor11updates the initial value of the commodities to the calculated number of commodities.

The processor11may transmit the increased or decreased number of commodities to an external device through the communication unit15or the like.

The processor11has a function of outputting an error when the processor11determines that the commodity indicated by the shelf label information is not exhibited.

Here, the processor11displays the error on the electronic shelf label30through the shelf label interface19. The processor11displays the error indicating that the exhibition of the commodity is wrong on the electronic shelf label30corresponding to a position at which the commodity indicated by the shelf label information is not exhibited. The processor11may display information indicating the commodity indicated by the shelf label information as the error.

The processor11may continue to output the error until the commodity indicated by the shelf label information is exhibited. For example, the processor11continues to display the error on the electronic shelf label30until the distance measured by the distance sensor40is identical with the distance before the change (or until the difference is equal to or less than the threshold).

The processor11may capture the image of the commodity and specify the commodity based on the image of the face of the commodity. In this case, the processor11may continuously output the error until the specified commodity is identical with the commodity indicated by the shelf label information.

FIG.5is a diagram illustrating an example of an electronic shelf label30in which an error is displayed.FIG.5is a diagram when the shelf100is viewed on the front side. In the example illustrated inFIG.5, the processor11determines that the commodity exhibited on the left is not identical with the commodity indicated by the shelf label information.

As illustrated inFIG.5, the processor11displays an error on the electronic shelf label30acorresponding to the commodity exhibited on the left.

The processor11may notify an external device of the error through the communication unit15. The processor11may display the error on the display unit17.

Next, an operation example of the management device10will be described.FIG.6is a flowchart illustrating an operation example of the management device10. Here, the user is assumed to complete the exhibition of the commodities on the shelf100.

First, the processor11sets the number of exhibited commodities as the initial value at ACT11. When the initial value is set, the processor11measures a distance between the back plate to the backmost side of the commodity line using the distance sensor40at ACT12.

When the distance is measured, the processor11determines whether the distance is changed at ACT13. When the processor11determines that the distance is changed (e.g., YES at ACT13), the processor11determines whether the thickness of the commodity matches the length of the commodity line based on the distance after the change at ACT14.

When the processor11determines that the thickness of the commodity matches the length of the commodity line (e.g., YES at ACT14), the processor11acquires the image of the exhibited commodity using the camera20at ACT15. When the image is acquired, the processor11specifies the commodity shown in the image through image processing at ACT16.

When the commodity is specified, the processor11determines whether the specified commodity is identical with the commodity indicated by the shelf label information at ACT17. When the processor11determines that the specified commodity is identical with the commodity indicated by the shelf label information (e.g., YES at ACT17), the processor11calculates the increased or decreased number of commodities based on the distance before the change, the distance after the change, the thickness of the commodity, and the like at ACT18. When the processor11calculates the increased or decreased number of commodities, the processor11updates the initial value of the commodities based on the increased or decreased number of commodities at ACT19.

When the processor11updates the initial value of the commodities, the processor11determines whether the operation ends at ACT20(e.g., determines whether or not the operation has ended). When the processor11determines that the distance is not changed (e.g., NO at ACT13) or determines whether the operation does not end (e.g., NO in ACT20), the processor11returns the process at ACT12.

When the processor11determines that the thickness of the commodity does not match the length of the commodity line (e.g., NO in ACT14) or determines that the specified commodity is not identical with the commodity indicated by the shelf label information (e.g., NO at ACT17), the processor11displays the error on the electronic shelf label30at ACT21.

When the processor11displays the error, the processor11determines whether the commodity indicated by the shelf label information is exhibited at ACT22. When the processor11determines that the commodity indicated by the shelf label information is not exhibited (e.g., NO at ACT22), the processor11returns the process at ACT22.

When the processor11determines that the commodity indicated by the shelf label information is exhibited (e.g., YES at ACT22), the processor11returns the process at ACT12. The processor11may end the display of the error.

Next, an example of the operation at ACT11in which the processor11sets the number of exhibited commodities as the initial value will be described.FIG.7is a flowchart illustrating the example of the operation at ACT11in which the processor11sets the number of exhibited commodities as an initial value.

First, the processor11acquires the shelf label information at ACT31. When the processor11acquires the shelf label information, the processor11acquires the distance between the back plate and the backmost side of the commodity line using the distance sensor40at ACT32.

When the processor11acquires the distance, the processor11calculates the number of commodities based on the thickness of the commodity indicated by the shelf label information, the acquired distance, and the like at ACT33. When the processor11calculates the number of commodities, the processor11sets the calculated number of commodities as the initial value at ACT34. When the processor11sets the initial value, the processor11ends the operation.

The processor11performs ACT11to ACT22and ACT31to ACT34at each position at which the commodity is exhibited.

The camera20may capture an image of a face of one commodity. The processor11may adjust the commodity based on the number of commodities extracted from the shelf100.

The shelf stage of the shelf100may have a structure tilted to the front side. In this case, the shelf stage is a biasing mechanism.

The shelf stage may be a stage that includes a guide supporting the commodities from the lateral side so that the commodities are arranged in a predetermined direction.

The shelf100may include a relay device connected to the camera20and the distance sensor40. In this case, the management device10is connected to the relay device.

Next, a modification example of the camera20will be described. Here, the camera20captures an image of each commodity (for example, the upper surface of each commodity) that forms the commodity line. For example, the camera20is installed facing downwards in the middle of the shelf stage.

FIG.8illustrates an example of an image (e.g., a captured image) captured by the camera20. In the example illustrated inFIG.8, the camera20captures an image of the commodity line formed by four commodities A from the upper side.

The processor11specifies a commodity line area in which the commodity line is shown in the captured image. For example, the processor11estimates a size and a position of the area in which the commodity line is shown based on the thickness of the commodity indicated by the shelf label information and the number (e.g., the initial value) of commodities. The processor11specifies the commodity line area based on the estimated area. Here, the processor11specifies a commodity line area200.

When the processor11specifies the commodity line area, the processor11specifies the commodity area in which each commodity is shown based on the number of commodities. That is, the processor11divides the commodity line area in the direction in which the commodities are arranged into the number of commodities and specifies each commodity area.

When the processor11specifies each commodity area, the processor11specifies the commodity in each commodity area. Through the operation, the processor11can determine that there is a mistake in the exhibition even when another commodity that has the same thickness is mixed in the commodity line. For example, the processor11may perform the foregoing operation after ACT19.

The management system that has the foregoing configuration measures the distance between the predetermined position and the backmost side of the commodity line. The management system calculates the increased or decreased number of commodities based on the distance before the change and the distance after the change when the distance is changed. As a result, the management system can efficiently calculate the increased or decreased number of commodities.

The management system determines whether the commodities indicated by the shelf label information are exhibited based on the distance after the change and the image of the face. As a result, the management system can determine whether the commodities are appropriately exhibited when the number of commodities is increased or decreased.