Patent Description:
Embodiments described herein relate generally to a reading apparatus and a method.

There has been used, in sales registration in a store, a technique for supporting commodity registration by performing processing for specifying a commodity using a reading apparatus having a function of object recognition (generic object recognition) or a processing for selecting candidates of a commodity and causing an operator to select a candidate. For example, as related art, there has been disclosed a technique concerning a commodity reading apparatus that specifies a commodity by switching code reading processing for reading a commodity code from a code symbol such as a barcode and object recognition processing and performs sales registration.

In recent years, for improvement of efficiency of registration processing or a reduction in a burden on a checker, there has been developed a reading apparatus that can perform commodity specifying or selection of a commodity candidate by object recognition even if a commodity to be registered is left placed in a basket or on a register counter. In such a reading apparatus, there has been desired a technique that can achieve optimization of switching of object recognition processing and code reading processing and achieve improvement of reading accuracy or reading efficiency. <CIT> relates to an article recognition apparatus including a first interface, a second interface, and a processor. The first interface acquires an image captured by photographing a photography range including an area where articles that are recognition targets are arranged. The second interface acquires depth information from a predetermined position relative to an area corresponding to the photography range. The processor extracts image areas of all products existing in the image by using the depth information, and determines whether an unidentified area due to overlapping of a plurality of the products exists in the image, based on the image areas of the products extracted from the image. <CIT> relates to an information processing apparatus comprising a first acquisition module which acquires an image from an image capturing device which photographs an image capturing range, a second acquisition module which acquires a measurement result from a distance measurement device which measures a distance to an object in the image capturing range, a segmentation module which segments, from the image, a region where the obj ect exists in the image according to the measurement result, an extraction module which extracts a feature amount from the region indicating the object, a calculation module which compares a feature amount of each article registered beforehand in a dictionary with the feature amount extracted to calculate the similarity degree of the object and the registered article, and a recognition module which recognizes, from the articles, a registered article the similarity degree of which is above a threshold value as a candidate for the photographed object.

<CIT> discusses a commodity reading apparatus that automatically determines whether to use barcode or object recognition based on how the commodity moves through predefined areas in the image frame. If the commodity passes through a second area before the first, object recognition is used; otherwise, barcode recognition is applied.

<CIT> describes a combination barcode imaging/decoding and real-time video capture system used at point-of-sale (POS) stations. The system includes a barcode scanner with an imager that automatically switches between barcode scanning mode and video surveillance mode. The switch happens based on the scanner's position in a cradle or by detecting an object near the scanner using a proximity sensor.

According to the present invention, it is provided a reading apparatus according to claim <NUM>, a reading method according to claim <NUM> and a point of sale (POS) terminal according to claim <NUM>.

An object of embodiments is to provide a reading apparatus and a program that can efficiently perform switching of object recognition processing and code reading processing.

A reading apparatus according to an embodiment includes a captured-image reading section, an article detecting section, an object recognizing section, and a code reading section. The captured-image reading section reads, from an imaging apparatus that captures a captured image associated with distance information indicating distances to articles, the captured image associated with the distance information. The article detecting section detects images of articles in a preset predetermined imaging region in the captured image. The object recognizing section performs, based on the distance information, object recognition processing of an object, the object being an article closest from the imaging apparatus among the articles detected in the predetermined region. The code reading section performs, based on the distance information, if a distance between the object and the imaging apparatus is equal to or smaller than a predetermined distance, code reading processing for detecting a code symbol from the predetermined imaging region and reading code information from the code symbol.

A reading apparatus and a program according to an embodiment are explained in detail below with reference to the accompanying drawings. In the embodiment explained below, an example is explained in which a scanner used while being connected to a POS (Point Of Sales) terminal used in a supermarket, a convenience store, or the like is applied as the reading apparatus. However, an application example is not limited to this example. The reading apparatus according to the embodiment may be used while being mounted on or connected to a self-checkout apparatus.

<FIG> is a block diagram illustrating an overall configuration of a POS system <NUM> according to this embodiment. The POS system <NUM> includes a POS terminal <NUM> and a scanner <NUM> connected to the POS terminal <NUM>.

The POS terminal <NUM> includes a microcomputer (a control section <NUM>) that executes information processing. The control section <NUM> is configured by connecting a CPU (Central Processing Unit) <NUM> that executes various kinds of arithmetic processing and controls sections, a ROM (Read Only Memory) <NUM>, and a RAM (Random Access Memory) <NUM> via a bus.

A drawer <NUM>, a keyboard <NUM>, a store clerk display <NUM>, a touch panel <NUM>, and a customer display <NUM> are connected to the CPU <NUM> of the POS terminal <NUM> via various input and output circuits. The CPU <NUM> controls these sections.

A HDD (Hard Disk Drive) <NUM> is connected to the CPU <NUM> of the POS terminal <NUM>. The HDD <NUM> has stored therein a commodity master file F1. The commodity master file F1 is delivered from a store computer SC at predetermined timing such as timing before store opening and stored in the HDD <NUM>.

The commodity master file F1 has stored therein, in association with a commodity code for identifying a commodity, information such as a commodity name, a unit price, and a classification of the commodity. The commodity master file F1 has stored therein, as reference data used for object recognition, a captured image (a reference image) of the commodity. The commoditymaster file F1 may have stored therein, as the reference data for the object recognition, feature values extracted from the reference image, that is, data indicating color information, a surface shape, texture of a surface, and the like of the reference image. In this way, the commodity master file F1 (commodity-information storing means) associates information related to the captured image of the commodity (the captured image or data related to the captured image) and identification information (a commodity code, a commodity name, or the like) for identifying the commodity.

A communication I/F (Interface) <NUM> for executing data communication with the store computer SC is connected to the CPU <NUM> of the POS terminal <NUM> via an input and output circuit (not illustrated in <FIG>). The store computer SC is set in a backyard or the like of a store. The commodity master file F1 delivered to the POS terminal <NUM> is stored in a HDD (not illustrated in <FIG>) of the store computer SC.

Further, a connection I/F <NUM> that enables data transmission and reception between the POS terminal <NUM> and the scanner <NUM> is connected to the CPU <NUM> of the POS terminal <NUM>. The scanner <NUM> is connected to the connection I/F <NUM>. A printer <NUM> that prints a receipt and the like is connected to the CPU <NUM> of the POS terminal <NUM>. The printer <NUM> prints transaction content of one transaction on the receipt under the control by the CPU <NUM>.

The scanner <NUM> includes a microcomputer (a control section <NUM>) to which a CPU <NUM>, a ROM <NUM>, and a RAM <NUM> are connected via a bus. The ROM <NUM> has stored therein a program PR according to this embodiment. An illuminating section <NUM>, an imaging section <NUM>, a sound output section <NUM>, and the like are connected to the CPU <NUM> via various buses and I/Fs. The CPU <NUM> controls operations of these sections. The control section <NUM> is connected to the POS terminal <NUM> via a connection I/F <NUM>.

The sound output section <NUM> is a sound circuit, a speaker, and the like for generating preset alarm sound and the like. The illuminating section <NUM> is configured by linearly arranging, for example, a plurality of LEDs (Light Emitting Diodes). The illuminating section <NUM> emits pulse light according to a light emission control signal input from the control section <NUM> and illuminates an imaging region of the imaging section <NUM>. The illuminating section <NUM> maybe configured by other light emitting elements.

The imaging section <NUM> is an imaging device including an image sensor (also referred to as imaging element) such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) and an optical element such as a lens. The imaging section <NUM> sequentially reads frame images into the RAM <NUM> at a frame rate (the number of frames read in one second) based on an imaging control signal.

In this embodiment, the imaging section <NUM> (also referred to as TOF camera) having a TOF (Time Of Flight) function is used as the imaging section <NUM>. For example, the TOF camera projects an infrared ray applied with frequency modulation from infrared LEDs arranged in an array shape and measures a phase of a modulation signal reflected from an object (an article) in a visual field to measure a distance to the article. The TOF camera is imaging means that can acquire distance information on a real time basis. That is, the imaging section <NUM> is an imaging device that captures a captured image associated with the distance information indicating the distance to the article.

The POS system <NUM> in this embodiment acquires distance information with the TOF camera. Consequently, it is possible to select an article at the top in a shopping basket, that is, an article closest from the imaging section <NUM>. In this embodiment, the distance from the imaging section <NUM> to a commodity is determined using the distance information. An object recognition mode and a code reading mode are automatically switched according to the distance between the commodity and the imaging section <NUM>.

The imaging section <NUM> sequentially outputs image data, that is, captures images associated with distance information to the control section <NUM> of the scanner <NUM>. If the POS terminal <NUM> performs respective kinds of processing, the captured images only have to be output to the control section <NUM> of the POS terminal <NUM>.

The imaging section <NUM> may include, respectively as separate sensor devices, the image sensor for acquiring a frame image and the TOF camera for acquiring a distance image or may include one sensor device having functions of both of the image sensor and the TOF camera.

An operation display section <NUM> for store clerks and a customer display <NUM> (for both of which, refer to <FIG>) are connected to the CPU <NUM> via connection I/Fs <NUM> and <NUM>. The operation display section <NUM> for store clerks and the customer display <NUM> are connected to the CPU <NUM> of the POS terminal <NUM> via the connection I/F <NUM> and the connection I/F <NUM> of the POS terminal <NUM>. The operation display section <NUM> for store clerks includes a keyboard <NUM>, a store clerk display <NUM>, and a touch panel <NUM> disposed on the store clerk display <NUM>.

The store clerk display <NUM> is configured by, for example, a liquid crystal display and has a configuration in which the touch panel <NUM> is laminated on a liquid crystal surface. As the customer display <NUM>, a liquid crystal display, a line indicator that can display a plurality of rows, and the like can be used.

<FIG> is an exterior view of the scanner <NUM>. As illustrated in <FIG>, the scanner <NUM> is set on the upper surface of a sacker table <NUM>. The sacker table <NUM> is a worktable used for placing a shopping basket, commodities, and the like on the upper surface. The sacker table <NUM> is used as a register counter. The scanner <NUM> includes the operation display section <NUM> for store clerks at the upper end of a flat supporting section <NUM>. The customer display <NUM> is provided substantially back to back with the operation display section <NUM> for store clerks on the opposite side of the operation display section <NUM> for store clerks.

The scanner <NUM> includes the imaging section <NUM> in a position on the near side when viewed from the operator facing the operation display section <NUM> and below a screen of the store clerk display <NUM>. The position (the height) and an angle of an optical axis of the imaging section <NUM> are set such that all contents of a shopping basket 3a placed on the sacker table <NUM> are included in the imaging region of the imaging section <NUM>. During operation, commodities may be directly placed on the sacker table <NUM> without using a shopping basket.

As a schematic operation method, if receiving, from a customer, the shopping basket 3a including commodities to be purchased, the operator places the shopping basket 3a on the sacker table <NUM> below the imaging section <NUM>. The scanner <NUM> recognizes the shopping basket 3a and starts object recognition processing. Concerning a target commodity of the object recognition processing such as a vegetable or a fruit, the scanner <NUM> performs the object recognition processing and specifies the commodity while keeping the commodity put in the shopping basket 3a. If detecting that the commodity is specified and taken out to the outside of the shopping basket 3a, the scanner <NUM> outputs information concerning the commodity to the POS terminal <NUM>. On the other hand, concerning a commodity other than the target commodity of the object recognition processing, the operator brings the commodity close to the imaging section <NUM> of the scanner <NUM>, holds up a barcode attached to the commodity over the imaging section <NUM>, and performs code reading. If detecting that the commodity is taken out to the outside of the shopping basket 3a after a commodity code is read, the scanner <NUM> outputs read commodity information to the POS terminal <NUM>.

In this embodiment, if an imaging target object is present within a predetermined distance L1 from the imaging section <NUM>, the scanner <NUM> automatically switches a reading mode from the object recognition mode to the code reading mode and starts code reading processing. The predetermined distance L1 can also be grasped as a predetermined height H1 decided with respect to a counter upper surface of the sacker table <NUM>. In other words, if the imaging target object reaches the predetermined code reading start height H1, the scanner <NUM> switches the reading mode from the object recognition mode to the code reading mode and starts the code reading processing.

In the object recognition mode, the scanner <NUM> captures a color image under natural light. On the other hand, in the code reading mode, the scanner <NUM> lights the illuminating section <NUM> (see <FIG>) and captures a monochrome image. Consequently, in the code reading mode, contrast of white and black of a code symbol is sufficiently increased to achieve improvement of code reading efficiency and reading accuracy.

In this embodiment, it is possible to automatically switch the reading mode at timing conforming to a series of operation of the code reading processing performed by the operator in the past for taking a commodity from the shopping basket 3a, holding up a barcode over the imaging section <NUM>, and moving the commodity after the reading to a shopping basket 3b. That is, the reading mode is switched from the object recognition mode to the code reading mode by being triggered by an act of bringing the commodity close to the imaging section <NUM> and holding up the commodity over the imaging section <NUM>. Therefore, the operator can start the code reading mode without a burden in a natural motion and can efficiently switch the object recognition processing and the code reading processing.

In <FIG>, the configuration example is shown in which the imaging section <NUM> is held in front of the supporting section <NUM>. However, a setting position of the imaging section <NUM> is not limited to this.

<FIG> is an exterior view of a scanner and the imaging section provided separately from the scanner. In <FIG>, the shopping basket 3a set as a reading target is placed in front of the scanner <NUM> and read. However, a placing position of the shopping basket 3a can be changed as appropriate. As illustrated in <FIG>, operation may be performed in which the shopping basket 3a set as the reading target, in which unregistered commodities are put, is placed on the right side of a scanner <NUM> and the shoppingbasket 3b, in which commodities after registration is put, is placed on the left side of the scanner <NUM>.

According to such an operation method, in <FIG>, an imaging section <NUM> separate from the scanner <NUM> is erected on the sacker table <NUM> on the right side of the scanner <NUM>. That is, the imaging section <NUM> is set on the upper surface of the sacker table <NUM> by a supporting section <NUM> of the imaging section <NUM>. The height and an angle of an optical axis of the imaging section <NUM> are set such that all contents of the shopping basket 3a set as a reading target are included in an imaging region. A hardware configuration of the imaging section <NUM> and connection to other components of the scanner <NUM> are not changed. As illustrated in <FIG>, the imaging section <NUM> is connected to the CPU <NUM> of the scanner <NUM> and the POS terminal <NUM> via the connection I/F <NUM>.

In the configuration illustrated in <FIG>, as in <FIG>, a position in changing the reading mode only has to be set as the predetermined distance L1 from the imaging section <NUM>. Alternatively, the code reading start height H1, which is height from the counter upper surface of the sacker table <NUM>, may be set as a determination reference for the mode switching.

It is possible to change, as appropriate according to a layout or the like of the store, on which of the left and the right of the scanner <NUM> the reading target shopping basket 3a, in which unregistered commodities are put, is placed. Oppositely from <FIG>, the supporting section <NUM> of the imaging section <NUM> may be set on the upper surface of the sacker table <NUM> on the left side of the scanner <NUM>.

The program PR (see <FIG>) executed by the scanner <NUM> in this embodiment is explained. A software configuration can be applied in both of the configurations of the scanners <NUM> and <NUM> without a difference. Therefore, in the following explanation, components are denoted by reference numerals and signs assuming implementation in the scanner <NUM>.

<FIG> is a block diagram illustrating a functional configuration realized by the CPU <NUM> of the scanner <NUM>. As illustrated in <FIG>, the program PR (see <FIG>) stored in the ROM <NUM> of the scanner <NUM> has a module configuration including a distance-image reading section <NUM>, an object selecting section <NUM>, a candidate extracting section <NUM>, a display control section <NUM>, an operation receiving section <NUM>, a code reading section <NUM>, an imaging-mode switching section <NUM>, and an information output section <NUM>. The CPU <NUM> (a processor) reads out the program PR from the ROM <NUM> and loads the sections onto a main storage device. Consequently, the sections are generated on the main storage device.

The program PR executed by the scanner <NUM> in this embodiment is incorporated in the ROM <NUM> in advance and provided. The program PR executed in the scanner <NUM> in this embodiment may be provided while being recorded in a computer-readable recording medium such as a CD-ROM, a flexible disk (FD) , a CD-R, or a DVD (Digital Versatile Disk) as a file in an installable form or an executable form.

Further, the program PR executed in the scanner <NUM> in this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded through the network. The program PR executed in the scanner <NUM> in this embodiment may be provided or distributed through the network such as the Internet.

The program PR in this embodiment may be stored in the HDD <NUM> or the like of the POS terminal <NUM>. The CPU <NUM> of the POS terminal <NUM> may execute the program PR to thereby realize functions of the sections. The functions of the sections are explained below.

The distance-image reading section <NUM> (captured-image reading means) reads a captured image (a distance image) captured by the imaging section <NUM> onto the RAM <NUM> of the scanner <NUM>.

The object selecting section <NUM> (article detecting means) detects an image of an article in a preset predetermined imaging region in the captured image. The predetermined imaging region only has to be a region of a guidance indicator <NUM> (see <FIG>) used in placing the shopping basket 3a, a region of an imaging region on the inner side of the shopping basket 3a, or the like.

The object selecting section <NUM> selects, based on the distance information, articles as an object of registration processing in ascending order of distances from the imaging section <NUM>. For example, the object selecting section <NUM> divides the distance image into images of a multilayer structure for each of the distances and extracts, based on the distance information and position information (information indicating two-dimensional positions), articles recognized as a group. The object selecting section <NUM> sets an article located in the top layer as an article closest from the imaging section <NUM> and selects the article as an object of the registration processing.

If an article is extracted (detected), processing such as edge detection may be performed to extract an article using a two-dimensional color image or monochrome image not particularly including the distance information. The distance information, the position information, and the processing such as the edge detection may be combined as appropriate. A procedure for combining the respective kinds of processing is not particularly limited.

The candidate extracting section <NUM> starts object recognition processing on an image of the object selected by the object selecting section <NUM>. That is, the object selecting section <NUM> and the candidate extracting section <NUM> function as object recognizing means for performing, based on the distance information, the object recognition processing of an object, the object being an article at the shortest distance from the imaging section <NUM> among articles detected by the object selecting section <NUM> in the predetermined imaging region.

The object recognition is a technique for identifying an exterior characteristic of a target article from captured image data and specifying a type and the like of the article. The candidate extracting section <NUM> extracts a feature value, which is data indicating an exterior characteristic, from an image region of an article selected by the object selecting section <NUM>. The candidate extracting section <NUM> collates the extracted feature value with feature value data of reference images of commodities included in the commodity master file F1 and calculates similarity. The candidate extracting section <NUM> extracts at least one commodity having high similarity as a commodity candidate.

The candidate extracting section <NUM> extracts, as a result of the object recognition processing, at least one candidate of a commodity (commodity candidate) similar to an article set as a registration target from the commodity master file F1 (see <FIG>). That is, the candidate extracting section <NUM> (candidate selecting means) selects, from the commodity master file F1, a commodity candidate corresponding to the object selected by the object selecting section <NUM>.

The display control section <NUM> respectively generates display data output to the store clerk display <NUM> and the customer display <NUM> of the scanner <NUM> and controls display processing in the displays. For example, the display control section <NUM> displays, on the store clerk display <NUM> of the scanner <NUM>, a reading screen <NUM> (see <FIG>) displayed during commodity reading processing. The display control section <NUM> (display output means) generates display data for selectably displaying, on the store clerk display <NUM>, information indicating the commodity candidate selected by the candidate extracting section <NUM> and outputs the display data to the store clerk display <NUM>. A screen configuration example is explained below.

The operation receiving section <NUM> receives operation on the keyboard <NUM> and the touch panel <NUM> of the scanner <NUM>. For example, the operation receiving section <NUM> receives operation for selecting one commodity with respect to the commodity candidates displayed on the store clerk display <NUM> by the display control section <NUM>. The operation receiving section <NUM> (operation receiving means) receives, via a display screen displayed and output by the display control section <NUM>, operation for selecting, as a sales target commodity, the commodity candidate selected by the candidate extracting section <NUM>.

The code reading section <NUM> (code reading means) starts the code reading processing based on the distance information included in the distance image if the distance between the imaging object and the imaging section <NUM> is equal to or smaller than the preset predetermined distance L1. In the code reading processing, the code reading section <NUM> detects a code symbol such as a barcode from a predetermined imaging region, decodes the code symbol, and reads code information such as a commodity code and a discount code. The predetermined imaging region only has to be the region of the guidance indicator <NUM> (see <FIG>) used in placing the shopping basket 3a, the imaging region on the inner side of the shopping basket 3a, or the like.

The imaging-mode switching section <NUM> (imaging-condition switchingmeans) switches the reading mode of the imaging section <NUM> from the object recognition mode to the code reading mode if the distance between the object selected by the object selecting section <NUM> and the imaging section <NUM> is equal to or smaller than the predetermined distance L1. In this case, the imaging-mode switching section <NUM> switches an imaging condition of the imaging section <NUM> from an imaging condition during the object recognition processing to an imaging condition during the code reading processing.

In the object recognition mode, imaging in the natural light is performed and the illuminating section <NUM> is turned off. In the object recognition mode, the distance-image reading section <NUM> reads a color image such as an RGB image from the imaging section <NUM>. In the object recognition mode, image quality of a degree for obtaining texture of a surface only has to be obtained. Therefore, since it is unnecessary to form a high-resolution image in the object recognition mode, the distance-image reading section <NUM> may capture a moving image from the imaging section <NUM>. If a still image is captured, it is unnecessary to increase shutter speed. That is, if a frame image is captured at a predetermined frame rate, the frame rate may be set rather low. Since a feature value of an image can be extracted even if noise is slightly superimposed on an image, a gain of the imaging element only has to be increased even in a state without illumination to obtain a captured image in a state in which sensitivity is increased.

On the other hand, in the code reading mode, an edge of a black portion has to be detected with respect to a code symbol such as a barcode or a two-dimensional code. Therefore, in the code reading mode, imaging using illumination is performed and the illuminating section <NUM> is lit. In the code reading mode, the distance-image reading section <NUM> reads a monochrome image with the imaging section <NUM>. If the gain of the imaging element is increased, noise is easily superimposed because the sensitivity increases. Therefore, in the code reading mode, the gain of the imaging element is reduced to achieve a reduction in noise to make it easy to detect the edge. In the code reading mode, the distance-image reading section <NUM> captures a still image with the imaging section <NUM>. It is desirable to increase the shutter speed such that an edge of a code symbol image does not blur.

In the above explanation, the illuminating section <NUM> is turned off in the object recognition mode. However, the illuminating section <NUM> may be lit at illuminance lower than illuminance in the code reading mode. In other words, the imaging-mode switching section <NUM> increases illuminance of illumination for imaging in the code reading processing.

If the object selected by the object selectin section <NUM> is on the outside of the predetermined imaging region, the information output section <NUM> (information output means) outputs the code information read by the code reading section <NUM> to the POS terminal <NUM> as information related to the sales target. The predetermined imaging region only has to be the region of the guidance indicator <NUM> (see <FIG>) used in placing the shopping basket 3a, the imaging region on the inner side of the shopping basket 3a, or the like.

If the distance between the object selected by the object selecting section <NUM> and the imaging section <NUM> is not equal to or smaller than the predetermined distance L1 and the object is on the outside of the predetermined imaging region (e.g., outside of the image region of the shopping basket 3a), the information output section <NUM> outputs information indicating the commodity candidate, the selection operation for which is received by the operation receiving section <NUM>, to the POS terminal <NUM> as information related to the sales target.

An output destination of the information related to the sales target is not limited to the POS terminal <NUM>. The information related to the sales target may be output to another apparatus according to a system to which the scanner <NUM> is applied.

The CPU <NUM> (see <FIG>) of the POS terminal <NUM> functions as a registration means for receiving commodity information output by the information output section <NUM> of the scanner <NUM> and performing information registration of an object based on the commodity information. The information registration is not limited to sales registration of a commodity. The CPU <NUM> may perform processing for reading information necessary for various kinds of information processing from an article such as a discount coupon or a card indicating necessity and unnecessity of a register bag or the like, an application to fund raising, and the like and registering the information. In this case, the CPU <NUM> may register member information read from a member card in association with the information.

A configuration example and a transition example of a display screen are explained with reference to <FIG>.

<FIG> is a diagram illustrating a configuration example of the reading screen <NUM>. A captured image <NUM> of the imaging section <NUM> is displayed on the reading screen <NUM> on a real time basis. The display control section <NUM> displays, in a display region of the captured image <NUM>, the guidance indicator (a guideline) <NUM> indicating a position where the shopping basket 3a should be placed. The guidance indicator <NUM> is clearly seen if the guidance indicator <NUM> is displayed as, for example, a colored frame line. As an example, the guidance indicator <NUM> is desirably displayed as a yellow rectangular frame line.

By displaying the guidance indicator <NUM> on the screen in this way, the shopping basket 3a can be placed to fit within the imaging region of the imaging section <NUM>. If commodities are directly placed on the sacker table <NUM> without using the shopping basket 3a, the commodities can be placed to fit within the imaging region. The object recognition processing and the code reading processing are executed on an article in the region of the guidance indicator <NUM>. Commodity information read by one of the object recognition processing and the code reading processing is output to the POS terminal <NUM> if it is detected that the object is on the outside of the region of the guidance indicator <NUM>. The image region of the shopping basket 3a may be automatically recognized without displaying the guidance indicator <NUM>. The object recognition processing and the code reading processing may be performed on a commodity in the shopping basket 3a. If it is detected that the commodity is on the outside of the shopping basket 3a, commodity information may be output to the POS terminal <NUM>.

The display control section <NUM> displays, on the reading screen <NUM>, a registration target article selected by the object selecting section <NUM> in a state in which the registration target article is distinguishable on a captured image. For example, the display control section <NUM> attaches a first identification indicator <NUM> to a commodity A set as a registration target and displays the first identification indicator <NUM>. As an example, the first identification indicator <NUM> is desirably formed by a colored frame line (e.g., a red frame) or the like generally surrounding an image region of the registration target article. Since there are a variety of shapes of commodities, the frame line does not have to be formed in a rectangular shape in particular and may be formed in a circular shape or an elliptical shape or may be changed as appropriate according to the shapes of the commodities. Since the first identification indicator <NUM> is attached to an article that is not specified as a commodity yet, the frame line may be displayed as a dotted line or the like or may be flashed and displayed. Alternatively, the first identification indicator <NUM> may be formed by a mark rather than the frame line. A text or an illustration may be included in the mark.

Concerning at least one commodity candidate extracted by the candidate extracting section <NUM>, the display control section <NUM> reads, for example, a commodity name, an illustration of a commodity, a photograph image, or the like from the commodity master file F1 as information indicating the commodity candidate and displays the commodity name, the illustration of a commodity, the photograph image, or the like on the store clerk display <NUM>. On the reading screen <NUM>, a display region on the right of the captured image <NUM> is used as a display region for displaying the commodity candidate. In <FIG>, an example is shown in which two commodity candidates 74a and 74b are extracted as commodity candidates similar to the commodity A. The display control section <NUM> arranges and displays illustrations corresponding to the commodity candidates 74a and 74b in descending order of similarities. If only one commodity candidate having high similarity is extracted, an illustration of the one commodity candidate is displayed.

<FIG> is a diagram illustrating a screen configuration example in which one commodity is selected out of commodity candidates. If the operation receiving section <NUM> receives selection operation for the commodity candidate 74a, the display control section <NUM> attaches, instead of the first identification indicator <NUM>, a second identification indicator <NUM> different from the first identification indicator <NUM> to the commodity A selected as a registration target and displays the second identification indicator <NUM>. As an example, the first identification indicator <NUM> is displayed as a red frame. If one commodity candidate is selected, the display control section <NUM> desirably displays the second identification indicator <NUM> as a blue frame. Alternatively, as the second identification indicator <NUM>, the inside of a frame may be hatched and a specified article may be displayed to be grayed out. Alternatively, a mark "finished" may be displayed as the second identification indicator <NUM> to clearly indicate that specifying processing is finished.

If one commodity candidate is selected, the display control section <NUM> leaves only an illustration of the selected commodity candidate 74a and grays out an illustration of the unselected commodity candidate 74b (see <FIG>).

If one commodity candidate is selected, the display control section <NUM> reads, from the commodity master file F1, commodity information associated with the selected commodity candidate and displays the commodity information in a read information display region <NUM> together with the number of commodities. The display control section <NUM> displays, in the customer display <NUM>, information equivalent to content displayed in the read information display region <NUM>.

As illustrated in <FIG>, a PLU (Price Look Up) button <NUM>, a code input button <NUM>, a re-recognition button <NUM>, a cancellation button <NUM>, and the like are provided on the reading screen <NUM>. The PLU button <NUM> is an operator for inquiring the commodity master file F1 and starting processing for selecting a commodity based on a commodity classification, a commodity name, and the like. The code input button <NUM> is an operator for inputting a commodity code via a tenkey and the like of the keyboard <NUM> and starting processing for reading commodity information of a pertinent commodity from the commodity master file F1. The re-recognitionbutton <NUM> is an operator for causing the CPU <NUM> to execute the object recognition again, for example, if an appropriate commodity candidate is not selected. The cancellation button <NUM> is an operator for, after one commodity candidate is selected and commodity information is displayed in the read information display region <NUM>, cancelling registration of the commodity.

If the commodity A surrounded by the second identification indicator <NUM> (e.g., a blue frame), a commodity name of the commodity A being specified, is taken out from the shopping basket 3a, the object selecting section <NUM> detects that the commodity A frames out. The display control section <NUM> updates the reading screen <NUM> and erases an operator of the commodity candidate 74a, information displayed in the read information display region <NUM>, and the like. The information output section <NUM> outputs the commodity information read from the commodity master file F1 such as the commodity name, the number of commodities, and the price displayed in the read information display region <NUM> to the POS terminal <NUM>. The control section <NUM> of the POS terminal <NUM> displays the received commodity information on a registration screen for commodity registration.

The object selecting section <NUM> selects, based on a captured image (a distance image) and distance information after the frame-out of the article, an article located in the top layer as the next registration processing target article. The display control section <NUM> attaches the first identification indicator <NUM> (e.g., a red frame) to the article selected by the object selecting section <NUM> and displays the first identification indicator <NUM>. The candidate extracting section <NUM> starts the object recognition processing on an image of the article selected by the object selecting section <NUM> and extracts a commodity candidate.

In this way, in this embodiment, the reading of the commodity automatically shifts to reading of the next commodity while being triggered by take-out of the read commodity from the shopping basket 3a (i.e., the imaging region of the guidance indicator <NUM>).

A screen example is explained in which the operator lifts a commodity attached with a barcode from a shopping basket, brings the commodity close to the imaging section <NUM>, and performs the code reading processing.

<FIG> is a diagram illustrating a screen example in which a commodity attached with a barcode is held up and the code reading processing is started. In the example illustrated in <FIG>, if a commodity B is lifted and brought close to the imaging section <NUM>, the commodity B is the closest from the imaging section <NUM>. Therefore, the object selecting section <NUM> selects, based on distance information of a distance image, the commodity B located in the top layer as a registration processing object. The display control section <NUM> attaches the first identification indicator <NUM> (e.g., a red frame) to the commodity B selected by the object selecting section <NUM> and displays the first identification indicator <NUM>. The candidate extracting section <NUM> starts the object recognition processing on an image of the object selected by the object selecting section <NUM>.

If the commodity B is held up higher, the distance from the imaging section <NUM> is equal to or smaller than the predetermined distance L1, and the height from the counter upper surface of the sacker table <NUM> is equal to or larger than H1, the imaging-mode switching section <NUM> switches the reading mode from the object recognition mode to the code reading mode. The candidate extracting section <NUM> suspends the object recognition processing. The code reading section <NUM> starts the code reading processing, decodes an image <NUM> of a barcode included in a captured image, and reads a commodity code. The code reading section <NUM> reads commodity information corresponding to the read commodity code from the commodity master file F1. The display control section <NUM> displays, in the read information display region <NUM>, the commodity information read from the commodity master file F1. As the commodity information, as illustrated in <FIG>, information such as the read commodity code, a commodity name, and a unit price read from the commodity master file F1 can be applied.

If the distance between the commodity B and the imaging section <NUM> is equal to or smaller than the predetermined distance L1 and the processing for the object is switched from the object recognition processing to the code reading processing, a third identification indicator (e.g., a green frame) different from the first identification indicator <NUM> may be added to the first identification indicator <NUM> (e.g., a red frame) to indicate that the processing is switched. As the third identification indicator, a message such as "code is being read", a mark of a barcode, or the like may be flashed and displayed to inform the operator that the code reading processing is being performed.

If, after the reading of the commodity information, that is, after the commodity information is displayed and output to the read information display region <NUM>, the commodity B is taken out from the shopping basket 3a and it is detected that an image of the commodity B is on the outside of the region of the guidance indicator <NUM>, the information output section <NUM> outputs the commodity information of the commodity to the POS terminal <NUM>. The POS terminal <NUM> displays the received commodity information on the store clerk display <NUM> and the customer display <NUM> as information concerning a sales target commodity and registers the information as sales data.

If the distance between the commodity B and the imaging section <NUM> is equal to or smaller than the predetermined distance L1 once but, thereafter, a commodity code is not read and the distance between the commodity B and the imaging section <NUM> is larger than the predetermined distance L1, the imaging-mode switching section <NUM> return the reading mode to the object recognition mode. In this way, the scanner <NUM> in this embodiment can easily switch the reading method for the commodity B according to the distance to the imaging section <NUM>.

In general, in the object recognition, the recognition processing can be sufficiently performed if a captured image captured under the natural light is used. Conversely, if illumination is used, illumination unevenness occurs and a recognition rate easily decreases. If illumination is use in the object recognition, a type and a setting place of an illumination instrument are sometimes different during machine mounting. A way of irradiation of the illumination on a commodity is likely to change for each apparatus. Therefore, if an image is captured under the illumination by the illumination instrument, learning of the object recognition has to be performed for each apparatus. Learning cost increases. In this way, in the object recognition, a recognition rate is more easily improved if an image is captured using the natural light.

On the other hand, in the code reading processing, in the captured image, a code reading rate can be improved if edge contrast of a code symbol is clearer. That is, it is easier to improve code reading efficiency if the illumination tool is used than when imaging is performed under the natural light.

In this way, preferred illuminating methods are different in the object recognition processing and the code reading processing. Therefore, there has been a demand for improving reading efficiency of both of the object recognition processing and the code reading processing with a simpler method.

On the other hand, in this embodiment, if the distance between the object of the object recognition processing and the imaging section <NUM> is equal to or smaller than the predetermined distance L1 (see <FIG>), the processing for the object is switched from the object recognition processing to the code reading processing. At the same time, the imaging condition (the imagingmode) of the imaging apparatus is switched. Consequently, if an act of holding up the commodity attached with the barcode over a reading window is performed as in the past, since the distance between the commodity and the imaging section <NUM> decreases, the imaging mode is switched to the imaging mode suitable for the code reading processing and the barcode can be imaged. Therefore, even in a reading apparatus that can perform the object recognition processing while keeping an object put in the basket, it is possible to automatically switch the imaging condition and achieve improvement of reading accuracy or reading efficiency of a code symbol.

A procedure example of reading processing executed by the scanner <NUM> is explained. <FIG> is a flowchart illustrating the procedure example of the reading processing executed by the scanner <NUM>.

The distance-image reading section <NUM> starts capturing of a captured image (a distance image) by the imaging section <NUM> (Act <NUM>). The imaging-mode switching section <NUM> sets the imaging section <NUM> in the object recognition mode and starts imaging at a predetermined frame rate (Act <NUM>). In the region of the guidance indicator <NUM>, the object selecting section <NUM> selects, as an article closest from the imaging section <NUM>, an article located in the top layer based on distance information (Act <NUM>).

The candidate extracting section <NUM> performs the object recognition processing on an image of the article selected in Act <NUM> and extracts commodity candidates (Act <NUM>). The display control section <NUM> displays illustrations (icons) indicating the commodity candidates extracted in Act <NUM> on the reading screen <NUM> (Act <NUM>). The operation receiving section <NUM> determines whether selection operation for any one commodity candidate is received (Act <NUM>). If receiving the selection operation (Yes in Act <NUM>), the display control section <NUM> reads commodity information of the selected commodity candidate from the commodity master file F1 and displays the commodity information in the read information display region <NUM>.

The object selecting section <NUM> determine whether an object of the object recognition processing is taken out from the shopping basket 3a and is on the outside of the region of the guidance indicator <NUM> (see <FIG>) (Act <NUM>). If it is determined No in Act <NUM>, the object selecting section <NUM> waits in Act <NUM>. If it is determined Yes in Act <NUM>, the information output section <NUM> outputs the commodity information read from the commodity master file F1 to the POS terminal <NUM> (Act <NUM>). Thereafter, the processing shifts to Act <NUM>.

If it is determined No in Act <NUM>, the object selecting section <NUM> determines whether the distance from the imaging section <NUM> to an image of the object of the object recognition, that is, an image of an article closest from the imaging section <NUM> is equal to or smaller than the predetermined distance L1 and the object is lifted to height equal to or larger than the code reading start height H1 (Act <NUM>). If it is determined No in Act <NUM>, the processing returns to Act <NUM>. If it is determined Yes in Act <NUM>, the imaging-mode switching section <NUM> switches the imaging mode of the imaging section <NUM> from the object recognition mode to the code reading mode and lights the illuminating section <NUM> (Act <NUM>). The code reading section <NUM> starts the code reading processing and extracts an image region of a barcode from a captured image. If the image region of the barcode is extracted, the code reading section <NUM> decodes the barcode, reads a commodity code, and reads commodity information corresponding to the commodity code from the commodity master file F1. The display control section <NUM> displays the read commodity information in the read information display region <NUM>.

If the code reading is not performed (No in Act <NUM>), the processing returns to Act <NUM>. After the code reading (Yes in Act <NUM>), the processing shifts to Act <NUM>. The object selecting section <NUM> determines whether the object of the code reading processing is taken out from the shopping basket 3a and is on the outside of the region of the guidance indicator <NUM> (Act <NUM>). If it is determined No in Act <NUM>, the object selecting section <NUM> waits in Act <NUM>. If it is determined Yes in Act <NUM>, the information output section <NUM> outputs the commodity information read from the commodity master file F1 to the POS terminal <NUM> (Act <NUM>). The processing shifts to Act <NUM>.

If the operation receiving section <NUM> receives key operation of a reading end button (not illustrated in <FIG>) or the like in the keyboard <NUM> and receives an instruction for a reading end (Yes in Act <NUM>), the distance-image reading section <NUM> ends the capturing of the captured image (Act <NUM>) and outputs an imaging OFF signal for turning off the imaging section <NUM> to the imaging section <NUM>. The imaging section <NUM> ends the imaging operation and, if the illuminating section <NUM> is lit, extinguishes the illuminating section <NUM>. If it is determined No in Act <NUM>, the processing returns to Act <NUM> and the object recognition processing is performed.

In the above explanation, it is determined that the commodity is take out from the shopping basket 3a if the object is on the outside of the region of the guidance indicator <NUM>. However, a method of determining whether the commodity is taken out from the shopping basket 3a is not limited to this. As another example, the shopping basket 3a may be identified by image recognition. The commodity information may be output to the POS terminal <NUM> while being triggered by the object being on the outside of the region of the image recognized as the shopping basket 3a.

In determining that the commodity is taken out from the shopping basket 3a, it may be determined whether the object is on the outside of the image region of the guidance indicator <NUM> in the lateral direction or may also be determined whether the object is on the outside of the image region in the height direction (i.e., the distance from the imaging section <NUM>). In this case, for example, the height of the shopping basket 3a only has to be set in the scanner <NUM>. If the commodity is in a height position higher than the height of the shopping basket 3a and is on the outside of the region of the shopping basket 3a in the lateral direction, it only has to be determined that the commodity is taken out from the shopping basket 3a. If the commodity is picked up to a position higher than the height of the shopping basket 3a and thereafter to a position higher than the code reading start height H1 (see <FIG> and <FIG>), the code reading processing only has to be started as explained above.

Responses to the re-recognition button <NUM>, the PLU button <NUM>, and the code input button <NUM> (for all of the buttons, see <FIG>) are interruptible at anytime. That is, if the operation receiving section <NUM> receives operation of the re-recognition button <NUM>, the processing proceeds to Act <NUM>. The candidate extracting section <NUM> selects an article closest from the imaging section <NUM> again (Act <NUM>), executes the object recognition processing, and retries the commodity candidate extraction (Act <NUM>). A procedure in Act <NUM> and subsequent acts is the same as the procedure explained above.

On the other hand, if the PLU button <NUM> is operated, the information output section <NUM> starts PLU processing for selecting a commodity from the commodity master file F1. Concerning the PLU processing, a technique generally used in the past only has to be applied. The information output section <NUM> receives selection of a commodity via a commodity selection screen or the like hierarchically configured according to commodity classifications. The information output section <NUM> reads commodity information of the selected commodity from the commodity master file F1. The processing shifts to Act <NUM>.

If the code input button <NUM> is operated, the information output section <NUM> starts the code reception processing for directly receiving a commodity code from the tenkey and the like. Concerning the code reception processing, a technique generally use in the past only has to be applied. That is, if a commodity code is input via the tenkey and the like of the keyboard <NUM>, the information output section <NUM> reads commodity information corresponding to the commodity code from the commodity master file F1. The processing shifts to Act <NUM>.

Claim 1:
A reading apparatus, comprising:
a captured-image reader (<NUM>) configured to read, from an imaging apparatus that captures a captured image associated with distance information indicating distances to articles, the captured image associated with the distance information;
an article detecting section (<NUM>) configured to detect images of articles in a preset predetermined imaging region in the captured image;
an object recognizing section (<NUM>) configured to perform, based on the distance information, object recognition processing of an object, the object being an article closest from the imaging apparatus among the articles detected in the predetermined region;
a code reader (<NUM>) configured to perform, based on the distance information, if a distance between the object and the imaging apparatus is equal to or smaller than a predetermined distance, code reading processing for detecting a code symbol from the predetermined imaging region and reading code information from the code symbol; and
an imaging-condition switching section (<NUM>) configured to switch from the object recognition processing to the code reading processing if the distance between the object and the imaging apparatus is equal to or smaller than the predetermined distance.