Object identification using barcode reader

A barcode reader for verifying a barcode is attached to an appropriate object particularly suitable for use at a point of sale. A database of expected object signatures in a vicinity of a properly affixed barcode properly is maintained. At a point of sale, the barcode reader obtains an image of a presented barcode (possibly not the correct barcode) and at least a portion of an object to which the presented barcode is affixed. Using data encoded on the presented barcode, the database is accessed to provide an expected signature of the object in the region of the presented barcode. A comparison is made between the expected signature of the object and a sensed signature derived from the image of the object. An improper barcode can thus possibly be identified and further investigation initiated.

TECHNICAL FIELD

The present disclosure relates to an object identifier and more particularly, an object identifier using a Barcode Reader.

BACKGROUND

Point of sale barcode readers may include a camera that captures a digital or pixilated image of the barcode. Such a camera has a pixel array made up of photosensitive elements such as a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) device. The barcode reader also typically includes an illumination system having light emitting diodes (LEDs) or a cold cathode fluorescent lamp (CCFL) that directs illumination toward a target object, to which a target barcode is affixed. Light reflected from the target barcode is focused through a lens such that focused light is concentrated onto the pixel array of photosensitive elements. The pixels of the array are sequentially read, generating an analog signal representative of a captured image frame. The analog signal is amplified by a gain factor and the amplified analog signal is digitized by an analog-to-digital converter and stored. Decoding circuitry and/or software of the barcode reader processes the digitized signals and decodes the imaged barcode.

SUMMARY

The present disclosure addresses the problem of fraudulent substitution of barcodes by customers. An image processing method and apparatus is used based on the capabilities of an existing image based barcode reader or scanner. The solution is applicable to imaging barcode scanners including imager-based bioptic scanners.

An exemplary method uses visual object features that are extracted from an item or object to which the barcode is affixed at the time of scanning a barcode. These features (which in combination make up a signature) are extracted by the barcode scanner from an area surrounding the barcode and used to verify that barcode is attached to a correct object.

An exemplary process maintains a database of object signatures expected to be found in a vicinity of barcode properly affixed to a variety of objects. When presented at the point of sale, an image is captured of a presented barcode and at least a portion of an object to which the presented barcode is affixed. Using the data encoded on the presented barcode, information in the database is accessed and used to determine the expected signature of the object in the region of the presented barcode. A comparison is made between the expected signature of the object with a sensed object signature derived from the object presented for purchase. A mismatch in the two signatures is a good indication that tampering has occurred so the store employee is alerted that steps should be taken to confirm the accuracy of the attempted purchase.

DETAILED DESCRIPTION

FIG. 1depicts a portable point of sale barcode reader50capable of capturing an image of a target object within the reader imaging field of view FV. The concepts disclosed herein have equal applicability to a stationary or fixed imaging based barcode reader such as one where products are moved or scanned past a window so that imaging optics behind the window can form an image of a barcode within the reader's field of view.

The reader50includes illumination and imaging optics that form the field of view FV for imaging a target object.FIG. 2is a schematic depiction of reader components including a memory52and imaging circuitry60that acquires and stores captured images62from the field of view. The reader decodes 1D or 2D barcodes affixed to an object40within the field of view. However, the reader50could image and/or read other indicia such as signatures codes, softbar code, finger prints and the like. In one embodiment the reader50transmits the information contained in the barcode64for evaluation by a point of sale computer66coupled to the reader50.

The process of encoding a 2D barcode is described in detail in U.S. Pat. No. 5,243,655 to Wang which issued Sep. 7, 1993 and which is incorporated herein by reference for all purposes. The '655 patent describes the PDF417 barcode specification and describes how data is encoded into this type of 2D barcode.

A store or retail establishment may include multiple portable or stationary point of sale barcode readers (FIG. 3) all coupled through an in store network to a store server68within a store69. The portable reader50is shown inFIG. 1. has a housing having a head70, a handle72, and an optional trigger74. Located in the housing is a protective window for protecting an imaging subsystem or scan engine78.

The scan engine78projects an aiming pattern toward a target barcode64(or barcodes) on the object40and attempts to decode that barcode. The scan engine78comprises a chassis that supports a printed circuit board (not shown). Attached to the printed circuit board are several optical components that include, illumination optics110, aiming optics for generating the aiming pattern, and imaging optics or camera112. Each of the optical components have a designed field-of-view for projecting or receiving light directed during operation. The imaging optics112includes focusing lens or lenses114that focus the reflected image from the object40onto a sensor array116located behind the focusing lens(es). A visible aiming pattern is generated by a laser diode and facilitates a user centering the barcode64within the captured image.

When enabled by a controller60(FIG. 2), the imaging optics112captures an image frame of a field of view FV of the reader50. When imaging a target barcode64, the imaging process may need to capture and store in the memory a series of image frames62(FIG. 2) in response to multiple user actuations of the trigger. A decoding system120analyzes each image frame of the series of image frames62and attempts to decode the imaged barcode. All or portions of the images may be stored in a the memory52.

The barcode reader circuitry is electrically coupled to a power supply, which may be in the form of an on-board battery or a connected off-board power supply. If powered by an on-board battery, the reader10may be a stand-alone, portable unit as depicted inFIG. 1. If powered by an off-board power supply, the reader10may have some or all of the reader's functionality provided by a connected host computer66. Circuitry associated with the imaging and decoding systems60,120may be embodied in hardware, software, firmware, electrical circuitry, or any combination thereof and may be disposed within, partially within, or external to a reader housing. The reader50also includes a display122for the display of text, a speaker for conveying audible indications and one or more output LEDs for simple visual indications such as an indication of a valid barcode decode.

The sensor array116may comprise a charged coupled device (CCD), a complementary metal oxide semiconductor (CMOS), or other imaging pixel array, operating under the control of the controller60. In one exemplary embodiment, the pixel array116comprises a two dimensional (2D) mega pixel array with a typical size of the pixel array being on the order of 1280×1024 pixels.

During an imaging session, multiple images of the field of view FV may be obtained by the imaging system10. An imaging session may be instituted by an operator, for example, pressing the trigger74to institute an imaging. Alternately, for a stationary imaging system, an imaging session might start when a lower or bottom edge of an item begin to move through a portion of the field of view FV. After an exposure period, some or all of the pixels of pixel array116are successively read out by the controller60, thereby generating an analog signal scaled by a gain factor which is converted by an analog to digital converter that forms part of the controller60. The digitized signal comprises a sequence of digital gray scale values typically ranging from 0-255 (for an eight bit processor, i.e., 28=256), where a 0 gray scale value would represent an absence of any reflected light received by a pixel (characterized as low pixel brightness) and a 255 gray scale value would represent a very intense level of reflected light received by a pixel during an integration period (characterized as high pixel brightness). In an alternate embodiment, the barcode reader50includes an array which captures and interprets color images.

Barcode Signature

One problem encountered by retailers that use barcode readers at their checkout or point of sales stations is instances of customers intentionally placing an incorrect barcode label on a item presented at the checkout or point of sale. If this fraud is successful, the customer pays a lower price than the original intended price. For example, one can present an expensive vacuum cleaner for purchase, but replace the original barcode with the barcode pulled from a much cheaper stores item. This may cost the store hundreds of dollars on a single transaction.

One technique stores use to mitigate the problem is to attach a scale to the barcode reader. After the item is scanned, it is placed in a bin to determine whether the scanned object has a proper weight. This method requires additional hardware and can only be applied to small items, and even when a proper system is in place it may not always work. A barcode mismatch will not be detected, however, if the cheaper item and the more expensive item have the same weight. Such problems are advantageously overcome through the novel features of the present disclosure.

The exemplary barcode reader50captures230(see flow chart ofFIG. 5) an image and detects235the borders or bounding box64aof the barcode64. The reader50has a field of view large enough to capture and interpret a region of interest around the barcode. A Region Surrounding a Barcode210(herein RSBC) contains information relating to the object or item to which the barcode is affixed. The RSBC may be fixed (e.g. the area of the barcode scaled three times, may depend on the item, or may be adaptive (e.g. 9 times the area of the barcode, but obtained in such a way that it has maximal information content. Since this flexibility is part of a control program executing in the decoding system120, it can be reprogrammed based upon the intended use. The exemplary system uses the barcode bounding box64a, and the actual image of the decoded barcode to adjust the image of the RSBC in order to eliminate perspective distortion, curvature, or apply illumination adjustment.

Once a barcode is captured and decoded240the RSBC image is adjusted250and stored in the memory52. The software of the decoding system120then determines260a set of graphical features of the RSBC210(excluding the barcode64). These features may include, but do not have to be limited to: colors or grey level values (i.e. background or foreground, lines or text color), edges265(FIG. 2), corners, line segments; moments, Fourier or wavelet coefficients of the original or a gradient image based on derivatives within the image, Zernike moments, principal components of the original or an edge image and or any combination thereof. In one embodiment, multiple features are compared individually, and different thresholds applied, and the signature will only be accepted if there is a strong similarity in all features.

In a bi-optic imaging scanner, multiple cameras are present that register images of an object from various viewpoints in order to decode a bar code that may be present on any of the objects different surfaces. Use of such a scanner allows more than one object view to be used to construct and then verify the bar code signature and one or more features contained within those views.

For recognition purposes the system uses different methods for different features or feature sets, such as correlation, Euclidean distance, k-nearest neighbors, Hidden Markov Models, support vector machines and other statistical pattern recognition processes.

This evaluation is modified by adjustments to the software that implements the recognition to incorporate additional information and in an alternate embodiment includes color information from the RSBC210. The collection of graphic features obtained from the RSBC210is referred to herein as a Barcode Signature.

The barcode content (determined at step240above) is then used to obtain270a Model or Reference Signature from a database280. The barcode signature retrieved from the database280is compared290to the barcode signature derived from the object presented for purchase in order to verify whether a barcode is attached to the object is proper. If an object has multiple barcodes attached to it, multiple Reference Signatures associated with the barcodes are retrieved from the database. Each time an object is scanned; the Reference Signature surrounding each barcode is compared with all areas present on the object.

The Reference Signature is constructed and adjusted as items are scanned following rules of statistical learning and stored it in the database. The Reference Signature or model may be built at the pixel level, in a similar way that is done for tracking applications, when we first create the background model, or at the feature level, in a same way that is done for face or fingerprint recognition. Once the model is created, each time the object is scanned; the area of interest and/or associated features is extracted and compared with the model. If the similarity between the stored image and scanned or captured image is low, an alarm or alert300(audible or visual) is conveyed to the store employee so that the cashier/employee can check to determine if a proper item is being scanned. In an alternative embodiment, if the similarity is low the reader will not register an item for purchase. If the similarity in the Signature is high, the model is updated310. Reference Signature verification may be effectively done at a store server68that maintains the database280, but can also be done by a point of sale computer66dedicated to the scanner50. In that case a Reference Signature may be uploaded to a scanner from the server68.

The scanner50can also store Signatures of most often scanned barcodes within its memory52to speed the confirmation. Reference signatures can be built using data from several scanners, several stores of a given retailer or even across all stores having other servers132in an industry by means of a network130. The Reference Signature definition can be uploaded to individual computers on a regular basis.

While the present disclosure has been described with a degree of particularity, it is understood that the invention is defined by the accompanying claims and it is the intent that the invention include all alternatives differing from the exemplary embodiment falling within the spirit or scope of the appended claims.