Patent Publication Number: US-10331969-B2

Title: Image processing for scale zero validation

Description:
BACKGROUND INFORMATION 
     The scale of a point of sale (POS) terminal barcode scanner has to be very responsive and accurate at the same time. One of the key considerations for a scale is maintaining a true zero weight before weighing. This true zero can be corrupted if someone presses the scale zero reset button while there is an item on the top plate. That action would create a false state of zero. 
     This can be an issue especially in self-checkout where the user can be confused when weighing multiple items. Even though the self-checkout provides verbal helps, the customer may not be sure on next steps, and language issues and the rush of shopping can add to the confusion. A consumer can inadvertently press scale zero without a clean or empty top plate. 
     Assisted lane cashiers are also faced with this problem when they are novice, or just in a hurry. The cashier may inadvertently press scale zero when there are items still on the top plate. There is also a potential for theft if the cashier is trying to remove weight for malicious reasons. For example, selling weighed items to friends or family at a lower value by first removing weight on purpose using the above process. 
     SUMMARY 
     Various embodiments herein each include at least one of systems, devices, software, and methods that leverage images captured by imaging devices of product scanners, which may include scanner-scale devices, and other imaging devices deployed at a Point of Sale (POS) terminal. 
     One such embodiment, in the form of a method, includes identifying a scale-zero error condition and determining, based on a view of a scale surface provided by at least one imaging device, whether an item is present on the scale surface. In such embodiments, when no item is present on the scale surface, the method includes providing an instruction via an output device for proceeding with a scale zeroing process. Further in such embodiments, when an item is present on the scale surface, the method includes providing an instruction via the output device to remove the item present on the scale surface and restarting the method. 
     Another method embodiment includes determining whether an item is present on a scale surface based on a view of the scale surface provided by an imaging device. In these embodiments, when an item is present on the scale surface, the method includes outputting an instruction via an output device to remove the item present on the scale surface. 
     A further embodiment is a device that includes at least one output device, an input, at least one processor, and a memory device. The memory device stores instructions executable by the at least one processor to perform data processing activities. The data processing activities may include determining whether an item is present on a scale surface based on a view of the scale surface received from the input and when an item is present on the scale surface, outputting an instruction via at least one output device to remove the item present on the scale surface. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is logical block diagram of an example checkout terminal upon which image processing for scale zero validation may be performed, according an example embodiment. 
         FIG. 2  is a block diagram of a computing device, according to an example embodiment. 
         FIG. 3  is a block flow diagram of a method according to an example embodiment. 
         FIG. 4  is a block flow diagram of a method according to an example embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     Various embodiments herein each include at least one of systems, devices, software, and methods that leverage images captured by imaging devices of product scanners, which may include barcode scanner devices, barcode scanner-scale devices, and other imaging devices deployed at a Point of Sale (POS) terminal. Such POS terminals may be cashier assisted terminals, self-service terminals (SSTs), and the like. These embodiments process these images to help customers utilizing SST POS terminals and cashiers at cashier assisted terminals to provide a true scale zero upon occurrence of a desire to zero a scale or upon detection of a scale zero fault condition for which a scale zero reset is called for. When a scale zero reset is to be performed, some embodiments include presenting a view of an image of what is on the scale along with instructions to remove all items and then to press a zero button of the scale. 
     In one such embodiment, a customer may be weighing multiple items on a SST POS terminal. If the scale is in need of an intervention and scale zero button press, such as upon detection of a scale zero fault condition, the customer can be directed to do so. However, the customer may not always understand what to do. In this case, the customer may attempt to reset the scale zero but may not know to first clear all items from the scale first. A proper zero may then not be set and customer may be frustrated and discouraged from utilizing an SST POS terminal in the future. Such embodiments operate to not only instruct the customer what to do but also to provide an image of the scale and explain exactly what to remove through one or more of graphical, textual, and audible instructions. Note that in some embodiments, the scale may be a scale of a barcode scanner/scale device, but the scale may also be a stand-alone scale. 
     In some cashier assisted POS terminal embodiment, a cashier may be pressing a zero on a scale when items are still on the top plate. By leveraging images captured by one or more imaging devices of the scale when the scale is a barcode scanner/scale or other imaging device and seeing what is on the scale, the embodiment may alert the cashier and provide an image of the items remaining on the top plate. This image-based knowledge may also prevent a user from pressing the scale zero button when any items are on the top plate if they try to proceed after the alert. 
     One or more images of the various embodiments herein may therefore be utilized to identify whether items are present on a scale, such as on a top plate of a scale, and to assist in instructing a user to remove the items. These and other embodiments are described herein with reference to the figures. 
     In the following detailed description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific embodiments in which the inventive subject matter may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice them, and it is to be understood that other embodiments may be utilized and that structural, logical, and electrical changes may be made without departing from the scope of the inventive subject matter. Such embodiments of the inventive subject matter may be referred to, individually and/or collectively, herein by the term “invention” merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed. 
     The following description is, therefore, not to be taken in a limited sense, and the scope of the inventive subject matter is defined by the appended claims. 
     The functions or algorithms described herein are implemented in hardware, software or a combination of software and hardware in one embodiment. The software comprises computer executable instructions stored on computer readable media such as memory or other type of storage devices. Further, described functions may correspond to modules, which may be software, hardware, firmware, or any combination thereof. Multiple functions are performed in one or more modules as desired, and the embodiments described are merely examples. The software is executed on a digital signal processor, ASIC, microprocessor, or other type of processor operating on a system, such as a personal computer, server, a router, or other device capable of processing data including network interconnection devices. 
     Some embodiments implement the functions in two or more specific interconnected hardware modules or devices with related control and data signals communicated between and through the modules, or as portions of an application-specific integrated circuit. Thus, the exemplary process flow is applicable to software, firmware, and hardware implementations. 
       FIG. 1  is logical block diagram of an example checkout terminal  100  upon which image processing for scale zero validation may be performed, according an example embodiment. The checkout terminal  100 , which may also be referred to as a POS terminal, is illustrated as a cashier assisted checkout terminal, although other checkout terminal types are equally relevant, such as SST POS terminals. 
     The checkout terminal  100  includes a scanner/scale  102  that has a scale zero button and one or more imaging devices (e.g., electronic cameras) that may be deployed at one or more locations  106 . The checkout terminal  100  further includes a computing device  108  and a display  110 . Some embodiments may further include one or more other cameras deployed at other locations, handheld scanning devices, audio and visual output devices, a keyboard, other input devices, and other elements. 
     In operation, the scanner/scale  102  weighs items placed on a top plate  107 . The scale of the scanner/scale  102  is a sensitive device and its weighing accuracy is typically subject to regulation. As such, the scanner/scale  102 , the computer  108 , or other computing element may monitor operation of the scanner/scale  102  to help ensure accurate weights are measured, a proper scale zero setting is maintained, when zeroed that too much weight is not zeroed off, and other possible scale error conditions. 
     To zero the scale of the scanner/scale  102 , a scale zero button  104  may need to be pressed. However, a zeroing process, as may be triggered by pressing the scale zero button  104 , may instead be triggered programmatically. 
     In operation, when a scale zero button  104  press is received or a scale zero process is triggered, such as upon detection of a scale error event, one or more of the various method illustrated and described herein may be triggered for execution on a processor of one or both of the scanner/scale  102  and the computer  108 . In such embodiments, images are captured by one or more imaging devices. The images are captured of the entirety or a portion of the top plate  107  of the scanner/scale  102 . The images are then processed to determine whether any items are present on top plate  107 . If not, the scale may be zeroed by pressing the scale zero button  104  in response to an instruction output via the display  110 , a speaker, and the like. Upon the button press or a triggering of a scale zeroing process, the scale zero is set. However, if an item is determined to be present on the top plate  107 , such as by subtracting a reference image from a captured image and considering remaining values at all or select image pixels, one or more instructions to remove the item on the top plate  107  may be output via the display  110 , a speaker, and other output devices. 
       FIG. 2  is a block diagram of a computing device, according to an example embodiment. In one embodiment, multiple such computer systems are utilized in a distributed network to implement multiple components in a transaction-based environment. An object-oriented, service-oriented, or other architecture may be used to implement such functions and communicate between the multiple systems and components. 
     One example computing device in the form of a computer  210  that may be deployed as the computer  108  of  FIG. 1 . The computer  210  may include a processing unit  202 , memory  204 , removable storage  212 , and non-removable storage  214 . Although the example computing device is illustrated and described as computer  210 , the computing device may be in different forms in different embodiments. For example, the computing device may instead be a smartphone, a tablet, smartwatch, or other computing device including the same or similar elements as illustrated and described with regard to  FIG. 2 . Devices such as smartphones, tablets, and smartwatches are generally collectively referred to as mobile devices. Further, although the various data storage elements are illustrated as part of the computer  210 , the storage may also or alternatively include cloud-based storage accessible via a network, such as the Internet. 
     Returning to the computer  210 , memory  204  may include volatile memory  206  and non-volatile memory  208 . Computer  210  may include—or have access to a computing environment that includes a variety of computer-readable media, such as volatile memory  206  and non-volatile memory  208 , removable storage  212  and non-removable storage  214 . Computer storage includes random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM) and electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD ROM), Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium capable of storing computer-readable instructions. 
     Computer  210  may include or have access to a computing environment that includes input  216 , output  218 , and a communication connection  220 . The input  216  may include one or more of a touchscreen, touchpad, mouse, keyboard, camera, one or more device-specific buttons, one or more sensors integrated within or coupled via wired or wireless data connections to the computer  210 , and other input devices. The computer  210  may operate in a networked environment using a communication connection  220  to connect to one or more remote computers, such as database servers, web servers, and other computing device. An example remote computer may include a personal computer (PC), server, router, network PC, a peer device or other common network node, or the like. The communication connection  220  may be a network interface device such as one or both of an Ethernet card and a wireless card or circuit that may be connected to a network. The network may include one or more of a Local Area Network (LAN), a Wide Area Network (WAN), the Internet, and other networks. In some embodiments, the communication connection  220  may also or alternatively include a transceiver device, such as a BLUETOOTH® device that enables the computer  210  to wirelessly receive data from and transmit data to other BLUETOOTH® devices. 
     Another example computing device is a controller of a scale or a scanner/scale, such as the scanner/scale  102  of  FIG. 1 . 
     Computer-readable instructions stored on a computer-readable medium are executable by the processing unit  202  of the computer  210 . A hard drive (magnetic disk or solid state), CD-ROM, and RAM are some examples of articles including a non-transitory computer-readable medium. For example, various computer programs  225  or apps, such as one or more applications and modules implementing one or more of the methods illustrated and described herein or an app or application that executes on a mobile device or is accessible via a web browser, may be stored on a non-transitory computer-readable medium. 
       FIG. 3  is a block flow diagram of a method  300  according to an example embodiment. The method  300  is an example of a method that may be performed by a controller of a scanner/scale, such as the scanner/scale  102  of  FIG. 1 , by the computer  108  of  FIG. 1 , or a combination of both. The method  300  includes determining  302  whether an item is present on a scale surface based on a view of the scale surface provided by an imaging device. When an item is present on the scale surface, the method  300  further includes outputting  304  an instruction via an output device to remove the item present on the scale surface. In some embodiments, the method  300  also includes restarting the method  300  following the outputting  304  of the instruction to remove the item present on the scale surface. In a further embodiment of the method  300 , when no item is present on the scale surface, the method  300  may provide an instruction via an output device for proceeding with a scale zeroing process. 
     The output device of some method  300  embodiments includes a display device. In some such embodiments, outputting  304  the instruction via the output device to remove the item present on the scale surface includes presenting, on the display device, an image captured by the imaging device. The presenting of the image captured by the imaging device, in some embodiments, first includes processing the image to be presented by subtracting a reference image captured by the imaging device from the image to be presented thereby leaving the item present on the scale surface and then presenting the processed image. 
     In a further embodiments of the method  300 , when an item is present on the scale surface, the method  300  includes processing image data of the view to identify and read a barcode present therein to obtain a product identifier and performing a product lookup based on the product identifier to retrieve product information. In some such embodiments, outputting  304  the instruction via the output device to remove the item present on the scale surface includes an output based at least in part on the retrieved product data. 
       FIG. 4  is a block flow diagram of a method  400  according to an example embodiment. The method  400  is an example of another method that may be performed by a controller of a scanner/scale, such as the scanner/scale  102  of  FIG. 1 , by the computer  108  of  FIG. 1 , or a combination of both. 
     The method  400  includes identifying  402  a scale-zero error condition (e.g., a scale or POS terminal condition requiring a scale to be calibrated/zeroed) and determining  404 , based on a view of a scale surface provided by at least one imaging device, whether an item is present on the scale surface. When no item is present on the scale surface, the method  400  includes providing  406  an instruction via an output device for proceeding with a scale zeroing process. When an item is present on the scale surface, the method  400  includes providing  408  an instruction via the output device to remove the item present on the scale surface and restarting  410  the method  400 . 
     In some such embodiments, providing  408  the instruction via the output device to remove the item present on the scale surface includes presenting an image captured by the at least one imaging device. In one such embodiment, presenting the image captured by the at least one imaging device first includes processing the image to be presented by subtracting a reference image captured by the at least one imaging device from the image to be presented thereby leaving item present on the scale surface and then presenting the processed image. 
     In some embodiments of the method  400 , providing  406  the instruction via the output device to remove the item present on the scale includes reading a barcode or other identifying data of an item present on the scale surface as included in the view provided by the at least one imaging device. The reading of the barcode is performed in some such embodiments to obtain a product identifier of the item. The output to remove the item present on the scale in such embodiments may include information with regard to the item retrieved based on the product identifier. 
     In some embodiments, the instruction provided  406  for proceeding with the scale zeroing process includes an instruction to press a zeroing button on the scale. The instruction to press the zeroing button may be provided at least in part with an image via a display device. 
     In some embodiments of the method  400 , identifying  402  the scale-zero error condition includes comparing a weight of a scanned product with a stored known weight of the scanned product to verify scale accuracy and invoking a scale-zero error condition when the accuracy cannot be verified. 
     It will be readily understood to those skilled in the art that various other changes in the details, material, and arrangements of the parts and method stages which have been described and illustrated in order to explain the nature of the inventive subject matter may be made without departing from the principles and scope of the inventive subject matter as expressed in the subjoined claims.