Abstract:
The subject matter described herein includes methods, systems, and computer readable media for tracking consumer interactions with products using electromagnetic beam sensors. One system includes at least one transmitter configured to generate a beam of electromagnetic energy. The system further includes at least one receiver positioned with respect to the at least one transmitter to detect a portion of the beam of electromagnetic energy reflected from a consumer or a product when the consumer physically interacts with the product. The system further includes the at least one circuit board on which the at least one transmitter and the at least one receiver are mounted. The at least one circuit board is configured to mount at or near a product location.

Description:
PRIORITY CLAIM 
       [0001]    This application claims the benefit of U.S. Provisional Patent Application No. 61/748,352, filed Jan. 2, 2013, the disclosure of which is incorporated herein by reference in its entirety. This application is a continuation-in-part of U.S. patent application Ser. No. 13/422,736, filed Mar. 16, 2012, which claims the benefit of U.S. Provisional Patent Application No. 61/453,942, filed Mar. 17, 2011, the disclosures of which are incorporated by reference. 
     
    
     TECHNICAL FIELD 
       [0002]    This disclosure relates to systems for tracking consumer interactions with retail products on merchandising fixtures in real time. 
       BACKGROUND 
       [0003]    Consumer packaged goods (CPG) retailers and manufacturers are under continuous pressure to improve operational efficiencies, given intense competition, smaller profit margins and ever-increasing operational costs. Manufacturers and retailers work with a fixed amount of retail space and seek to maximize the productivity of that space in order to maximize revenues while controlling costs. Retailers seek to maximize overall revenues through optimal placement of top-performing existing store keeping units (SKUs), proper placement of new SKUs and elimination of less productive SKUs. This usually places a high demand on available shelf space and creates significant competition for shelf space in CPG retail stores. Both retailers and manufacturers commit significant data gathering and analysis to optimizing the productivity of that space. 
         [0004]    Currently, CPG manufacturers and retailers use various product tracking techniques, including collecting scanner data from the systems of the retailers, and data consolidated by vendors such as Nielsen® or Information Resources, Inc. (IRI®). There are two limitations to using just scanner data, namely, (1) aggregation and (2) time. In terms of aggregation, scanner data is usually available at store level or at a national chain level (e.g., Kroger®, and Safeway®). In terms of time, data is typically available daily, weekly, or monthly. Because of these two limitations, testing of new products or merchandising arrangements must be done across a large number of stores and over a long period of time (e.g., weeks or months) to accurately detect the effect of the change. These limitations incur significant costs (e.g., typically testing of a new product or new merchandising arrangement requires 20+ stores over 2-3 months for any given “cell” in a test) and limit the number of experimental cells that can be executed. Hence, many valuable experiments that could be conducted are precluded by cost considerations, slowing the overall learning process by retailers and manufacturers and resulting in inefficient use of space. 
         [0005]    Therefore, a need exists to address the problems noted above and other problems previously experienced. 
       SUMMARY 
       [0006]    The subject matter described herein includes methods, systems, and computer readable media for tracking consumer interactions with products using electromagnetic beam sensors. One system includes at least one transmitter configured to generate a beam of electromagnetic energy. The system further includes at least one receiver positioned with respect to the at least one transmitter to detect a portion of the beam of electromagnetic energy reflected from a consumer or a product when the consumer physically interacts with the product. The system further includes the at least one circuit board on which the at least one transmitter and the at least one receiver are mounted. The at least one circuit board is configured to mount at or near a product location. 
         [0007]    The subject matter described herein can be implemented in software in combination with hardware and/or firmware. For example, the subject matter described herein can be implemented in software executed by a processor. In one exemplary implementation, the subject matter described herein can be implemented using a non-transitory computer readable medium having stored thereon computer executable instructions that when executed by the processor of a computer control the computer to perform steps. Exemplary computer readable media suitable for implementing the subject matter described herein include non-transitory computer-readable media, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits. In addition, a computer readable medium that implements the subject matter described herein may be located on a single device or computing platform or may be distributed across multiple devices or computing platforms. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0008]    Preferred embodiments of the subject matter herein will now be explained with reference to the accompanying drawings of which: 
           [0009]      FIG. 1A  is a schematic diagram illustrating detection of an object by a diffuse beam of infrared energy according to an embodiment of the subject matter described herein; 
           [0010]      FIG. 1B  is a schematic diagram illustrating detection of an object by infrared triangulation according to an embodiment of the subject matter described herein; 
           [0011]      FIG. 2A  is a schematic diagram illustrating a vertical infrared curtain in front of a merchandising unit, where the curtain is produced by an array of electromagnetic beam sensors according to an embodiment of the subject matter described herein; 
           [0012]      FIG. 2B  is a schematic diagram illustrating single versus multiple sensors per inventory zone according to an embodiment of the subject matter described herein; 
           [0013]      FIG. 2C  is a schematic diagram illustrating sliding sensors adjustable to the center of each inventory zone according to an embodiment of the subject matter described herein; 
           [0014]      FIG. 2D  is a schematic diagram illustrating options to project beams from the bottom and/or top of an inventory zone using electromagnetic beam sensors according to an embodiment of the subject matter described herein; 
           [0015]      FIG. 2E  is a schematic diagram illustrating electromagnetic beam sensors mounted on a flexible circuit board according to an embodiment of the subject matter described herein; 
           [0016]      FIG. 2F  is a schematic diagram illustrating a flexible circuit strip mounted on top of a shelf lip according to an embodiment of the subject matter described herein; 
           [0017]      FIG. 2G  is a schematic diagram illustrating a flexible circuit strip mounted below a shelf according to an embodiment of the subject matter described herein; 
           [0018]      FIG. 2H  is a schematic diagram illustrating a flexible circuit strip allowing projection and detection of infrared energy in the plane of the strip according to an embodiment of the subject matter described herein; 
           [0019]      FIG. 2I  is a schematic diagram illustrating a flexible circuit strip installed inside a shelf lip according to an embodiment of the subject matter described herein; 
           [0020]      FIG. 3  is a schematic diagram illustrating electromagnetic beam sensors installed on shelf dividers according to an embodiment of the subject matter described herein; 
           [0021]      FIG. 4  is a schematic diagram illustrating electromagnetic beam sensors installed above and behind each inventory zone according to an embodiment of the subject matter described herein; 
           [0022]      FIGS. 5A and 5B  are schematic diagrams illustrating electromagnetic beam sensors detecting removal of planar objects according to an embodiment of the subject matter described herein; 
           [0023]      FIGS. 6A-6C  are schematic diagrams illustrating the use of electromagnetic beam sensors to distinguish between removal of planar objects at different distances from the sensors according to an embodiment of the subject matter described herein; 
           [0024]      FIGS. 7A and 7B  are schematic diagrams illustrating arrangements of electromagnetic beam sensors for detecting removal of planar objects at different distances according to an embodiment of the subject matter described herein; 
           [0025]      FIG. 8  is a schematic diagram illustrating the combining of measurements from electromagnetic beam sensors with measurements from weight sensors according to an embodiment of the subject matter described herein; 
           [0026]      FIGS. 9A-9C  are schematic diagrams illustrating an electromagnetic beam sensor mounted near a peg hook to detect insertion and removal of a product from the peg hook according to an embodiment of the subject matter described herein; 
           [0027]      FIGS. 10A-10C  are schematic diagrams illustrating an electromagnetic beam sensor positioned in a product cell array to detect insertion and removal from the cell array according to an embodiment of the subject matter described herein; 
           [0028]      FIGS. 11A-11F  are schematic diagrams illustrating sensor placement at or near a gravity feed or pusher track to detect product insertion and removal from the gravity feed or pusher track according to an embodiment of the subject matter described herein; 
           [0029]      FIGS. 12A and 12B  are schematic diagrams illustrating electromagnetic beam sensors positioned inside of a drawer to detect product insertion and removal from the drawer according to an embodiment of the subject matter described herein; 
           [0030]      FIGS. 13A and 13B  are schematic diagrams illustrating the positioning of electromagnetic beam sensors near a cash register drawer to detect opening and closing of the drawer according to an embodiment of the subject matter described herein; 
           [0031]      FIGS. 14A and 14B  are schematic diagrams illustrating the positioning of electromagnetic beam sensors in the motion path of a credit card reader according to an embodiment of the subject matter described herein; 
           [0032]      FIGS. 15A-15C  illustrate electromagnetic beam sensors positioned near a receipt printer according to an embodiment of the subject matter described herein; and 
           [0033]      FIG. 16  is a flow chart illustrating an exemplary process to tracing user interactions with products using electromagnetic beam sensors according to an embodiment of the subject matter described herein. 
       
    
    
     DETAILED DESCRIPTION 
       [0034]    This disclosure relates to systems for tracking consumer interactions with retail products on merchandising fixtures in real time. 
         [0035]    Described herein are novel applications of infrared and other types of sensors to study consumer interactions that produce the following benefits: versatile to track interactions with a broad range of package types and merchandising configuration; detection of items that do not have significant weight (e.g. financial services leaflets, phone cards, etc. . . . ); produce consistent and reproducible readings; accurately detect real interactions and avoid false positives induced by either people or store environment; do not require adulteration of product (for example as with RFID sensing); small form factor hardware allowing for rapid and discrete installation; high reliability and extended service life; and low cost equipment. 
         [0036]    Also, described herein is an example consumer tracking system (CTS), such as an on-shelf tracking (OST) system, using reflected energy sensors to monitor shopper interactions with product in an inventory zone. In these arrangements the reflection of energy on a shopper&#39;s hand, arm, or product item may be detected. Direct contact with the sensor may not be required, promoting extended service life. While the description below refers to infrared energy, it should be noted that any form of energy capable of being reflected and detected (for example ultrasonic) may be used. 
         [0037]    Also, the CTS can track consumer activity with respect to individual retail product units from a retail carton, while the retail carton may be positioned on an array of sensors mounted on or in close proximity to a retail display shelf of a merchandising fixture. The CTS&#39;s various product sensors provide a way to determine when a consumer interacts with a retail product unit positioned on a retail shelf, a description of the interaction, and stores information about the interaction as an event in an event log for later retrieval and analysis. The retail product unit may be the smallest increment of retail product offered by a retailer for purchase (e.g., a single pack of Wrigley® Five Rain gum, 15 sticks). The retail carton, also referred to as a product container, may contain retail product units in a standard number and configuration (e.g., 10 ct box of Wrigley® Five Rain gum). Multiple retail cartons containing the same product may be grouped together into an inventory zone. The merchandising fixture may display retail products to consumers in an appealing fashion. Example merchandising fixtures include a checkout shelf, a peg hook, an aisle shelf, and a temporary cardboard display. 
         [0038]    Infrared reflectance sensors may be used to monitor shopper interactions with products in an inventory zone. In these arrangements the reflection of infrared energy on the shoppers hand, arm, or product item may be detected. In these arrangements, there is no direct contact between the sensor and either product of the shopper, thus promoting extended service life. 
         [0039]    The system can be better understood with reference to the following drawings and description. The elements in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the system. In the figures, like-referenced numerals designate corresponding parts throughout the different views. 
         [0040]      FIG. 1A  illustrates detection of an object by a diffuse beam of infrared energy. A transmitter  101  creates a diffuse beam of infrared energy  102 , which illuminates any foreign object  103  entering the inventory zone. A portion of the infrared energy is reflected as a reflected beam  104  which is then detected by a receiver  105 . A suitable device may include the Vishay TCRT5000 reflective sensor which includes both a transmitter and detector. The distance of the object  103  from transmitter  101  may be estimated from the strength of the reflected signal, closer objects producing a stronger reflection and vice versa. 
         [0041]      FIG. 1B  illustrates detection of an object by infrared triangulation. A transmitter  106  produces a narrow beam  107  at an angle  108  less than 90 degrees. In the case of a nearby object  109  crossing the path of the beam, energy is reflected to a detection array  110 , illuminating a portion of the detection array  111 . In the case of a more distant object  112 , energy is reflected to a different portion of the detection array  113 . The distance of the detected object from the transmitter can be accurately estimated by the position at which the detection array is illuminated. A suitable device for this purpose may include the Sharp GP2D120XJ00F, which includes both a transmitter and a detection array in a single device. 
         [0042]    In some embodiments reflectance sensors may be mounted immediately in front of the product so as to create a partial or full curtain. 
         [0043]    For example,  FIG. 2A  shows sensors projecting a detection zone vertically. An array of sensors  201  may illuminate the area in front of and above a number of product inventory zones  202 . When an object  203  enters the illuminated area, energy is reflected to the nearest of receivers  201 , allowing identification of which item the shopper interacted with. Either reflectance or triangulation sensors may be used for this purpose. 
         [0044]    In some embodiments it may be desirable to have more than one sensor per inventory zone, for example, in situations with inventory zones broader than the operative range of one sensor, or to provide redundancy. 
         [0045]      FIG. 2B  shows two different arrangements where single sensors  204  may be assigned to inventory zones and also where multiple sensors  205  may be assigned to each inventory zone. 
         [0046]      FIG. 2C  shows an arrangement where sensors on moveable mounts  206  may be repositioned to align with the center of each inventory zone  207 . 
         [0047]      FIG. 2D  shows two possible arrangements for sensors. Sensors may be mounted at the bottom of the detection zone  208  offering the advantage of closest possible proximity to product and strongest signal changes. Alternatively sensors may be mounted at the top of the detection zone  209  having the advantage of discreteness, making sensors less prone to dust collection and other fouling. Either arrangement  208  or  209  may be used individually or both arrangements may be used at the same time. 
         [0048]      FIG. 2E  shows an arrangement where sensors may be mounted on a flexible circuit strip. Reflectance sensors  210  may be mounted in fixed positions at regular intervals on a flexible strip  211 . The strip may be a millimeter or less in thickness offering the advantage of very low form factor. The strip may be backed with adhesive  212  to allow simple mounting onto a shelf. Each strip may have traces  213  of a conductive material transmitting power to the sensors and signals from the sensors to an electromagnetic signal processor. Further each strip may be cut to length at any location  214  between sensors to fit any desired shelf width. Such strips may be mounted in at least 2 different configurations: 
         [0049]      FIG. 2F  shows a flexible circuit strip mounted on top of a shelf lip. A flexible circuit strip  215  may be mounted on top of a shelf lip  216 . The strip  215  may be connected to an electromagnetic signal processor  217 . 
         [0050]      FIG. 2G  shows a flexible circuit strip mounted below a shelf. A flexible circuit strip  218  may be mounted below a shelf  219 . Sensors project a detection zone downwards so as to detect shopper activity on the shelf immediately below shelf  220 . 
         [0051]      FIG. 2H  shows a further arrangement where sensors may be mounted on a flexible circuit strip with their beams oriented parallel to the plane of the strip. Reflectance sensors  221  may be mounted in fixed positions at regular intervals on a flexible strip  222 . Sensor legs may be bent at an angle  223  so as to allow projection and detection of infra-red energy parallel to the plane of the strip. Sensors may be adhesive backed  224 . 
         [0052]      FIG. 2I  shows a sensor strip in this arrangement  225  installed inside a shelf lip  226  between the shelf lip and product containers  227 . 
         [0053]    In some embodiments, such as those illustrated by  FIG. 3 , sensors may be mounted on shelf dividers at an angle to the vertical, creating overlapping detection zones with two or more sensors covering a specific inventory zone. Reflectance sensors  301  may be mounted on shelf dividers  302  so as to project and detect energy at an angle to the vertical. Shelf dividers  302  serve to separate inventory zones containing different products  303  and also secure the sensors  301  in a fixed position and angle relative to the product in order to ensure repeatable measurements. Since any pickup will occur in the field of view of at least two sensors, the exact location of a pickup may be pinpointed by comparing the relative strength of signals reflected to each sensor. 
         [0054]    Alternatively, sensors may be mounted behind product to create a detection zone above the inventory zone as shown in  FIG. 4 . Reflectance sensors  401  may be mounted above and behind a number of product inventory zones  402 . This mounting position confers discreteness as sensors are out of normal shopper field of view and also robustness as sensors are unlikely to be touched by shoppers. 
         [0055]      FIG. 5A  shows an arrangement where reflectance sensors are used to detect the removal of planar objects. One or more sensors  501  may be mounted at the opening of an inventory zone  502  containing planar objects  503  such as magazines, leaflets, phone cards or any other planar item. Sensor  501  is oriented such that its beam  504  crosses the path of the stored objects  504 . In  FIG. 5B , when one of the objects  505  is removed from inventory zone  502 , the infrared beam is reflected 506, allowing detection of activity. 
         [0056]      FIG. 6A  shows an arrangement where reflectance sensors are used to detect and the removal of planar objects from multiple banks of storage locations and also identify the storage location. One or more sensors  601  may be mounted at the opening of several inventory zones  602 ,  603  and  604 , positioned one in front of the other and containing different products  605 ,  606  and  607 . When any of these items are removed from their inventory zones, the item will traverse the infrared beam. 
         [0057]    In  FIG. 6B , the most distant item  605  from the sensor is being removed, producing a reflected beam  608 . In  FIG. 6C , the closest item  607  to the sensor is being removed, producing a different reflected beam  609 . The intensity of the measured reflection  609  will be consistently greater than that of reflection  608 , so allowing accurate identification of the storage location from which the item was removed. 
         [0058]    In some embodiments, sensors may be mounted laterally to a shelf to measure the distance of a pickup from one side of shelf. This arrangement has the advantage of monitoring a relatively large number of facings with a small number of sensors and minimal wiring, thus simplifying the networking and processing electronics. 
         [0059]    For example,  FIG. 7A  shows an arrangement where a single triangulation sensor  701  is mounted at the side of a shelf  702  with the beam  703  projecting across the front of a set of inventory zones  704 . On entry of an object  705  such as a shopper&#39;s hand into an inventory zone, the sensor  701  may identify the entry of the object and also measure the distance  706  of the object from the sensor. By processing the measured distance, the inventory zone from which product was handled may be identified. 
         [0060]      FIG. 7B  shows a further arrangement where triangulation sensors  707  and  708  are used to produce two independent measurements  709  and  710  of the point of an object  705  entering a set of inventory zones. By analyzing the distance readings  709  and  710 , greater accuracy and/or redundancy can be achieved in identifying the position of object  705 , as well as an estimate of the size of the object since opposite edges of object  705  are measured. 
         [0061]    In some embodiments, the sensors may be combined with an ambient lighting correction system to isolate the light being reflected from infrared transmitters. Infrared detectors will typically respond to some portion of ambient lighting and this could create false positive events, for example, by passers by moving in front of unit. By taking readings alternating with emitter subsequently on and off, it may be possible to correct for such events. 
         [0062]    In some embodiments, the sensors may be combined with a weight sensing system. 
         [0063]    For example,  FIG. 8  shows reflectance sensors  801  mounted at an angle to the vertical on dividers  802 . The same dividers also support weight sensors  803  which measure the weight of boxes of product  804  in real time. By combining these signals it is possible to further confirm the nature of shopper interactions, for example, whether items were picked up, put back, rearranged, restocked, and/or fully removed. The further use of weight data also reduces possibility of false positives as noted above from ambient lighting. By adding the optical system to a weight based system it is possible to overcome sensitivity to vibrations inherent in weight sensing systems. The net result is a system suitable for high vibration environments and also with variations in ambient lighting. 
         [0064]    In some embodiments, sensors may be installed to survey the area in front of a peg hook. 
         [0065]    For example,  FIG. 9A  shows an item of product  901  suspended on a peg hook  902 .  FIG. 9B  shows a reflectance sensor  903  mounted above and in front of the peg hook.  FIG. 9C  shows item  901  being removed from peg hook  902  and reflecting the beam from sensor  903 , so allowing detection of the item&#39;s removal. 
         [0066]    In some embodiments, sensors may be installed to measure removal of items such as bottles that may be typically stored in a cell array. 
         [0067]    For example,  FIG. 10A  shows an item  1001  stored in a cell array  1002 . In  FIG. 10B  an infrared sensor  1003  may be installed within the cell array  1002 —when items are present the beam is reflected.  FIG. 10   c  shows item  1001  being removed from the cell array—as the bottom of item is removed; the beam from sensor  1003  will no longer be reflected, allowing detection of the removal. 
         [0068]    In some embodiments, sensors may be installed to survey the area in front of items (for example, bottles) that may be mounted on a track where product items are pushed forward to the front of the shelf by gravity or by spring loading. 
         [0069]    For example, in  FIG. 11A , a reflectance sensor  1101  is installed in front of a push-forward track  1102  on which items  1103 ,  1104  and  1105  are mounted. In  FIG. 11B , item  1103  is being removed from the track, breaking the beam of sensor  1101 . As item  1103  is removed, items  1104 ,  1105  and any subsequent items are advanced on track  1102  either by gravity or spring loading. 
         [0070]    In  FIG. 11 , shown is example alternative reflectance sensor  1106  that is mounted in front of and above track  1102  so that its beam is normally clear. In  FIG. 11D , when an object  1103  is removed from the track, the sensor beam is reflected allowing detection of the activity. 
         [0071]    In  FIG. 11E  an alternative reflectance sensor  1107  is mounted inside track  1102  so that its beam is normally reflected. In  FIG. 11F  when an object  1103  is removed from the track, the sensor beam is temporarily clear, producing a more distant reflection on object  1104 , so allowing detection of the activity. Note that this particular sensor positioning could also be achieved with the sensor positioned on the base of the track so as to reflect off the bottom of container  1103 . This could also be achieved with a micro switch. 
         [0072]    In some embodiments, sensors may be installed to detect items that may be merchandised in drawers. In  FIG. 12A  sensor  1201  may be installed inside a drawer  1202  housing multiple items of product  1203  and  1204  and secured inside a merchandising unit  1205 . When the drawer is closed, the beam is reflected. In  FIG. 12B , drawer  1202  has been opened to allow removal of product  1204 . The reflection of the beam from sensor  1204  is more distant and magnitude of reflection is reduced, allowing detection of the drawer opening. By measuring the reflection magnitude, the distance to which the drawer was opened may be computed, which in some cases can be used to identify what position product was removed from the drawer. 
         [0073]    In some embodiments, infrared sensors may be installed to gather information on other shopper activities in store. 
         [0074]    For example, in the arrangement shown in  FIG. 13A  reflectance sensors  1301  and  1302  may be positioned such that their beams cross the path of a cash drawer  1303  on a cash register  1304 . When the drawer is closed, the beams are not reflected. In  FIG. 13B  case drawer  1303  is open, obscuring the beams of sensor  1301  and  1302  and allowing detection of a transaction event. Either sensor position  1302  and/or  1301  may be used. 
         [0075]    In the arrangement shown in  FIG. 14A  reflectance sensors  1401  and  1402  may be positioned such that their beams cross the motion path of credit card reader  1401 . In  FIG. 14B , credit cards  1404  and  1405  have been introduced into the credit card reader, obscuring the beams of sensors  1401  and  1402  and allowing detection of a transaction event. 
         [0076]    In the arrangement shown in  FIG. 15A  reflectance sensor  1501  may be positioned such that its beam crosses the motion path of a paper strip  1502  in a receipt printer  1503 . In  FIG. 15B , when paper strip  1502  advances, the sensor beam is reflected, allowing detection of a transaction event. In  FIG. 15C , multiple sensors  1504  and  1503  are positioned with beams crossing the paper path. In the example shown, sufficient paper has advanced to obscure sensors  1504  and  1505  but not  1506 . By comparing which sensors are obscured vs. not, it is possible to measure the amount of paper removed which will give some indication of size of transaction. 
         [0077]      FIG. 16  is a flow chart illustrating user interactions with products using electromagnetic beam sensors according to an embodiment of the subject matter described herein. Referring to  FIG. 16 , in step  1600 , at least one transmitter for projecting a beam of electromagnetic energy is positioned at or near a product location. In step  1602 , At least one receiver is positioned with respect to the at least one transmitter to detect a portion of the beam reflected when a consumer interacts with a product and to produce a corresponding output signal. In step  1604 , the signal produced by the at least one receiver is processed to identify a type of consumer interaction with the product. 
         [0078]    The above-disclosed subject matter is to be considered illustrative, and not restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other embodiments, which fall within the true spirit and scope of the description. Thus, to the maximum extent allowed by law, the scope is to be determined by the broadest permissible interpretation of the following claims and their equivalents, and shall not be restricted or limited by the foregoing detailed description.