Method of theft detection and prevention

A method of theft prevention and detection includes detecting movement of warehoused product by a person using one or more cameras; obtaining a first thermal image of a body of the person, with the one or more cameras, while the person is handling or moving the warehoused product; storing a physical size or shape of the warehoused product; obtaining a second thermal image of the body of the person, with the one or more cameras, subsequent to the obtaining of the first thermal image; determining thermal intensity differences by comparing the first thermal image of the body of the person to the second thermal image of the body of the person; detecting potential theft when the thermal intensity differences correlate to the physical size or shape of the warehoused product; and informing the person that the potential theft has been detected.

BACKGROUND

Field of the Invention

The present invention relates to detection and prevention of product theft in a brick-and-mortar store.

Background of the Invention

Theft is the cause of millions of dollar of loss each year for many retailers. Theft prevention technologies that detect theft are usually focused on or associated with a customer or shopper leaving a store. Such technologies usually rely on exit gates to detect electromagnetic signals indicating a theft is happening. When caught, a thief is usually embarrassed or starts to run away leading to an awkward moment for the store and the thief. An automated theft deterrent system does not currently exist which detects theft at or near the initial thieving of a product, before the thief attempts to leave a store.

SUMMARY

A method of theft prevention and detection includes detecting movement of warehoused product by a person using one or more cameras; obtaining a first thermal image of a body of the person, with the one or more cameras, while the person is handling or moving the warehoused product; storing a physical size or shape of the warehoused product; obtaining a second thermal image of the body of the person, with the one or more cameras, subsequent to the obtaining of the first thermal image; determining thermal intensity differences by comparing the first thermal image of the body of the person to the second thermal image of the body of the person; detecting potential theft when the thermal intensity differences correlate to the physical size or shape of the warehoused product; and informing the person that the potential theft has been detected.

The one or more cameras may include still image cameras, video cameras, thermographic cameras, infrared cameras, forward looking infrared cameras, or a combination thereof. A thermal marker may be attached to the warehoused product. A defined camera location or a defined pixel grouping of an optical sensor in one of the one or more cameras may be used to determine a location of the person. Detecting movement may be based, in part, on movement of the thermal marker attached to the warehoused product. One or more cameras may identify a type or kind of the warehoused product being moved. A type or kind of warehoused product may be identified by a product barcode attached to the warehoused product. A thermal marker attached to the warehoused product may be a thermal barcode. A thermal barcode may comprise machine readable thermal markings identifiable by a thermal camera, a standard camera, or a standard barcode reader. Storing of the physical size or shape of the warehoused product may be a result of the identifying the type or kind of the warehoused product being moved. Storing of the physical size or shape of the warehoused product may be a result of storing a digitized image of the warehoused product as the warehoused product is being moved. Obtaining a second thermal image of the body of the person may be between 2 seconds and 30 seconds after the first thermal image is obtained. Obtaining a second thermal image of the body of the person may be between 2 seconds and 30 seconds after movement of the warehoused product. A second thermal image may be selectively chosen from a plurality of thermal images taken subsequent to the first thermal image. A location of the person may be reported to a device of an employee who is associated with the warehoused product. Informing the person may be accomplished by a physical notification attached to a product transportation device. Informing the person may be accomplished by an employee, associated with the warehoused product, speaking to the person. Informing the person may be accomplished by sending a text message to a mobile device of the person. A second thermal image may be selectively chosen based on a position of the warehoused product intersecting, overlaying, or covering a portion of a thermal image of the person's body. A first thermal image and A second thermal image may be obtained by comparing frames of digital thermal images of the person's body until an intensity difference of the first thermal image of the person's body and the second thermal image of the person's body is obtained.

DETAILED DESCRIPTION

FIG. 1shows a perspective view100of a person holding a warehoused product in accordance with an embodiment of the invention. One or more store cameras106may track a customer/person102and determine that the customer/person102has removed a warehoused product104from an isle shelf108of a store or warehouse. One or more cameras106may comprise one or more of: still image cameras, video cameras, thermographic cameras, infrared cameras, forward looking infrared cameras, motion sensors, or a combination thereof. One or more or the one or more cameras may employ thermal imaging in addition to poly chromatic images/video to track the customer and any items removed from a shelf. One or more thermal imaging cameras106may detect and discriminate a product's thermal image104(shown in light gray) from a customer's thermal image102(shown in dark grey). Temperature differences between objects in a field-of-view of a thermographic camera appear with different colors or shades which directly correlate to the temperatures of the objects. Items with similar temperatures have similar colors or shading. A person,102, shown has a relatively uniform temperature profile shown by the uniform grey coloring. Warehoused products104/110, have a relatively uniform temperature profile but are at a different temperature (ambient) compared to customer/person102at or near body temperature. As person/customer102handles or moves warehoused product104, one or more thermal cameras106may take one or more thermal images or may start taking thermal video frames. A movement of product104may be detected by one or more cameras106and trigger the one or more cameras to start taking thermal images or frames of a body of person102to obtain a first thermal image. One or more cameras106may be used as a motion sensor/detector or a motion sensor may be co-located with one or more cameras106and trigger thermal image capture of a body of person102.

FIG. 2shows a perspective view200of a person pocketing a warehoused product in accordance with an embodiment of the invention. One or more cameras206which may include one or more camera systems, one or more computers, one or more wireless transmitters, one or more wireless receivers, one or more networks, may be used to detect a customer/person with pocketed product204. Product204may include one or more products removed from a shelf or display208. The one or more cameras206may employ thermal imaging in addition to poly chromatic images to track the customer and any items removed from a shelf. One or more cameras106may comprise one or more of: still image cameras, video cameras, thermographic cameras, infrared cameras, forward looking infrared cameras, motion sensors, or a combination thereof. One or more or the one or more cameras may employ thermal imaging in addition to poly chromatic images/video to track the customer and any items removed from a shelf. One or more thermal imaging cameras206may detect and discriminate a product's thermal image204(shown in light gray) from a customer's thermal image202(shown in dark grey). Temperature differences between objects in a field-of-view of a thermographic camera appear with different colors or shades which directly correlate to the temperatures of the objects. Items with similar temperatures have similar colors or shading. A person,202, shown has a relatively uniform temperature profile shown by the uniform grey coloring except for the thermal image of the product204shown in light grey. Warehoused product204, has a relatively uniform temperature profile but is at a different temperature (ambient) compared to customer/person202being at or near body temperature. A difference of two thermal images may reveal a product item that has been recently pocketed. A first thermal image of a body of person202shown inFIG. 1as102may be taken as a product is handled or moved. At a subsequent time, a second thermal image is taken and compared to the first thermal image revealing a difference in intensity (shown inFIG. 9). A first thermal image is important because a product item brought in the store by the customer, such as a cold soda, may have been in their pocked before a pocketed item was placed in the same pocket. A comparison of a first thermal image to a second thermal image is also important to detect a potential theft. Obtaining a first and second thermal image at or near a time of pocketing a product is also important because of possible temperature equalization resulting in an undetectable thermal profile difference between products pocketed and a temperature of a person's body. The first and second thermal images may be taken within 30 seconds of a movement of a product item. As person/customer202handles or moves warehoused product204, one or more thermal cameras206may take one or more thermal images or may start taking thermal video frames. A movement of product204may be detected by one or more cameras206and trigger the one or more cameras to start taking thermal images or frames of a body of person202in order to obtain a first thermal image. A second thermal image may be obtained by comparing a first thermal image to video frames or images taken after or subsequent to the first thermal images and detecting images intensity differences about, overlaid, or superimposed onto a body of a person handling the product. If an intensity difference if found, the two images may be chosen as a first and second thermal image. Both the first and second thermal images may be taken within a 30 second interval of each other. One or more cameras206may be used as a motion sensor/detector or a motion sensor may be co-located with one or more cameras206and trigger thermal image capture of a body of person202. An image of the customer with a customer location superimposed in the image or connected to the image may be sent to an employee's mobile device by means of wired or wireless communications. The image may be sent by text message, by email, by instant messaging, or directly through a custom program application. The machine vision may track customer's movements in real-time while a customer moves inside of a store. The camera system206may use a defined camera location or a defined pixel grouping to determine a customer location identifier. One or more cameras206may be used and a camera may be assigned to a particular area, isle or region within a store. If only one camera is used, pixel groupings of a camera's optical sensor may be related to isles, regions, and/or areas of a store. Cameras may use thermal imaging in addition to optical image recognition. Machine vision may also identify the one or more items for purchase using barcodes, lettering, or dimensional constraints of a product. If a customer's location changes, an updated image may be sent to the employee with an updated location attached to the image. An employee may deliver a product transportation device, container, cart or basket to a customer's current location and ask the customer to “please use a product transportation container instead of your clothing to transport product to be purchased” as a method of preventing theft. A customer may be notified of an arrival of the product transportation device either by the employee, a text message, a display screen on the product transportation device, or by an audio store loud speaker broadcast.

FIG. 3shows a diagram depicting devices and methods of theft detection and prevention in accordance with an embodiment of the invention. System300may include machine vision system302, one or more computers or processors304, a cloud based wide area network306, databases308including database servers, customer devices312, and employee devices310which all may communicate wirelessly and/or by wire. Machine vision system302may include one or more cameras used to detect customers of a brick-and-mortar store carrying one or more items beneath clothing, as described earlier. The machine vision may use a defined camera location or a defined pixel grouping to determine a customer location identifier. One or more cameras may be used and a camera may be assigned to a particular area, isle or region within a store. If only one camera is used, pixel groupings of a camera's optical sensor may be related to isles, regions, and/or areas of a store. Cameras may use thermal imaging in addition to optical image recognition. The machine vision may also identify the one or more items for purchase using a barcode, lettering, or dimensional constraints. One or more computers302may be connected wirelessly or by wire to each camera of vision system302. One or more cameras may employ thermal imaging in addition to poly chromatic images to track the customer and any items removed from a shelf. Thermal imaging cameras may detect and discriminate a products thermal image from a thermal image of a person attempting to steal the product, as described in relation toFIGS. 1 and 2. Computers304may provide processing and data storage allowing tracking and identification of customers and items being carried by the customers. Computers304may detect customers carrying items by comparing thermal image data of customers without hidden items to thermal image data of customers carrying items hidden under clothing. Alternatively, or additionally, products may be detected by cameras302and computers304that are removed from off of shelves in order to trigger a first and second thermal image of a person's body to be to be taken. If the products being tracked by vision system302are never placed under a person's clothing, a thermal intensity of an image of the product will not change and no potential theft will be triggered. This may be verified as an employee receives an image at an employee device and confirms whether the customer is hiding a product under clothing. Computers304may form a local area network within a company with multiple locations with a shared database. Wide area network306may include the Internet, cloud based servers and databases, which may be local or remote from a store using vision system302. Computers304may transmit and receive data over a local area network or a wide area network. Digital image processing and tracking may be performed on a cloud based server or on local computers. An electronic image of the customer may taken by the machine vision system302/304looking at or handling specific items to be purchased. The image obtained is overlaid with a location of the customer using computers304or cloud processing306and the image is sent to an employee device310. The employee may determine, by looking at the picture/thermal image, that the customer has placed product beneath clothing. An employee may leave a shopping cart in a location close to the customer and an electronic device312connected to the shopping cart may identify the customer and invite them to place the items in the basket. The electronic device may audibly and/or visibly invite the customer to use the provided cart. A location of the shopping cart, with the attached electronic customer device, may be associated with a customer profile by machine vision system302and/or computers304. A global positioning system within customer device312may be used to report location information to network304or network306in order to associate a shopping cart with a customer profile. The customer, when presented a shopping cart, may take the shopping cart and decide to remove items under clothing the customer originally planned on thieving. The electronic customer device312on the shopping cart may contain images of the customer placing items under their clothing.

FIGS. 4A and 4Bare front views400of products402/406with thermal markers404/408in accordance with an embodiment of the invention. Products402/406contain thermal markers404and408. Thermal markers404/408may be thermal barcodes or other machine recognizable indicia. Thermal markers may include structures configured to reflect infrared light/heat or absorb infrared light/heat such as: thermally reflective tapes, thermally reflective ink, thermally reflective materials, thermally absorptive materials, and/or combinations thereof.

FIG. 5shows a process flow chart in accordance with an embodiment of the invention. At step502a vision system detects a person moving a warehoused product. One or more cameras take a first thermal image504of a body of a person handling or moving the warehoused product. A physical size and/or shape of the warehoused product is stored506. A second thermal image is obtained subsequent or after the first thermal image is taken508. A computer system with a program, performs digital image processing to determine differences in image intensities of the first and second thermal images510. At step512, a determination is made if the differences in the first thermal image compared to the second thermal image correlate to a stored physical size or shape of the product moved512. If a match is found a notification of theft including a location and images to an electronic device. The electronic device may be an employee device, a store loud speaker, a shopping cart device, or a device of the person who is concealing the product514.

FIG. 6shows a process flow chart in accordance with an embodiment of the invention. shows a process flow chart in accordance with an embodiment of the invention. At step602a vision system detects a person moving a warehoused product with a thermal marker. One or more cameras take a first thermal image604of a body of a person handling or moving the warehoused product. A physical size and/or shape of the warehoused product is stored606. A second thermal image is obtained subsequent or after the first thermal image is taken608. A computer system with a program, performs digital image processing to determine differences in image intensities of the first and second thermal images610. At step612, a determination is made if the differences in the first thermal image compared to the second thermal image correlate to a stored physical size or shape of the product moved612. If a match is found a notification of theft including a location and images to an electronic device. The electronic device may be an employee device, a store loud speaker, a shopping cart device, or a device of the person who is concealing the product614.

FIG. 7shows a perspective view700of a person holding a warehoused product704/710including thermal markers712/714in accordance with an embodiment of the invention. One or more store cameras706may track a customer/person702and determine that the customer/person702has removed a warehoused product704from an isle shelf708of a store or warehouse. Thermal markers712/714may enable one or more cameras706to detect movement of products704/710. One or more cameras706may comprise one or more of: still image cameras, video cameras, thermographic cameras, infrared cameras, forward looking infrared cameras, motion sensors, or a combination thereof. One or more or the one or more cameras may employ thermal imaging in addition to poly chromatic images/video to track the customer and any items removed from a shelf. One or more thermal imaging cameras706may detect and discriminate a product's thermal image704(shown in light gray) using thermal markers712/714from a customer's thermal image702(shown in dark grey). Temperature differences between objects in a field-of-view of a thermographic camera appear with different colors or shades which directly correlate to the temperatures of the objects. Items with similar temperatures have similar colors or shading. A person,702, shown has a relatively uniform temperature profile shown by the uniform grey coloring. Warehoused products704/710, have a relatively uniform temperature profile but are at a different temperature (ambient) compared to customer/person702at or near body temperature. As person/customer702handles or moves warehoused product704, one or more thermal cameras706may take one or more thermal images or may start taking thermal video frames. A movement of product704may be detected by one or more cameras706and trigger the one or more cameras to start taking thermal images or frames of a body of person702to obtain a first thermal image. One or more cameras706may be used as a motion sensor/detector or a motion sensor may be co-located with one or more cameras706and trigger thermal image capture of a body of person702. Thermal markers may include thermally absorptive and thermally reflective materials as described in relation toFIG. 4.

FIG. 8shows a perspective view800of a person pocketing a warehoused product in accordance with an embodiment of the invention. One or more cameras806which may include one or more camera systems, one or more computers, one or more wireless transmitters, one or more wireless receivers, one or more networks, may be used to detect a customer/person with pocketed product804. Product804may include one or more products removed from a shelf or display808including products with a thermal marker812/814. Thermal markers812/814may include machine readable indicia enabling vision system806to determine theft of a product in a pocket or under clothing of person802. The one or more cameras806may employ thermal imaging in addition to poly chromatic images to track the customer and any items removed from a shelf. One or more cameras806may comprise one or more of: still image cameras, video cameras, thermographic cameras, infrared cameras, forward looking infrared cameras, motion sensors, or a combination thereof. One or more or the one or more cameras may employ thermal imaging in addition to poly chromatic images/video to track the customer and any items removed from a shelf. One or more thermal imaging cameras806may detect and discriminate a product's thermal image804(shown in light gray) from a customer's thermal image802(shown in dark grey). Temperature differences between objects in a field-of-view of a thermographic camera appear with different colors or shades which directly correlate to the temperatures of the objects. Items with similar temperatures have similar colors or shading. A person,802, shown has a relatively uniform temperature profile shown by the uniform grey coloring except for the thermal image of the product804shown in light grey. Warehoused product804, has a relatively uniform temperature profile but is at a different temperature (ambient) compared to customer/person802being at or near body temperature. A difference of two thermal images may reveal a product item that has been recently pocketed. A first thermal image of a body of person802shown inFIG. 7as702may be taken as a product is handled or moved. At a subsequent time, a second thermal image is taken and compared to the first thermal image revealing a difference in intensity (shown inFIG. 9). A first thermal image is important because a product item brought in the store by the customer, such as a cold soda, may have been in their pocked before a pocketed item was placed in the same pocket. A comparison of a first thermal image to a second thermal image is also important to detect a potential theft. Obtaining a first and second thermal image at or near a time of pocketing a product is also important because of possible temperature equalization resulting in an undetectable thermal profile difference between products pocketed and a temperature of a person's body. The first and second thermal images may be taken within 30 seconds of a movement of a product item. As person/customer802handles or moves warehoused product804, one or more thermal cameras806may take one or more thermal images or may start taking thermal video frames. A movement of product804may be detected by one or more cameras806and trigger the one or more cameras to start taking thermal images or frames of a body of person802in order to obtain a first thermal image. A second thermal image may be obtained by comparing a first thermal image to video frames or images taken after or subsequent to the first thermal images and detecting images intensity differences about, overlaid, or superimposed onto a body of a person handling the product. If an intensity difference if found, the two images may be chosen as a first and second thermal image. Both the first and second thermal images may be taken within a 30 second interval of each other. One or more cameras806may be used as a motion sensor/detector or a motion sensor may be co-located with one or more cameras806and trigger thermal image capture of a body of person802. An image of the customer with a customer location superimposed in the image or connected to the image may be sent to an employee's mobile device by means of wired or wireless communications. The image may be sent by text message, by email, by instant messaging, or directly through a custom program application. The machine vision may track customer's movements in real-time while a customer moves inside of a store. The camera system806may use a defined camera location or a defined pixel grouping to determine a customer location identifier. One or more cameras806may be used and a camera may be assigned to a particular area, isle or region within a store. If only one camera is used, pixel groupings of a camera's optical sensor may be related to isles, regions, and/or areas of a store. Cameras may use thermal imaging in addition to optical image recognition. Machine vision may also identify the one or more items for purchase using barcodes, lettering, or dimensional constraints of a product. If a customer's location changes, an updated image may be sent to the employee with an updated location attached to the image. An employee may deliver a product transportation device, container, cart or basket to a customer's current location and ask the customer to “please use a product transportation container instead of your clothing to transport product to be purchased” as a method of preventing theft. A customer may be notified of an arrival of the product transportation device either by the employee, a text message, a display screen on the product transportation device, or by an audio store loud speaker broadcast.

FIG. 9is a perspective view of intensity differences of a pocketed warehoused product in accordance with an embodiment of the invention. Digital signal processing has taken the thermal images ofFIGS. 7 and 8and found differences in intensity relating to a products size and or shape904and determined that a pocketed product matches a product just removed from shelf908. Difference image904/912can be easily recognized by machine vision system/camera(s)906, employing a subtraction routine, thus allowing a shape of the product to be seen and compared to a previously stored shape/size and/or image of the product.

FIG. 10shows a perspective view of intensity differences of a pocketed warehoused product and a potential theft notification in accordance with an embodiment of the invention. After machine vision system/camera(s)1006determine potential theft of product1004, a notification may be sent to an employee1016by wave of an employee device1020. Employee device1020may include a mobile phone or barcode scanner or other portable electronic device. The notification may include one or more images of the person1002, thermal images of the product under clothing of person1002, and combinations thereof. One or more of the images may have a location of person1002superimposed over the image allowing employee1016to find person1002quickly. Employee1016may provide a shopping basket to person1002and ask the person to deposit the hidden items in the cart. In another embodiment, a notification may be sent directly to a mobile device associated with person1002, such as a mobile phone or shopping cart with a mobile device attached. An image of the theft may be displayed to person1002and/or employee1016any mobile device of either party. An audible notification may be given to alert person1002to not carry unpurchased items under clothing.