System and method for inventory management

Disclosed is a system and method for using photos of the interior of a retail store to update an inventory database. The inventory management system receives a data capture from a mobile computing device carried by a customer of a retail store. The system sends a data capture request to the mobile computing device, which can include a request for a photo capture. The customer captures an image using the mobile computing device, and sends it to the inventory management system. The inventory management system analyzes the photo, and gleans from the photo inventory information such as products that are in stock, products that are out of stock, where products are located, etc. Photo inventory information obtained from the photo is used to update the inventory database. Each item of inventory data obtained from a photo relieves an employee of the responsibility of obtaining the inventory data.

FIELD OF THE INVENTION

The invention relates generally to inventory management of retail establishments, and specifically to a system and method of using photos of the interior of a retail store to obtain inventory data.

BACKGROUND

Within a retail store, much employee time and effort is often expended determining the relative location of products and their quantity. Some inventory data is captured automatically, such as the inventory information that is received as products are scanned for check out. However, most inventory information has to be obtained by having an employee visually inspecting available inventory and recording the results. Thus, retail store employees spend a considerable amount of time conducting physical inventories. It is desirable to be able to request, and receive, specific inventory information from a customer. Every piece of inventory information that can be obtained from a customer means an employee does not have to spend time gathering that piece of information. Customers are usually equipped with smart phones or other portable device capable of capturing photos while they shop. It is desirable to be able to request photos of the interior of the retail store from a customer, so that inventory information can be gleaned from the photos.

DETAILED DESCRIPTION

Disclosed herein is a system and method for managing inventory for a retail store or other similar establishment, including a method of using a photo of the inside of a retail store to update inventory data. The system includes a means to communicate with the mobile computing devices of customers. Customers often use their mobile computing devices, for example cellphones, tablets, or smartphones, to obtain data from store products, such as scanning a product barcode as they place the product in their shopping cart. These mobile computing devices are often capable of capturing photos. Photos of the interior of a retail store can be valuable resources for inventory data. When the system for managing inventory receives a data capture from the mobile computing device of a customer, a request for an informational data capture is formulated and sent to the mobile computing device. The informational data capture request can be or include a request for a photo capture. The request for informational data capture will be formulated based on the original data capture received, and the type of inventory information that may be needed to update the inventory database. The request for informational data capture can include a request for many different forms of data capture, such as an additional product barcode scan, a scan of a location identifier, a QR code scan, a scan of a store feature barcode, a photo capture, a near field communication (NFC) tag data capture, a Microsoft™ tag data capture, or any other type of informational data capture that can provide useful inventory information to the system for managing inventory. The inventory management system analyzes the photo or photos received, and extracts inventory information from the photo. The inventory management system uses the inventory information from the photo to update an inventory database. Inventory information obtained from a customer relieves employees from having to spend time and energy to gather the inventory information.

Inventory management is a perpetual task for establishments that sell products. It is a constant struggle to know what items exist in a store, which items are selling, which items are not selling, and where the items are. Automated checkout systems have helped by keeping track of items that are sold. However, store employees are still needed to gather missing inventory data by monitoring and recording the status of inventory on store shelves. Having store employees gather inventory data costs the retail establishment in employee salaries. Additionally, the employee is not available to perform tasks other than gathering inventory data. It is desirable to obtain as much inventory information as possible from customers. Customers are often more prevalent in stores than employees, and the store does not have to pay customers a salary for providing inventory data. Every bit of inventory data that can be obtained from a customer frees up an employee to complete a different task rather than gathering the inventory data.

Mobile computing devices such as tablets, cell phones, and portable scanners are often used by customers as they move about a store. Customers are often provided with mobile computing devices to use while they are in a store for the purpose of scanning products for information or remote check-out, for example. Retail stores also provide applications to reside on the personal mobile computing devices of customers for use in remote check out, obtaining coupons, or tracking products. These mobile computing devices are capable of, or already are, providing wireless data pertaining to what products are around them at any particular point in time. Thus, customers already interact wirelessly with one or more store servers using their portable scanners or tablets. The mobile computing devices carried by customers are often capable of capturing and transmitting photos of the interior of the store the customer is shopping in. Photos contain much information on the status of inventory in a store. Organizing the means to collect photos, scanned barcodes and other captured data from customers and employees as they move about the store, requesting specific data or image captures from customers, and using the specific data and image captures obtained from customers can result in an enhanced understanding of the inventory status of a retail store. The disclosed system and method of inventory management collects data captures from mobile computing devices, sends requests for additional image and data captures in response to receiving an initial data capture, and collects and analyzes the customer response. The customer responses are used to obtain inventory data and to update the inventory database.

FIG. 1illustrates a system401for managing inventory of a retail store. System401is an inventory management system of a retail store, and a system that gathers inventory data from photos. System401in this embodiment includes a server102that includes an inventory management system410, and an inventory database120. System410is an inventory management system of a retail store, and a system that gathers inventory data from photos. System401in this embodiment also includes a mobile computing device106that is carried by a customer104. Server102receives communication434from mobile computing device106. Customer104can be any customer or other individual within or around the retail store.

Server102in the embodiment shown inFIG. 1is a retail store server resident in the retail store, but this is not meant to be limiting. Server102may be available via the internet, cloud computing services, or other local or remote server services. Server102can be any type of electronic or computing equipment capable of containing system410and inventory database120.

In this embodiment, inventory management system410is a computer program, but it is to be understood that inventory management system410can take many forms and implementations, including hardware, software, or both. Inventory management system410is resident on server102in this embodiment, but it is to be understood that system410can be a part of any computational equipment. In a particular embodiment, system410is implemented in hardware processors. The hardware processors can be application specific integrated circuits (ASICs) or hardware accelerators, for example. The hardware processors can be part of one or more special purpose computers that execute computer program instructions which implement one or more functions and operations of inventory management system410. In a specific embodiment, inventory management system410includes specialized hardware accelerator integrated circuits designed to perform specific inventory management tasks.

Inventory database120is any type of database useful for the purpose of storing inventory data. Database120can be electronic and/or computerized. In the embodiment shown in the figures, database120is a database stored on memory on server102, in electrical communication with inventory management system410. Inventory database120can be on paper or other storage medium.

Communication434in the embodiment shown inFIG. 1is wireless communication434between mobile computing device106and inventory management system410of server102, but this is not meant to be limiting. Wireless communication434can be a Bluetooth communication, a wireless local area network communication, wireless cellular telephone communication, or any other form or type of wireless communication. In some embodiments, communication434is wired or optical communication. Communication434can be implemented by any type of transmitter/receiver elements known in the art now or in the future for data transfer between a mobile computing device and an electronic or computer system.

Communication434can include many different types of communication between mobile computing device106and server102.FIG. 2shows system401for managing inventory, with customer104scanning a barcode154of a product180and interfacing with inventory management system410via communication434.FIG. 3shows a block diagram of an embodiment of inventory management system410ofFIG. 1andFIG. 2. In this embodiment, communication434includes a data capture122, an informational data capture request424, a customer response426to the informational data capture request424, and a reward132, as shown inFIG. 2andFIG. 3. In some embodiments, communication434includes additional types of communication between mobile computing device106and inventory management system410. The different elements of communication434are discussed in more detail below.

Data capture122can be any type of data captured by mobile computing device106and provided to inventory management system410. Customer104can capture many different types of data in a retail store using mobile computing device106. Mobile computing device106can be used to scan product barcodes, quick response (QR) codes, or Microsoft™ Tags as customer104is putting a product in their cart to purchase the product, for example. Customer104can capture data from NFC tags to obtain information on a product or a location, for example. Customer104can scan a location barcode to determine information about where they are in the retail store or where certain products are located. In some embodiments, customer104will enter data manually into mobile computing device106, such as entering a barcode number manually when the barcode has been damaged. Customer104can use a camera or mobile computing device106to capture an image of a barcode, a QR code, a product, or a location in the store. Any of these data captures can be included in data capture122. For example, data capture122can be an image or photo captured by customer104of a product and/or a store feature. If customer104wishes to obtain information about a product, they can send data capture122that includes an image or photo of the product, or the store shelf with the product, to computer system410. Customer104can be provided with a video, recipe, or other information about the product in response.

Each time mobile computing device106captures data from a store product, location, or other apparatus, mobile computing device106sends this data to system410of server102as data capture122. In some embodiments, data capture122is a product barcode scan. In some embodiments, data capture122is a QR code scan. In some embodiments, data capture122is a Microsoft™ Tag scan. In some embodiments, data capture122is a data capture from a near field communication (NFC) tag. An NFC tag can be any type of tag that can receive or provide data to a nearby mobile computing device. In some embodiments, data capture122is an image capture (photo). In some embodiments, data capture122can be a different type of data capture from a store product or device. Thus, data capture122can be any type of communication where data is captured by mobile computing device106from a product, location, device, structure, or other apparatus in the retail store, and the information is transmitted to system410.

Mobile computing device106can be one of many different types of devices suited for customer104to communicate with server102and inventory management system410. In this embodiment, mobile computing device106is a smartphone device, but this is not meant to be limiting. Mobile computing device106can be a mobile phone, a tablet, a handheld computer, a portable scanner, or any other mobile computing device with the ability to capture images or data from barcodes or other data devices in a retail store, and communicate with server102and/or inventory management system410.

Mobile computing device106is carried by customer104while shopping in or around the retail store. Communication434between mobile computing device106and system410is initiated by mobile computing device106capturing data in the retail store, and transmitting this data capture122to system410. For example, mobile computing device106can be used by customer104to scan a product barcode154on a product180, as shown inFIG. 2. In this example, mobile computing device106sends data capture122to inventory management system410, as shown inFIG. 2andFIG. 3, where data capture122is a barcode scan of barcode154on product180.

Mobile computing device106can be used by customer104to scan barcode154as shown inFIG. 2for many different reasons. For example, customer104can scan product barcode154as they put product180with barcode154in their cart. In this example, mobile computing device106can keep track of the products and the total cost of the products in the shopping cart of customer104. In some embodiments, mobile computing device106can execute remote checkout of customer104. In another example, customer104may scan product barcode154to get information or a price of product180with barcode154. Customer104has many different reasons for scanning barcode154with mobile computing device106. Inventory management system410may or may not take note of why customer104scans barcode154. Inventory management system410takes note that customer104has scanned barcode154, and begins the process of obtaining additional inventory information from customer104.

Inventory management system410includes a reward module112, a communication module414, a photo analysis module436, a photo gather module138, and an inventory module418. A “module”, as used in this document, can be a circuit, a software program, a hardware apparatus, an application specific integrated circuit, a special purpose computer, or a combination of hardware and software circuits, systems, or implementations. A module can be any element, thing, or program that accepts one or more input and provides one or more output. Inventory management system410in this embodiment is a computer implemented system, but this is not meant to be limiting. System410can be implemented in hardware, firmware, software, or any combination of these. In a particular embodiment, reward module112, communication module414, photo analysis module436, photo gather module138, and inventory module418are each implemented as an application specific integrated circuit that includes specialized hardware accelerator integrated circuits designed to perform specific inventory management tasks. In another specific embodiment, reward module112, communication module414, photo analysis module436, photo gather module138, and inventory module418are each implemented as a hardware processor. In some embodiments, each hardware processor is a hardware accelerator that interfaces with a computer program. In some embodiments, each hardware processor is a part of one or more special purpose computer that executes computer program instructions.

Communication module414accepts as input communication434from mobile computing device106. Communication434includes data capture122. Inventory module418is coupled to communication module414, and receives data capture122from communication module114. Inventory module418sends an informational data capture request424to communication module414in response to receiving data capture122from communication module414. Inventory module418formulates informational data capture request424based on data capture122. Inventory module418uses data capture122to determine what other products or apparatus may be around customer104, and what inventory or location information would be helpful to inventory management system410. Informational data capture request424is formulated to request an additional data capture from customer104that will provide useful inventory information to inventory management system110.

In the embodiment shown inFIG. 2, data capture122is a barcode scan of barcode154of product180. Informational data capture request424in this embodiment is a request for an image capture (photo) of the store shelves that hold product180. Informational data capture request424in this embodiment is a photo capture request.

Communication module414transmits informational data capture request424to mobile computing device106, and waits for a customer response426. Customer response426can take a number of forms, including a null response, the requested informational data capture, or a response that informational data capture request424cannot be fulfilled. Customer response426is a null response if no further data is received from mobile computing device106. In other words, a null response is when customer104does not respond to informational data capture request424. Communication module414will consider customer response426to be a null response after a predetermined period of time passes without receiving customer response426.

Customer response426can include a photo140. A photo is also called an image capture, and is the image data output from a digital camera when a photo is taken. Customer response426will include photo140when customer104responds to informational data capture request424by capturing the requested photo140.FIG. 4illustrates customer104capturing a photo140of an interior170of a retail store with mobile computing device106. In this embodiment, mobile computing device106is capturing photo140of shelves172that hold product180, as requested in the example informational data capture request424as described above in connection withFIG. 2andFIG. 3. Note that in some embodiments, a photo140is received as data capture122. In this embodiment, informational data capture request124may include a request for an additional photo140.

In the example embodiment shown inFIG. 2,FIG. 3, andFIG. 4, customer response426includes photo140as shown being captured inFIG. 4, and as shown as one possible example of photo140inFIG. 5, where photo140is a photo capture of the store shelves that contain product180. Customer104received informational data capture request424with mobile computing device106, where informational data capture request includes the request for a photo of the store shelves that hold product180. Customer104captures photo140of the shelves containing product180, as was requested in informational data capture request424and shown inFIG. 4andFIG. 5. Mobile computing device106transmits customer response426to communication module114, where customer response426is or includes photo140. In this embodiment, communication module114is a means for collecting photos of the interior of a retail store. In this embodiment communication module114collects photo140from mobile computing device106of customer104.

Referring back toFIG. 3, communication module414receives customer response426and determines, in this example, that customer response426comprises photo140. Communication module414performs at least two actions in response to receiving customer response426that comprises photo140. First of all, communication module414transmits photo140to photo analysis module436in response to communication module414receiving customer response426that comprises photo140. And, communication module414transmits a reward request130to reward module112.

Photo analysis module436is coupled to communication module414, and receives photo140from communication module414. Photo analysis module436receives photo140from communication module414, and sends photo inventory information448to inventory module418in response. Photo analysis module436is designed and configured to recognize photo140, analyze photo140, and extract photo inventory information448from photo140. Photo inventory information448can be any type of information that is useful to inventory module418and can be identified from photo140, such as products that are recognizable in photo140, where products are located, how many products are present, empty shelves where products should be, etc.

For example, photo140as shown inFIG. 5is received from customer104in response to informational data capture request424, as discussed above. Photo140includes much useful inventory information. Photo140shows products180,181,182,183,184,185,186,187, and188on shelves172. Photo analysis module436will extract photo inventory information448that includes the identity of products181,182,183,184,185,186,187, and188, and that these products are in stock and on shelves172.

Photo140also includes a location identifier barcode156. If location identifier barcode156is readable in photo140, the information in location identifier barcode156can be included in photo inventory data448. Photo analysis module436will identify the location indicated by location identifier barcode156and insert this information in photo inventory data448. For example, location identifier barcode156in this embodiment identifies shelves172, but this is not meant to be limiting. Location identifier barcode156can identify an aisle, a section, or a location category of the retail store. In some embodiments, photo analysis module436can identify a location in photo140by recognizing features other than location barcodes156, such as aisle signs or other identifying features in photo140. Photo analysis module436can determine a location in photo140by identifying a product or a number of products on a shelf or other store feature, and determining the location from the products identified.

Photo140also includes a number of product barcodes154. For each product barcode154that is readable in photo140, photo inventory information448will identify the product, and include this information in photo inventory information448. Product barcodes can be QR codes or other types of data.

Photo140also includes a number of product location barcodes153and155. For each of the product location barcodes153and155that are readable, this data will be included in photo inventory information448. For example, photo140as shown inFIG. 5shows that product185is in stock above product location barcode153. If a photo440as shown inFIG. 6is received by photo analysis module436instead of photo140, photo analysis module436will identify a missing product. Photo analysis module436will identify that product location identifier153identifies a product that is not on shelves172. The missing product information will be sent with photo inventory information448to inventory module418.

It should be evident that photo analysis module436can obtain a large amount of photo inventory information448from any particular photo140or440, limited only by the amount of the interior170of the retail store that is visible in photo140or440, and the information extraction and analysis capabilities of photo analysis module436.

Inventory module418is coupled to photo analysis module436, and receives photo inventory information448from photo analysis module418. Inventory module418uses photo inventory information448to update inventory database120. There are many different actions that inventory module418can take in response to receiving photo inventory information448. Inventory module418will save photo inventory information448, and analyze and take appropriate inventory actions according to the information received in photo inventory information448. It is to be understood that photo inventory information448can take many different forms. Photo inventory information448can take different forms depending on the specific informational data capture request424that was transmitted, and the quality and quantity of information that can be gleaned from photo140.

Updating inventory database120can take many forms, depending on the information received in photo140and photo inventory information448. In some embodiments, updating inventory database120in response to receiving photo140from customer104includes flagging a product as being on feature in response to receiving a photo140that shows the product on feature. In some embodiments, updating inventory database120in response to receiving photo140from customer104includes flagging multiple products as being on feature.

In some embodiments, updating inventory database120in response to receiving photo140from customer104includes flagging a product as mis-placed. For example, inventory module418determines, in a particular example, that product184as shown in photo140is mis-placed. Inventory module418can either change the location of product184in database120, or output an inventory action request150to have product184moved on the store shelves to its proper location. Inventory action request150can be a request for any type of inventory action by an individual, including moving products, re-stocking shelves, replacing old products with new, etc.

In some embodiments, updating inventory database120in response to receiving photo140from customer104includes flagging a product as in stock. In some embodiments, updating inventory database120in response to receiving photo140from customer104includes flagging a product as being out of stock.

In some embodiments, inventory module118checks one or more products against a planogram in response to receiving photo140. Inventory module418can check any or all of products180,181,182,183,184,185,186,187, or188of photo140against a planogram and determine if all are in their right location. In some embodiments, inventory module418verifies the street date of a product in response to receiving photo140. In some embodiments, inventory module418checks whether a product location barcode155matches a product location indicator in response to receiving photo140. In some embodiments, inventory module418updates a product location indicator in response to determining from photo140that a product location indicator does not match the product location barcode155for that product in inventory. In some embodiments, inventory module118generates an inventory action request150to move a product to a new product location in response to determining from photo140that a product location indicator does not match the product location barcode155for that product. Updating inventory database120can take these or many other forms, depending on the particular inventory information received from photo140of customer response426.

As can be understood from the above examples, informational data capture request424and customer response426with photo140can take many forms, depending on the type of inventory information determined to be needed by inventory module418, and the ability of customer104to supply inventory information using mobile computing device106. Inventory management system410is designed to gather as much inventory information as possible from customer104and photos140, which maximizes the accuracy of inventory database120and minimizes the amount of employee time and effort spent on gathering inventory data.

Inventory management system410in the embodiment shown inFIG. 3also includes photo gather module138. Photo gather module138is coupled to photo analysis module436, and internet146. Photo gather module138is configured to find photos152of the interior170of the retail store from internet146, and use the photos152from the internet the same as photos140captured by customer104. If photo gather module138can find photos152that it can identify as the interior170of the retail store, and can identify the time they were taken, these photos152can be analyzed and used just as if the photos152were captured by a customer, as described above. In some embodiments, photo gather module138is coupled to other possible sources of photos152, such as security cameras, for example. Inventory management system410is designed to glean and use information from any number of sources to obtain inventory data.

When communication module414receives data capture122or customer response426from mobile computing device106that includes photo140, communication module414transmits a reward request130to reward module112. Providing customer104with a reward for providing photo140incentivizes customer104to participate in gathering inventory data. Reward module112is coupled to communication module414and receives reward request130. Reward module112transmits a customer reward132to communication module414in response to receiving reward request130.

Reward132can take many forms. Reward132can be a coupon, a product information guide, a website, a recipe, a video, for example, but not by way of limitation. In some embodiments, customer104gets a choice of what reward they would like. In some embodiments, reward132is related to data capture122initially provided by customer104via mobile computing device106. For example, when mobile computing device106sends barcode scan of barcode154on product180, customer reward132might be a recipe that uses product180. Or reward132might be a coupon for product180, or an information guide on product180. By way of another example, customer104may send a photo140of a motor oil product. Reward132could be a video of how to change the oil in a car. Customer reward132can be any type of reward that works to obtain inventory information from customer104and mobile computing device106.

Reward module112sends customer reward132to communication module114. Communication module114transmits customer reward132to mobile computing device106of customer104for customer104's use. Inventory management system410then waits for another data capture122to arrive, and the inventory data gathering process begins again.

FIG. 7illustrates a method500of using a photo of the inside of a retail store to update inventory data. Method500of using a photo of the inside of a retail store to update inventory data includes element510of analyzing a photo of an interior of the retail store. Method500also includes element520of updating an inventory database using photo inventory data obtained from the photo. Method500of using a photo of the inside of a retail store to update inventory data can include many other elements. In some embodiments, method500of using a photo of the inside of a retail store to update inventory data includes obtaining a photo of an interior of the retail store from a customer of the retail store. In some embodiments, method500of using a photo of the inside of a retail store to update inventory data includes obtaining a photo of an interior of the retail store from the internet. In some embodiments, method500of using a photo of the inside of a retail store to update inventory data includes sending a photo capture request to a mobile computing device of a customer in response to the customer scanning a barcode of a product in the retail store. In some embodiments, method500of using a photo of the inside of a retail store to update inventory data includes delivering a reward to the customer for capturing the photo of the interior of the retail store. In some embodiments, method500of using a photo of the inside of a retail store to update inventory data includes sending a request to the customer for a second photo capture in response to receiving the photo from the customer.

Step510of analyzing a photo of an interior of a retail store can include many elements. In some embodiments, step510includes identifying a product in the photo. In some embodiments, step510includes identifying a product location in the photo. In some embodiments, step510includes identifying a missing product from the photo. In some embodiments, step510includes identifying one of an aisle, a section, or a category of the retail store in a photo. In some embodiments, step510includes identifying one or more product barcode in a photo. In some embodiments, step510includes identifying a QR code in a photo.

Step520of updating an inventory database using photo inventory data obtained from the photo can contain many additional elements. In some embodiments, updating an inventory database includes flagging a product as being on feature in response to receiving the photo. In some embodiments, updating an inventory database includes flagging a product as out of stock in response to receiving the photo. In some embodiments, updating an inventory database includes flagging a product as mis-placed in response to receiving the photo. In some embodiments, updating an inventory database includes flagging a product as in stock. In some embodiments, updating an inventory database includes checking one or more items against a planogram. In some embodiments, updating an inventory database includes verifying the street date of a product. In some embodiments, updating an inventory database includes checking whether a product location barcode matches a product location indicator. In some embodiments, updating an inventory database includes updating a product location indicator in response to determining from the photo that the product location indicator does not match the product location barcode. In some embodiments, updating an inventory database includes generating an inventory action request to move a product to a new product shelf location in response to determining from the photo that the product location indicator does not match the product location barcode.

Disclosed is a system and method for using photos of the interior of a retail store to update an inventory data base. The system receives a data capture from a mobile computing device carried by a customer of a retail store. The system sends an informational data capture request to the mobile computing device, which can include a request for a photo capture. The customer captures an image using the mobile computing device, and sends it to the system for inventory management described herein. The inventory management system analyzes the photo and gleans from the photo inventory information such as products that are in stock, products that are out of stock, where products are located, etc. Photo inventory information obtained from the photo is used to update the inventory database. Photos can contain much inventory information, and each item of inventory data obtained from a photo relieves an employee of the responsibility to obtain that piece of inventory data. The employee is able to do other work, and the retail store is able to save the cost of having employees gather inventory data.