Patent Document ID: 9972158
Application ID: 15283379
Patent Status: 1

Claim One:
1. A method of automatically determining a planogram in a vending machine comprising the steps of: (a) acquiring a Consumer Packaged Good (CPG) reference image database comprising an each color reference image for an each product ID; (b) creating a each reference image color vector for the each color reference image, first discarding outlier pixels responsive to a pixel value; wherein each reference image color vector is responsive to a two-dimensional Gaussian map in a two-dimensional color space plane; adding the each reference image color vector into the reference image database associated with the corresponding each product ID; (c) creating an each gray-scale reference image from the each color reference image and adding the each gray-scale reference image into the reference image database associated with the corresponding each product ID; (d) running a plurality of feature detection/extraction algorithms on the each gray-scale reference image, saving a set of reference keypoints detected by each of the plurality of algorithms in the reference image database, associated with the corresponding each product ID; (e) acquiring a color vending image of a slot in a vending machine; (f) creating a vending image color vector for the color vending image, first discarding outlier pixels responsive to pixel value; wherein the vending image color vector is responsive to the two-dimensional Gaussian map in the two-dimensional color space plane; (g) creating a gray-scale vending image from the color vending image; (h) comparing the vending image color vector to a subset of the each reference image color vectors in the reference image database; creating a sorted comparison list responsive to the comparing; (i) selecting a subset of the sorted comparison list responsive to the best comparison values—the “candidate product ID list;” (j) running the plurality of feature detection/extraction algorithms on the gray-scale vending image, saving a resulting set of gray vending keypoints detected by each of the plurality of algorithms; wherein method steps (k) through (q) below are performed for each candidate product in the candidate product ID list; (k) running one or more matching algorithms between the set of gray vending keypoints and the set of reference keypoints for the each candidate product; wherein matching is restricted to keypoints of a same keypoint type; the result being, for each candidate product, a candidate feature list; (l) culling the candidate feature list responsive to a confidence of each match in the candidate feature list; wherein culling comprises reducing the number of entries or altering the confidence of one or more entries, or both; thus revising the candidate feature list; (m) pruning the candidate feature list from the prior step responsive to a distance between a vending image small area color vector and a reference image small area color vector; wherein the small area color vectors are computed, for a subset of entries in the candidate feature list, for a small reference image area around the reference keypoints and a small vending image area around a corresponding vending image keypoint; thus revising the candidate feature list; (n) pruning the candidate feature list from the prior step responsive to a distance between a vending image large area color vector and a reference image large area color vector; wherein the large area color vectors are computed, for a subset of entries in the candidate feature list, for a large reference image area around the reference keypoints and a large vending image area around a corresponding vending image keypoint; wherein the large image areas are larger than the small image areas; thus revising the candidate feature list; (o) executing an outlier/relationship algorithm on the candidate feature list, saving a resulting feature confidence for each feature in the candidate feature list; (p) discarding outlying features from the candidate feature list, responsive to the previous step; (q) iterate steps (k) through (p) for all candidates in the candidate product ID list until all candidates in the list are so processed; (r) selecting the best match, for each candidate in the candidate product ID list, from each prior iteration of steps (k) through (p), for the slot in the vending machine, responsive to the number of non-discarded feature matches or highest total confidence values, or both, from steps (o) and (p); wherein the best match is the identity of a product in the slot in the vending machine; (s) iterating above steps (e) through (r) for all slots in the vending machine; wherein the color vending image and the slot update for each iteration; wherein the planogram of the vending machine comprises the results of the steps (r) and (s).