PATENT CLAIM ANALYSIS

Application Number: 16404667
Application Type: Utility
Filing Date: 2019-05
Publication Date: 2020-01
Patent Classification: ["382", "103000"]

Abstract:
A projected image item tracking system that analyzes projected camera images to determine items taken from, placed on, or moved on a shelf or other area in an autonomous store. The items and actions performed on them may then be attributed to a shopper near the area. Projected images may be combined to generate a 3D volume difference between the state of the area before and after shopper interaction. The volume difference may be calculated using plane-sweep stereo, or using convolutional neural networks. Because these methods may be computationally intensive, the system may first localize a change volume where items appear to have been displaced, and then generate a volume difference only within that change volume. This optimization results in significant savings in power consumption and in more rapid identification of items. The 3D volume difference may also indicate the quantity of items displaced, for example from a vertical stack.

Claim (Index 23):
A projected image item tracking system comprising:\n a processor coupled to\n a sensor configured to generate\n an enter signal when a shopper reaches into or towards an item storage area in a store, wherein said item storage area contains items; and, \n an exit signal when said shopper retracts from said item storage area; and, \n \n a plurality of cameras oriented to view said item storage area; \n wherein said processor is configured to\n obtain a plurality of before images captured by said plurality of cameras, each before image of said plurality of before images corresponding to a camera of said plurality of cameras, wherein said each before image is captured at a time before said enter signal; \n obtain a plurality of after images captured by said plurality of cameras, each after image of said plurality of after images corresponding to said camera of said plurality of cameras, wherein said each after image is captured at a time after said exit signal; \n project said plurality of before images onto two or more planes in said item storage area to generate a plurality of projected before images corresponding to each combination of a plane of said two or more planes and said camera of said plurality of cameras; \n project said plurality of after images onto said two or more planes to generate a plurality of projected after images corresponding to said each combination of said plane of said two or more planes and said camera of said plurality of cameras; \n analyze\n said plurality of projected before images, and \n said plurality of projected after images, to\n identify an item of said items taken from or put into said item storage area between said enter signal and said exit signal; and, \n associate said item with said shopper; and, \n \n \n wherein said analyze said plurality of projected before images and said plurality of projected after images comprises\n input at least a portion of said plurality of projected before images and at least a portion of said plurality of projected after images into a neural network trained to output said item of said items taken from or put into said item storage area between said enter signal and said exit signal,\n wherein said neural network is further trained to output an action that indicates whether said item of said items is taken from or is put into said item storage area between said enter signal and said exit signal; and, \n wherein said neural network comprises\n a feature extraction layer applied to each of said at least a portion of said plurality of projected before images and at least a portion of said plurality of projected after images, wherein said feature extraction layer outputs image features; \n a differencing layer applied to each pair of said image features associated with a portion of a projected before image from said camera and said image features associated with a portion of a projected after image from said camera, wherein said differencing layer outputs feature differences associated with said camera; \n one or more convolutional layers applied to said feature differences associated with each camera of said plurality of cameras; \n an item classifier layer applied to an output of said one or more convolutional layers; and, \n an action classifier layer applied to said output of said one or more convolutional layers.

Metadata:
- Claim Count in Document: 36.0
- Percentile: 100.0
- Lexical Diversity: 1.77174
- Patent Class: 382.0
- Transitional Phrase Type: open
- Component Type: 1
- Foreign Priority: False
- Related Applications: ['16254776', '16138278', '16036754', '15907112', '15175997']

Analysis Scores:
- 35 USC 101 Eligibility (BERT): 0.3609499303173118
- 35 USC 102 Novelty (BERT): 0.5565373767767148
- Combined Prediction Score: 0.3805086749632521
- Mean Citation Score: 317.190252
- Max Citation Score: 438.41324
- Similarity Product: 328.91202563545465

Labels:
- Claim Label 101: 1
- Claim Label 102: 1
- Claim Label 103: 1
- Claim Label 112: 1
- Combined Label: 1
- Label 101 Adjusted: 1

Dataset: test