Patent ID: 11869065
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 6:
7. A method comprising:
generating, using one or more cameras in an environment that includes inventory locations holding items, image data representing a user in the environment;
generating feature data using the image data, the feature data comprising:
a segmentation map indicating at least a first location of a hand of the user;
a customer-interaction score map indicating that a second location of the image data represents an interaction between the user and an item; and
an indication of at least one of a direction or velocity of the hand of the user in the image data over time;

inputting the feature data into a first classifier that has been trained, using first training data, to determine whether the image data represents the interaction between the user and the item;
generating, using the first classifier and based at least in part on inputting the feature data to the first classifier, interaction data indicative of a first interaction between the user and the item, the interaction data indicating a time-range of interest of the image data and at least one inventory location of interest of the environment;
inputting the interaction data into a second classifier that has been trained, using second training data, to determine whether the first interaction corresponds to a predefined activity in the environment;
determining, using the second classifier and based at least in part on inputting the interaction data to the second classifier, that the image data represents the predefined activity in the environment, the predefined activity comprising a performance of an action in the environment with respect to a first item at the at least one inventory location of interest;
receiving virtual-cart data representing a current state of a virtual cart of the user;
inputting the interaction data and the virtual-cart data into a third classifier to determine the action performed by the user and an identity of the first item;
generating, using the third classifier and based at least in part on inputting the interaction data and the virtual-cart data into the third classifier, event data indicating the action performed by the user and the identity of the first item; and
automatically updating the virtual cart of the user at least partly in response to the generating of the event data to indicate the action performed by the user and the identity of the first item.