Patent ID: 11887384
Assignee: BLACK SESAME TECHNOLOGIES INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G  B | IPC B  G

Claim 0:
1. A method of describing a temporal event, comprising:
receiving a live stream video sequence of the temporal event recorded by an in-cabin camera of a vehicle;
dividing the video sequence into multiple video clips;, extracting at least one physical characteristic of an at least one occupant within video clip;
extracting at least one action of the at least one occupant within the video clip based on a previous action with a previous video clip;
extracting at least one interaction of the at least one occupant with a secondary occupant within the video clip based on a previous action with a previous video clip;
determining a safety level of the temporal event within a vehicle based on the at least one action and the at least one interaction;
describing the at least one physical characteristic of the at least one occupant and the at least one action and the at least one interaction of the at least one occupant; and
wherein the extracting at least one action and extracting at least one interaction is performed by a convolutional gated recurrent unit (GRU), wherein the output of the GRU of the previous video clip is sent to the GRU of the at least one video clip; wherein the GRU allows action and interaction description for occupants detected within the video sequence by retaining spatial information during a forward pass to process temporal features.