Patent ID: 11948361
Assignee: GRACENOTE, INC.
Field: Audio-visual technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 5:
6. The method of claim 5, wherein analytically constructing the runtime media graph from the runtime sequence of runtime media frames comprises:
extracting from each respective runtime media frame timing information indicating temporal position in the runtime sequence, and a respective runtime feature vector characterizing media data of the respective runtime media frame;
creating a respective runtime node associated with each respective runtime media frame and its respective runtime feature vector and timing information; and
for every respective pair of created runtime nodes, determining a connecting edge having a length corresponding to a temporal distance between the pair of runtime media frames associated with respective pair of runtime nodes, and a weight corresponding to a similarity metric of the respective runtime feature vectors of the pair of runtime media frames associated with the respective pair of runtime nodes,
and wherein applying the trained ANN to the runtime media graph to compute both (i) the predicted runtime label for each runtime node of the runtime media graph and (ii) the predicted runtime clusters of the runtime nodes comprises:
applying the trained GNN to the runtime media graph to predict a respective embedding runtime vector for each respective runtime node of the runtime media graph; and
applying the trained clustering ANN to the respective embedding runtime vectors to predict clusters of the runtime nodes, and to predict node labels corresponding to the respective runtime labels of the runtime media frames,

and wherein further applying the trained ANN to respectively determine both the predicted runtime media segment associated with each predicted runtime cluster, and the predicted runtime segment class of each predicted runtime media segment comprises:
applying the trained clustering ANN to both the predicted clusters of the runtime nodes and the predicted node labels in order to predict the runtime media segment associated with each predicted runtime cluster, and to predict the runtime segment class of each predicted runtime media segment.