Patent ID: 11948361
Assignee: GRACENOTE, INC.
Field: Audio-visual technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 3:
4. The method of claim 1, wherein the ANN comprises a graph neural network (GNN) and a clustering ANN,
wherein analytically constructing the training media graph from the training sequence of training media frames comprises:
extracting from each respective training media frame the respective label, timing information indicating temporal position in the training sequence, and a respective training feature vector characterizing media data of the respective training media frame;
creating a respective training node associated with each respective training media frame and its respective training feature vector and timing information, and labeled with the respective label of the associated training media frame; and
for every respective pair of created training nodes, determining a connecting edge having a length corresponding to a temporal distance between the pair of training media frames associated with respective pair of training nodes, and a weight corresponding to a similarity metric of the respective training feature vectors of the pair of training media frames associated with the respective pair of training nodes,

and wherein training the ANN to compute both (i) the predicted training labels for each node of the training media graph and (ii) the predicted clusters of the nodes corresponding to predicted membership among the respective training media segments of the corresponding training media frames comprises:
training the GNN to predict a respective embedding training vector for each respective node of the training media graph, the respective embedding training vector comprising a reduced-dimension mapping of the respective training feature vector associated with the respective node; and
using the respective embedding training vectors as input, training the clustering ANN to predict clusters of the nodes corresponding to the ground-truth clusters, and to predict node labels corresponding to the respective labels of the training media frames.