Patent ID: 11876986
Assignee: GOOGLE LLC
Field: Audio-visual technology (Electrical engineering)
Classification: CPC H  G | IPC G  H

Claim 0:
1. A computer-implemented method for moment localization in a video corpus comprising a plurality of videos, the computer-implemented method comprising:
obtaining, by a computing system comprising one or more computing devices, a user query, the user query comprising text;
identifying, by the computing system, one or more highest likelihood videos of the plurality of videos, each highest likelihood video of the one or more highest likelihood videos identified based at least in part on a video-query compatibility score between the user query and a video representation of the highest likelihood video that is output by a machine-learned hierarchical video encoder model; and
determining, by the computing system, a moment localization within a matching video of the one or more highest likelihood videos, the moment localization comprising a moment beginning and a moment end;
wherein the moment beginning comprises a beginning frame of the matching video, the beginning frame having a frame representation that is classified as representing a beginning of a moment described by the user query; and
wherein the moment end comprises an end frame of the matching video, the end frame having a frame representation that is classified as representing an end of the moment described by the user query.