Patent ID: 11954910
Assignee: MASSACHUSETTS INSTITUTE OF TECHNOLOGY
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 15:
16. The apparatus of claim 12, wherein given a hidden state, the policy network estimates a policy distribution and samples an action at at∈Ω={0, 1, . . . L+M−1} via a Gumbel Softmax operation:

at˜GUMBEL(ht,θG), wherein, if at<L, the video frame is rescaled to spatial resolution 3×Hat×Wat and forwarded to the corresponding backbone network ψa t(⋅; θψat) to get a frame-level prediction,

ytat=ψat(Itat;θψat), wherein, I
   t
   
    a
    t
   
  
  ∈
  
   ℝ
   
    3
    ×
    
     H
     
      a
      t
     
    
    ×
    
     W
     
      a
      t, is the rescaled video frame and ytat∈C is the frame-level prediction; and wherein, if the action at>=L, the video frame is skipped for prediction and a subsequent (Fat−L−1) frames are skipped by the policy network, wherein at represents an action at time t, L is a number of resolutions, M is a number of frames in a skipping sequence, (Hx, Wx) is a frame resolution, ht is a hidden state, and θG denotes learnable parameters, and ψa t(⋅; θψat) is a corresponding backbone network.