text: 207,108,1245370187567816708,2983164057,Mohsen Fayyaz,"#CVPR2020 Learning To Temporally Segment Untrimmed Videos from Set of Actions: ""SCT: Set Constrained Temporal Transformer for Set Supervised Action Segmentation"" A new paper by me and my supervisor Juergen Gall. <LINK> <LINK> In a set supervised action segmentation problem, for each training video, only the list of actions is given that occur in the video, but not when, how often, and in which order they occur. Our network divides a video into smaller temporal regions. For each region, the network estimates its length and the corresponding action label. Then the upsampling module uses the lengths and the action probabilities of all regions to estimate the framewise probabilities. Although it is possible to do the upsampling by linear interpolation this operation is not differentiable w.r.t predicted lengths. Therefore, we use our novel upsampling module which is differentiable w.r.t. predicted lengths. Since we do not know the ground-truth lengths and orders but only the set of present actions, we cannot directly use the predicted frame-wise probabilities of the network. Therefore, we introduce a novel temporal transformation method that transforms a temporal sequence to the set of action probabilities w.r.t predicted lengths and temporal locations of actions. Using the set of action probabilities we can use the GT to train the model end-to-end. We will release the source code as soon as possible. Thanks @yassersouri for his valuable comments on our work.",https://arxiv.org/abs/2003.14266,"Temporal action segmentation is a topic of increasing interest, however, annotating each frame in a video is cumbersome and costly. Weakly supervised approaches therefore aim at learning temporal action segmentation from videos that are only weakly labeled. In this work, we assume that for each training video only the list of actions is given that occur in the video, but not when, how often, and in which order they occur. In order to address this task, we propose an approach that can be trained end-to-end on such data. The approach divides the video into smaller temporal regions and predicts for each region the action label and its length. In addition, the network estimates the action labels for each frame. By measuring how consistent the frame-wise predictions are with respect to the temporal regions and the annotated action labels, the network learns to divide a video into class-consistent regions. We evaluate our approach on three datasets where the approach achieves state-of-the-art results. ","SCT: Set Constrained Temporal Transformer for Set Supervised Action