SMAKNet: SelfSupervised Multilevel Spatial Attention Network for Knowledge Representation towards Imitation Learning In this paper, we propose an endtoend selfsupervised feature representation network for imitation learning. The proposed network incorporates a novel multilevel spatial attention module to amplify the relevant and suppress the irrelevant information while learning taskspecific feature embeddings. The multilevel attention module takes multiple intermediate feature maps of the input image at different stages of the CNN pipeline and results a 2D matrix of compatibility scores for each feature map with respect to the given task. The weighted combination of the feature vectors with the scores estimated from attention modules leads to a more task specific feature representation of the input images. We thus name the proposed network as SMAKNet, abbreviated from Selfsupervised Multilevel spatial Attention Knowledge representation Network. We have trained this network using a metric learning loss which aims to decrease the distance between the feature representations of simultaneous frames from multiple view points and increases the distance between the neighboring frames of the same view point. The experiments are performed on the publicly available MultiView pouring dataset 1. The outputs of the attention module are demonstrated to highlight the task specific objects while suppressing the rest of the background in the input image. The proposed method is validated by qualitative and quantitative comparisons with the stateofthe art technique TCN 1 along with intensive ablation studies. This method is shown to significantly outperform TCN by 6.5% in the temporal alignment error metric while reducing the total number of training steps by 155K.

Attention alignment by linear space projection for video features extraction Video feature extraction is the basis of research on video data. At present, many models based on convolution neural networks for features extracting need to stack deeper network layers to improve the robustness, which means that the amount of parameters and computation is greatly increased while the practicability of such models is constrained. In this paper, a global projection selfattention mechanism is proposed to empower shallow models generate robust video representations. Compared to Nonlocal 17 and Squeeze Excitation 18 networks, we adopt more succinct but sufficient context modeling strategy to establish complete global interaction space, introduce a lighter transform block for attention coefficients predicting, which improves the running efficiency of attention models greatly. We further embed our attention module to several modern networks including ResNet3,  mathbfR(mathbf21)mathbfDmathbf9  and I3D7, in that ResNet18 and  mathbfR(mathbf21)mathbfDmathbf18  meet considerably promotion upon its original version by more than 18.9 percent on UCF101 dataset, this result implies huge potential of proposed attention blocks.