  Automatic detection of students' engagement in online learning settings is a
key element to improve the quality of learning and to deliver personalized
learning materials to them. Varying levels of engagement exhibited by students
in an online classroom is an affective behavior that takes place over space and
time. Therefore, we formulate detecting levels of students' engagement from
videos as a spatio-temporal classification problem. In this paper, we present a
novel end-to-end Residual Network (ResNet) and Temporal Convolutional Network
(TCN) hybrid neural network architecture for students' engagement level
detection in videos. The 2D ResNet extracts spatial features from consecutive
video frames, and the TCN analyzes the temporal changes in video frames to
detect the level of engagement. The spatial and temporal arms of the hybrid
network are jointly trained on raw video frames of a large publicly available
students' engagement detection dataset, DAiSEE. We compared our method with
several competing students' engagement detection methods on this dataset. The
ResNet+TCN architecture outperforms all other studied methods, improves the
state-of-the-art engagement level detection accuracy, and sets a new baseline
for future research.
