  Interpreting human actions requires understanding the spatial and temporal
context of the scenes. State-of-the-art action detectors based on Convolutional
Neural Network (CNN) have demonstrated remarkable results by adopting
two-stream or 3D CNN architectures. However, these methods typically operate in
a non-real-time, ofline fashion due to system complexity to reason
spatio-temporal information. Consequently, their high computational cost is not
compliant with emerging real-world scenarios such as service robots or public
surveillance where detection needs to take place at resource-limited edge
devices. In this paper, we propose ACDnet, a compact action detection network
targeting real-time edge computing which addresses both efficiency and
accuracy. It intelligently exploits the temporal coherence between successive
video frames to approximate their CNN features rather than naively extracting
them. It also integrates memory feature aggregation from past video frames to
enhance current detection stability, implicitly modeling long temporal cues
over time. Experiments conducted on the public benchmark datasets UCF-24 and
JHMDB-21 demonstrate that ACDnet, when integrated with the SSD detector, can
robustly achieve detection well above real-time (75 FPS). At the same time, it
retains reasonable accuracy (70.92 and 49.53 frame mAP) compared to other
top-performing methods using far heavier configurations. Codes will be
available at https://github.com/dginhac/ACDnet.
