  Human skeleton, as a compact representation of human action, has received
increasing attention in recent years. Many skeleton-based action recognition
methods adopt graph convolutional networks (GCN) to extract features on top of
human skeletons. Despite the positive results shown in previous works,
GCN-based methods are subject to limitations in robustness, interoperability,
and scalability. In this work, we propose PoseC3D, a new approach to
skeleton-based action recognition, which relies on a 3D heatmap stack instead
of a graph sequence as the base representation of human skeletons. Compared to
GCN-based methods, PoseC3D is more effective in learning spatiotemporal
features, more robust against pose estimation noises, and generalizes better in
cross-dataset settings. Also, PoseC3D can handle multiple-person scenarios
without additional computation cost, and its features can be easily integrated
with other modalities at early fusion stages, which provides a great design
space to further boost the performance. On four challenging datasets, PoseC3D
consistently obtains superior performance, when used alone on skeletons and in
combination with the RGB modality.
