PWOC3D: Deep OcclusionAware EndtoEnd Scene Flow Estimation In the last few years, convolutional neural networks (CNNs) have demonstrated increasing success at learning many computer vision tasks including dense estimation problems such as optical flow and stereo matching. However, the joint prediction of these tasks, called scene flow, has traditionally been tackled using slow classical methods based on primitive assumptions which fail to generalize. The work presented in this paper overcomes these drawbacks efficiently (in terms of speed and accuracy) by proposing PWOC3D, a compact CNN architecture to predict scene flow from stereo image sequences in an endtoend supervised setting. Further, large motion and occlusions are wellknown problems in scene flow estimation. PWOC3D employs specialized design decisions to explicitly model these challenges. In this regard, we propose a novel selfsupervised strategy to predict occlusions from images (learned without any labeled occlusion data). Leveraging several such constructs, our network achieves competitive results on the KITTI benchmark and the challenging FlyingThings3D dataset. Especially on KITTI, PWOC3D achieves the second place among endtoend deep learning methods with 48 times fewer parameters than the topperforming method.

Learning Residual Flow as Dynamic Motion from Stereo Videos We present a method for decomposing the 3D scene flow observed from a moving stereo rig into stationary scene elements and dynamic object motion. Our unsupervised learning framework jointly reasons about the camera motion, optical flow, and 3D motion of moving objects. Three cooperating networks predict stereo matching, camera motion, and residual flow, which represents the flow component due to object motion and not from camera motion. Based on rigid projective geometry, the estimated stereo depth is used to guide the camera motion estimation, and the depth and camera motion are used to guide the residual flow estimation. We also explicitly estimate the 3D scene flow of dynamic objects based on the residual flow and scene depth. Experiments on the KITTI dataset demonstrate the effectiveness of our approach and show that our method outperforms other stateoftheart algorithms on the optical flow and visual odometry tasks.