  This work proposes a new end-to-end DCNN based approach for motion
segmentation, especially for video sequences captured with such non-static
cameras, called MOSNET. While other approaches focus on spatial or temporal
context only, the proposed approach uses 3D convolutions as a key technology to
factor in, spatio-temporal features in cohesive video frames. This is done by
capturing temporal information in features with a low and also with a high
level of abstraction. The lean network architecture with about 21k trainable
parameters is mainly based on a pre-trained VGG-16 network. The MOSNET uses a
new feature map fusion technique, which enables the network to focus on the
appropriate level of abstraction, resolution, and the appropriate size of the
receptive field regarding the input. Furthermore, the end-to-end deep learning
based approach can be extended by feature based image alignment as a
pre-processing step, which brings a gain in performance for some scenes.
Evaluating the end-to-end deep learning based MOSNET network in a scene
independent manner leads to an overall F-measure of 0.803 on the CDNet2014
dataset. A small temporal window of five input frames, without the need of any
initialization is used to obtain this result. Therefore the network is able to
perform well on scenes captured with non-static cameras where the image content
changes significantly during the scene. In order to get robust results in
scenes captured with a moving camera, feature based image alignment can
implemented as pre-processing step. The MOSNET combined with pre-processing
leads to an F-measure of 0.685 when cross-evaluating with a relabeled LASIESTA
dataset, which underpins the capability generalise of the MOSNET.
