## 2 Related Work  
Deep learning caused a quantum leap in the performance of visual recognition systems. The progress was first demonstrated on the core computer vision task of image classification [14; 15], but the momentum quickly transferred to other tasks as well. For example, fully convolutional networks [16] successfully adapted image classification architectures for dense prediction of semantic classes at the pixel level. Further work [17; 11; 18; 19] utilized the classification architectures as the backbones of the semantic segmentation models and considered various options to recover the spatial details lost due to the downsampling. SegNet [18] recovers exact positions of pooled features in the backbone by memorizing corresponding indices and gradually recovers the spatial resolution through upsampling and convolutional layers. DeepLab [19] drops some of the pooling layers from the backbone to avoid downsampling and dilates the corresponding convolutional layers to increase the receptive field. PSPNet [17] introduces a module based on pyramid pooling of a convolutional feature map, which introduces a global context into the local features. U-Net [20] architecture is specialized for medical image segmentation and introduces skip connections between the downsampling and the upsampling path. Skip-connections enable the model to mix semantically poor and spatially accurate features from the backbone with semantically rich features from the upsampling path. LadderDenseNet [11] proved that the lean upsampling path with skip connections is a more accurate and efficient solution for recovering the lost spatial details in large-resolution images. Skip-connections have emerged as an indispensable part of the state-of-the-art semantic segmentation architectures [21; 22; 23]. This is particularly the case in models aiming at real-time inference speed [24; 25]. Our semantic segmentation models follow the SwiftNet baseline [25] architecture with spatial pyramid pooling and skip connections in the upsampling path. We demonstrate that models trained with the proposed MixSkip training procedure are able to perform well with and without skip connections. This suggests that such models are more resistant to noise in skip connections. This is particularly important for forecasting applications because of the noise introduced due to the future uncertainty. Semantic forecasting deals with predicting the future at the semantic level while observing only frames from the past. Dense semantic forecasting predicts future outputs of dense prediction algorithms such as: semantic segmentation [5], instance segmentation [26], panoptic segmentation [7], etc. Early work focused on predicting the future semantics by observing semantic predictions from the past. Such an approach is often called semantics-to-semantics (S2S) mapping. The first such work considered forecasting semantic segmentation with a dilated convolutional model that receives past per-class semantic maps [5]. This approach was extended with Bayesian variational inference in order to allow multi-modal forecasts [27]. Large inter-frame motion has been alleviated by means of an encoder-decoder architecture with Conv-LSTM skip connections [28]. This is related to our approach, since we also propose a dense semantic forecasting model with skip connections; however, we propose to transform skip connections through forecasted displacements. Additionally, our method is more efficient as heavy computation is done on a single most-condensed representation. Many state-of-the-art methods follow the feature-to-feature (F2F) forecasting approach [8; 9; 12]. The first such approach was applied to instance segmentation based on the Mask R-CNN model above the feature pyramid network [26]. Each level of the feature pyramid is forecasted separately with a sequence of dilated convolutions. APANet [12] improves their approach by introducing connections between the pyramid levels and by forecasting skip connections through ConvLSTM modules.