# Deep Learning Applied to Scenario Classification for Lane-Keep-Assist Systems
## 2 Related Work  
The most prominent driver for the development of deep neural network models capable of time series or sequence prediction and classification has been natural language processing and machine translation [15; 16; 17]. The main idea behind it is to translate words, or phonemes if speech is used, into a fixed length vector embedding. This vector embedding can then be used as a time series sequence. The research resulting from the natural language processing is widely adopted to other fields that deal with time series data. Until recently, the _de facto_ choice for time series modeling have been RNNs. RNNs are networks that use hidden states that get propagated during time to compress valuable information from the past. Especially successful RNNs are the Long Short-Term Memory (LSTM) [18] or Gated Recurrent Unit (GRU) [19]. Both use several gates to control which information should be stored and which information should be forgotten. Outside the natural language processing, the LSTM and GRU have been applied in various tasks such as general time series classification [20], classification of ECG data [21], time series forecasting [22], etc. Other researchers have also combined CNNs with RNNs to increase model performance. The main idea is to use the CNN in the first layers as feature extractors and later use RNNs for sequence classification. This method has been applied to trajectory clustering [23], human activity classification [24], sentiment classification [25], and others. However, RNNs have several limitations that can impact model accuracy and training duration. The training data for RNNs needs to be sequential and divided in such a way that the hidden outputs from one batch match the inputs of the next batch. This batching procedure could lead to deficient performance as some classes are only rarely seen by the network. Normally, shuffling of the data could greatly improve the training. However, shuffling of the data is not possible because we need to follow the constraint that each previous batch needs to lead to the current batch to preserve the propagation of the hidden states. In addition, RNNs process only one set of features at a time. After each step new hidden states and outputs are calculated which are then used in the next step. This sequential calculation procedure leads to increased training duration. CNNs do not suffer from these limitations. The data can be shuffled as individual batches do not have to be sequential. Furthermore, as the CNN is a feed-forward network the whole batch is processed directly. The advantages of CNNs over RNNs have led researchers to apply only CNNs for time series classification [26]; however, their performance has been inferior until recently. [PERSON] et al. [27] have shown that CNNs can produce even better results than RNNs if new architectural improvements such as dilatation and residual networks are used. Because of the recent breakthrough and their superior performance, we will focus on the usage of CNNs for the task of time series classification and compare their performance to RNNs. Outside the scope of time series classification, CNNs have also been used for classification of driving scenarios. [PERSON] et al. [28] proposed a spatiotemporal representation (Grid Map) that can capture dynamic traffic behavior. The representation is constructed using object information obtained by fusing data from several sensors. The representations are then used as inputs for a CNN which classifies the data into five classes: _Validity of frame_, _Ego vehicle speed larger than 1m/s_, _Leading vehicle ahead in lane_, _Other vehicle overtaking the ego vehicle on adjacent lane_, _Cross-traffic in front of the ego vehicle_. Our research differs from the mentioned methods as we perform time series classification using sensor data only. This data is used as an input to a CNN with separable and dilated convolutions, residual connections, and self-normalizing layers.