# Convolutional Neural Network-Based Travel Mode Recognition Based on Multiple Smartphone Sensors
## 1 Introduction  
Understanding different travel modes of human travel will benefit numbers of applications such as smart city, urban transportation planning, and urban traffic management. As a part of the concept of smart city, smart mobility aims to manage transportation infrastructure more efficiently and provide more efficient transportation services, which will need the big data to understand how people travel and how the transportation is used [1, 2]. For example, understanding and modelling different travel modes will shorten the travel time and help to reduce car traffic congestion on the most loaded routes by introducing collective transport, which will also reduce the air pollution. Moreover, this will also help to track human mobility, which enables the monitoring of the spread of diseases and other potential hazards [2]. Traditional methods to collect travel mode data of human mobility are based on household surveys or telephone interviews for over four decades, which are time-consuming and labor-intensive [1]. Furthermore, the collected data tends to be inefficient and inaccurate if low response and incomplete information are involved. Since the mid-1990s, GPS (Global Positioning System) data has become popular for travel mode detection [1, 2, 3], but GPS signals are easily lost in indoor and high-rise urban environments due to the impact of multipath [4]. For example, if the target travel mode is walkingindoors or in a shopping mall, it is difficult for GPS data to support the identification of the \"walking\" travel mode. With the rapid development of mobile computation, a smartphone-based method is becoming a promising way for travel mode recognition. Smartphones with built-in sensors can be considered as primary motion capture sensors [5]. Their enormous market penetration rate and being relatively close to users is another advantage for travel mode detection [1]. Recently, a number of smartphone-based activity detection methods have been developed [6; 7; 8; 9; 10; 11]. It should be noted that individuals' activity modes can reflect their travel mode. Thus, many studies of smartphone-based approaches on classification of travel modes have been proposed [2; 12; 13; 14]. A smartphones-based travel modes recognition method generally includes two steps: the first step is activity features computing (e.g., mean, variance, and maximum) based on raw sensors data, and for the second step, the extracted features are adopted to classification algorithms for estimating the travel mode. The travel mode recognition performance depends highly on the feature representation of the sensor data [12; 13; 14]. However, current methods rely on complex hand-crafted features extraction, which usually require laborious human intervention. The feature selection process is costly and inefficient. To solve this problem, researchers exploit deep learning algorithms to extract multiple levels of feature automatically [15; 16; 17; 18]. In this paper, to develop a convenient prototype application of travel mode recognition and compare the effectiveness of different sensor data fusion for travel mode recognition, we proposed a convolutional neural network (CNN) based method for travel mode recognition by fusing multiple types of smartphone sensor data. Four types of sensors were selected to detect users' travel modes, which contained accelerometer, gyroscope, magnetometer, and barometer. Compared with conventional machine learning techniques based on manual selected features, the proposed method was able to automatically extract different moving features. We also developed a prototype application to collect smartphone sensor data which did not need to set an initial position or orientation value while the application is running. The collected smartphone sensor data was under five different transportation modes (walk, bike, bus, car, and metro). In the experiment analysis part, we compared and analyzed the performance of the proposed system under different hyperparameters and combinations of sensors.