# Data Quality Analysis and Improvement: A Case Study of a Bus Transportation System
## 4 Data-Cleaning Methods
### Cleaning Method for Missing Data  
In this section, we introduce missing values in the spatiotemporal information of bus arrivals and departures and then propose a corresponding solution. In this city, each bus has been equipped with a GPS device, which continuously reports its position information to the backend via the mobile internet. When a bus is entering a station, the driver manually reports a message by pressing a button attached to the GPS device. Ideally, it will produce a full ordered sequence for each bus trip, and each element in this sequence contains information about when a bus enters a station and where the station is located, and other information such as \"numberPlate\", \"routeCode\", and \"stopCode\". However, on occasion, some drivers may forget to press the button when the bus arrives at a station, leading to data gaps in the corresponding table. We propose a solution based on multi-source data fusion to repair the missing data to complete the information on bus arrival and departure. The detailed steps are as follows. Firstly, we check the continuity of \"stopCode\" by the \"Route_station\" table to determine whether there are missing data on the information of entering a station. Secondly, missing spatiotemporal information is filled with information \"stopCode\", \"routeCode\", and \"direction\", and then filled with \"stationName\", \"longitude\", and \"latitude\" according to information from the \"Route_station\" table. Finally, we repair the entry \"timestamp\" in conjunction with each bus's corresponding GPS trajectory. In this dataset, the \"GPS\" table includes eight fields: \"numberPlate\", \"timestamp\", \"longitude\", \"latitude\", \"runningStatus\", \"vehicleCode\", \"speed\", and \"direction\", respectively. In the previous steps, we have already successfully repaired the information about \"longitude\" and \"latitude\". In this step, we will find the timestamp \(\mathrm{T}^{\mathrm{repair}}_{\mathrm{k,in}}\), at which the bus enters the station \(\mathrm{S}_{\mathrm{k}}\) and the time \(\mathrm{T}^{\mathrm{repair}}_{\mathrm{k,out}}\), at which the bus leaves the station \(\mathrm{S}_{\mathrm{k}}\). The method is to measure two different spatiotemporal points: one is the location of a station denoted by a GPS point, and the other is a location listed in the \"IAO_station\" table. If the distance between these two points is less than the threshold dt, then we can fill the temporal information about the bus's arrival and departure. The implementation principle is shown in Figure 2:  
We choose two metrics from ref. [(39)] to evaluate our algorithm. 1. The Missing Repair Rate (MRR) is a metric to measure the repairing accuracy of the spatial information, which includes \"routeCode\", \"stopCode\", \"direction\", \"station Name\", \"longitude\", and \"latitude\". MRR is defined by Formulas (1) and (2):  
Figure 2: Schematic diagram of bus arrival and departure time recovery. \[\text{MRR}=\frac{1}{N}\ \sum_{k=1}^{N}\delta\Big{(}x_{k,original},x_{k,repair}\Big{)} \times 100\% \tag{1}\]  
\[\delta\Big{(}x_{k,original},x_{k,repair}\Big{)}=\left\{ \begin{array}{cc}1,&x_{ k,original=}x_{k,repair}\\ 0,&x_{k,original\
eq x_{k,repair}}\end{array} \right. \tag{2}\]  
In Formulas (1) and (2), \(x_{k,original}\) represents the true value of the missing information of station \(S_{k}\) and \(x_{k,repair}\) is the corresponding repaired value. Note that the value in this context is a set of aforementioned fields about spatial information. \(\delta\Big{(}x_{k,original},x_{k,repair}\Big{)}\) is a Boolean function that is used to compare two different variables. If two variables are equal, the function returns 1; otherwise, the function returns 0. We take each combination of the original and repair values as a sample. N is the number of samples. The steps of calculating MRR are as follows. Firstly, the algorithm applies the Boolean function to each sample and collects the return value. Secondly, it sums all the returned values. Finally, it calculates the average by the sum and the number of samples.