# Data Quality Analysis and Improvement: A Case Study of a Bus Transportation System
###### Abstract  
Due to the rapid development of the mobile Internet and the Internet of Things, the volume of generated data keeps growing. The topic of data quality has gained increasing attention recently. Numerous studies have explored various data quality (DQ) problems across several fields, with corresponding effective data-cleaning strategies being researched. This paper begins with a comprehensive and systematic review of studies related to DQ. On the one hand, we classify these DQ-related studies into six types: redundant data, missing data, noisy data, erroneous data, conflicting data, and sparse data. On the other hand, we discuss the corresponding data-cleaning strategies for each DQ type. Secondly, we examine DQ issues and potential solutions for a public bus transportation system, utilizing a real-world traffic big data platform. Finally, we provide two representative examples, noise filtering and filling missing values, to demonstrate the DQ improvement practice. The experimental results show that: (1) The GPS noise filtering solution we proposed surpasses the baseline and achieves an accuracy of 97%; (2) The multi-source data fusion method can achieve a 100% missing repair rate (MRR) for bus arrival and departure. The average relative error (ARE) of bus arrival and departure times at stations is less than 1%, and the correlation coefficient (R) is also close to 1. Our research can offer guidance and lessons for enhancing data governance and quality improvement in the bus transportation system. data quality; dirty data; data cleaning; bus transportation system +
Footnote †: journal: Medical  
public buses running on the urban road network. The datasets generated by the public transit system include information such as bus routes and stations, vehicle GPS trajectories, card swipe records, and vehicle schedules. A number of DQ problems arise in the data generation and transmission stages, which have a negative impact on downstream applications. In bus transportation systems, certain applications, such as travel time estimation and traffic flow prediction, require high-quality input data [5]. For instance, in travel time estimation applications, it is necessary to initially calculate the speeds of road segments within the route. During this process, it is essential to assess how many vehicles are currently running within a specific road segment and determine the instantaneous speed of each vehicle using GPS trajectories. However, many GPS points generated by vehicles deviate from their actual positions on the digital map. Obtaining accurate speed information for a road segment directly from these raw GPS points can be challenging. Therefore, we must correct these raw GPS points to align with the road network and calculate the segment speed using the matched GPS points. Our present research is a component of an application-driven project for a major Chinese city. The operators of the public bus transportation are designing smart applications to refine the scheduling tactics of the bus transportation system, enhance service quality, and decrease operating expenses. The information technology infrastructure is the foundation that supports the bus transportation system. Over the past decades, many application systems and sensor networks have been developed and deployed for various operational and management purposes, including GPS devices, smart card readers, and their associated data collection systems. However, these application systems were developed by different providers with different technological architectures. Therefore, all systems operate independently and do not interact with one another. Despite each system collecting a substantial amount of datasets, their values have not been fully explored. Currently, although a few isolated data-driven applications have been developed in the bus transportation system, each of them independently addressed its own data quality issues. Furthermore, most of them did not provide details about the corresponding data-cleaning solutions for readers.