2021:  Towards understanding the temporal accuracy of OpenStreetMap: A quantitative experiment
Channel: State of the Map
Date: 2021-09-19

https://media.ccc.de/v/sotm2021-academic-10398-towards-understanding-the-temporal-accuracy-of-openstreetmap-a-quantitative-experiment



This talk presents results of an experiment conducted on the temporal accuracy of OpenStreetMap, and provides insights into the temporal dynamics with which changes in real-life appear in OSM.

The ability to provide timely information compared to traditional collection methods of geographic information is generally considered as one of the main advantages of volunteered geographic information (VGI) since its emergence in the 2000s (Goodchild, 2007). In addition to several anecdotal examples illustrating how VGI data can provide more up-to-date information than authoritative sources, the literature provides ample evidence on the usefulness of VGI in applications that require timely geodata, such as disaster management (Horita et al., 2013; Neis & Zielstra, 2014). For example, the Haiti earthquake relief effort in 2010 laid the foundations of how remote contributors of OpenStreetMap (OSM) and other platforms can make a difference and aid responding humanitarian agencies after a crisis (Zook et al., 2010). The Humanitarian OpenStreetMap Team has made numerous contributions and helped save lives at numerous instances ever since (Herfort et al., 2021). However, apart from these examples, the temporal dimension of VGI has not received much research attention outside the application of disaster management, and there is a huge gap between assessing temporal accuracy and other factors of data quality, such as spatial accuracy (Antoniou & Skopeliti, 2015; Yan et al., 2020). Aubrecht et al. (2017) highlighted the lack of formal acknowledgment of temporal aspects in the concept of VGI and proposed a framework called ‘Volunteered Geo-Dynamic Information’ to fully integrate spatial and temporal aspects of VGI. Other works utilizing the temporal component in VGI often focus on the behavior of contributors rather than the currency and temporal validity of map features they contributed (Bégin et al., 2018; Haklay et al., 2010; Neis & Zipf, 2012), or studied the evolution of data over time (Girres & Touya, 2010; Zielstra & Hochmair, 2011). While these approaches are useful, by nature they cannot provide a quantitative measure of how current OSM (or VGI in general) is. Arsanjani et al. (2013) noted during their investigations that the temporal accuracy of OSM could not be measured using their traditional extrinsic method, because OSM data was compared to authoritative data that did not contain temporal information (i.e. most recent street configuration regardless of when road segments were built or renovated). Another project, ‘Is OSM up-to-date?’ recognizes the lack of information on temporal accuracy and developed a tool that uses an intrinsic approach to visually show features that potentially contain outdated information (Minghini & Frassinelli, 2019). However, by nature, an intrinsic approach can also not provide an absolute measure of how up-to-date OpenStreetMap is.

This research attempts to fill a gap in the literature by conducting an experiment on the currency of VGI. Using OSM data as a case study, it will measure the temporal accuracy of selected map features. This research overcomes previous limitations by using official data provided by the Florida Department of Transportation (FDOT). The dataset contains details about state-funded highway construction projects, including the date these projects were completed, therefore accurately measuring the temporal accuracy of OSM features is possible by comparing dates projects were finished with the time at which corresponding OSM edits in the database were made. This time difference describes how long it took for the OSM community to adapt to real-world changes and update the map database accordingly.

The historical version of highway construction projects was filtered to projects completed between May 15, 2016 and April 1, 2021. Further, only a subset of projects were used, that resulted in either 1) new infrastructure (new roadways, roundabouts or highway ramps), 2) new lanes in existing roadways (excluding bike lanes), and 3) new bike lanes or paths. Other construction projects, such as traffic improvements, road resurfacing, regular maintenance (e.g. bridge rehabilitation), etc. were excluded, since a useful, high-quality road network database can be maintained without the addition of these information, therefore, they are less likely to migrate into OSM. The methodology uses augmented diffs from the Overpass API to find all changes that occurred on OSM highway features (creation, modification and deletion) and are spatially and temporally close to construction projects. These changes are then matched with a record from the highway construction dataset. Irrelevant changes (i.e. changes made to other highway features) are removed. This is done by manually interpreting and evaluating changes and construction projects using a descr