2021:  Introducing OpenStreetMap User Embeddings: Promising Steps Toward Automated Vandalism and Com
Channel: State of the Map
Date: 2021-10-12

https://media.ccc.de/v/sotm2021-academic-10188-introducing-openstreetmap-user-embeddings-promising-steps-toward-automated-vandalism-and-community-detection



We develop and test user embeddings approaches to vandalism detection in OSM. We successfully demonstrate improvements to previous vandalism detection methods, and additionally how the user embeddings can further be applied to detect different communities of mappers.  We validated the embedding model with a prepared vandalism corpus that we are also releasing to the OSM community.

With more than 11B edits from 1.6M unique mappers and openly editable by anyone, the OpenStreetMap (OSM) database inevitably contains vandalism. Our approach to detecting it leverages the analytical power and scalability of machine learning through OSM user embeddings. Embeddings are effective in capturing semantic entity similarities that are not explicitly represented by the data. Since word embeddings were first introduced based on the assumption that words adjacent to each other share similar meanings [1,2], the concept of embeddings has been extended beyond word representations to any entity, so long as one can produce a meaningful sequence of the entities. Therefore, we build OSM user embeddings with mappers as entities by constructing sequences of mappers based on shared editing histories and similar behaviors.

**Methods**

_Creating a Vandalism Corpus_

Development of automated vandalism detection methods in OSM has been slow in part because there is no published corpus of bad or vandalized edits from which to train and validate [3]. Vandalized name attributes are especially problematic because this text is rendered on the basemap. The most infamous instance of this type of vandalism was the changing of "New York City" to an ethnic slur; this name attribute was subsequently rendered on maps drawing from OSM data [4]. As part of this work, we construct and make available the first OSM vandalism corpus for the name attribute of OSM features.  Potential examples of vandalism are collected from the OSM Changeset Analyzer (OSMCha) web-based validation tool. These records are then manually reviewed by the Facebook mapping team to identify egregious name changes. Negative samples (non-vandalism) were randomly sampled from a previously validated vandalism-free snapshot of OSM. All of our examples are extracted from OSM data only, no external or conflated data sources.

_User Embeddings_

To construct meaningful sequences of OSM users where adjacent users share similar mapping patterns, we analyzed the edit history of every OSM object and the temporal/semantic editing patterns of individual mappers. These sequences were then fed into a word2vec skip-gram model to train OSM user embeddings. 

**Shared object editing histories** are sequences of OSM users who have edited the same object, in chronological order of editing.  These sequences represent mappers who share interest in the same objects on the map.  This yields 2B sequences of mappers. 

**Semantic and temporal mapping patterns** are sequences of OSM users that have shared editing characteristics with regard to how and when they edit the map.  Starting with _changesets_, we extract the following keys for each OSM element edited in a given changeset when present: `addr:country`, `admin_level`, `amenity`, `building`, `highway`, `natural`, `place`, `source`. Additionally, we extract the following metadata: the presence of `name` tag, the `version` number, the editing software (e.g. iD editor, JOSM), and any hashtags (possibly denoting specific mapping campaigns). Finally, we group all of these edits by two types of temporal patterns: first, the date of the changeset, and second, the hour of the week of the changeset, per year (with 168 hours in a week, we aggregate across each _week-hour_ in a given year). This yields 30M sequences of mappers.

**Results**

_Community Detection_

OSM is comprised of many distinct groups of mappers; considering each of these groups a different sub-community makes OSM a "community of communities" [5].  The creation of the temporal and semantic editing patterns was specifically designed to create sequences of mappers with high likelihood of belonging to the same community. One type of easy-to-identify communities are corporate editing teams: groups of employees that are paid to edit OSM [6]. Results of corporate editing team detection can be easily validated against published lists of known editors.

The five largest corporate mapping teams are Apple (&gt1,200 mappers), Amazon (&gt700), Grab (&gt550), Facebook (&gt250), and Kaart (&gt200).  These counts are based on extracting affiliation from a mapper’s OSM user profile, looking for sentences such as “I work for Amazon" and are likely an under-representation [7].

To validate the performance of the model’s ability to successfully identify members of an editing team based on editing semantics, we used the cosine similarity to compare users.