dataesr/scientific-paragraphs-categorization huggingface dataset:
Usability As mentioned above, usability (or fitness-for-use) refers to the external quality of a dataset and is focused on the needs of the user. The five aforementioned data quality elements may be aggregated in order to describe the overall usability of a specific dataset for a particular use, i. e. , fitness-for-purpose. In other words, usability acts as a complementary element by linking both user requirements and data quality measures to check whether the data for a specific application can be used  (Guptill and Morrison, 1995; Devillers et al. , 2007). Table  1  summarises the requirements and specific aspects regarding the application of ISO quality measures to VGI. In Section 3, establishing workflows and combining quality indices to assess VGI quality in order to assess usability is further developed.
Quality Measures Specific to VGI When considering VGI, other data quality indicators are required to supplement those proposed in the ISO framework. This occurs not only because in many situations comparison with authoritative datasets is not possible, but also because the characteristics and nature of VGI enable the use of indicators that do not usually make sense when applied to data created by professionals. These indicators may provide valuable information even though in most situations they do not assess accuracy but instead assess data reliability or credibility (which are considered as synonyms in this chapter). As these indicators may   2 ). For example,  Goodchild and Li (2012)  provide three broad categories of measures to ensure VGI data quality: i) crowdsourcing revision, where data quality can be ensured by multiple contributors; ii) social measures, which focus on the assessment of contributors themselves as a proxy measure for the quality of their contributions; and iii) geographic consistency, through an analysis of the consistency of contributed entities.
Data-based Indicators One important group of quality indicators of VGI are those that involve comparison with other sources of crowdsourced data (Table  3 ). One possibility is to measure the 'agreement' to the corresponding data, which we define here as the coherence of the data with other sources of crowdsourced data. Agreement can be measured between datasets using a Boolean measure or a continuous variable with traditional measures such as distance between corresponding elements, attribute comparisons, etc. , and may be considered an indicator of data reliability. Logical consistency of data available in different data sources can also be used to estimate data reliability, identifying if, according to the types of features present in all available data sources, a particular contribution is likely to be correct or not.
As stressed by  Sui et al. (2013) , approaches that compare data based on their geographic location have not yet been developed enough. Note, however, that all these indicators may be used to measure data reliability, but not to assess data accuracy if none of the data under comparison can be considered as reference data. Another set of indicators can also be calculated that could reveal VGI quality by solely examining the VGI dataset itself and the associated metadata (Table  3 ). The work in this area has focused primarily on assessing OSM data quality. Such indicators could include the total length of features and the point density in a square-based grid, as calculated by  Ciepłuch et al. (2010) , or the number of versions, the stability against changes and the corrections and rollbacks of features, as examined by  Keßler and de Groot (2013).
The provenance of features contributed to OSM (i. e. whether the data were captured using a GPS, were manually digitised or resulted from a bulk import) has been the Some of the facets of traditional metadata are of particular interest in assessing and using VGI. For example, the lineage of a record or dataset may include its edit history and information on how it was measured, and can be especially important in the automated assessment of VGI fitness-for-use. Examples of metadata potentially useful for VGI are equipment used in measurements; data about the volunteer (contributor indicator); date and time of data collection; or atmospheric conditions at the time a particular observation was taken. Individual metadata about heterogeneous observations can be extremely useful in identifying bias and likely trustworthiness, as seen, for example, in the context of amateur weather monitoring  (Bell et al.
This trust model therefore takes both spatial context and reputation, through user ratings and the relationships between contributors, into account. The model remains theoretical and was not applied in the paper cited above, but an example of data collection for an urban growth scenario was outlined. The inclusion of relationships via social networking could give greater weight to the ratings of certain individuals. Jokar Arsanjani et al. (2015a) have for their part proposed a multivariate indicator, referred to as the contribution index (CI), that combines diverse classic quality indicators, as well as user perspectives of data, including the number of volunteers involved in mapping a particular feature along with the frequency of contributions (Figure  2 ). However, the main problem with the assessment of VGI based on fitnessfor-use is that many methods and measures are designed to assess a specific VGI dataset or a single use case, and are not generalisable or transferable to other VGI datasets or purposes.
However, some papers have appeared in which quality assurance workflows have been proposed. For example,  Bordogna et al. (2015)  propose a flexible system that allows users to specify minimum acceptable quality levels based on their requirements (Figure  3 ). The system contains a series of quality indicators, including both standard  Assessing VGI Data Quality 155 internal quality measures such as positional accuracy and ones specifically geared towards VGI (see Section 2. 2). The user can rank the importance of the different indicators and specify a minimum acceptable level of quality for each indicator, and then the system acts as a filter to return only those items from the VGI database that meet all of these minimum levels; the authors perform a demonstration of the system on a VGI dataset of glaciological observations.
The creation of workflows that allow for the assessment of different aspects of quality has also been proposed. The framework proposed by COBWEB includes a quality assessment workflow that uses some automatic validation procedures to obtain data quality indicators to insert in the information metadata  (Meek et al. , 2016) , while  Ballatore and Zipf (2015)  have proposed a multidimensional framework to assess conceptual quality. The need to assess fitness-for-use has been present even without considering VGI, and methodologies to make this assessment have already been proposed in other contexts. For example,  Lush (2015)  proposed the creation of a GEO label that aims to be a mechanism to assist users to determine the fitness-foruse of datasets: a visual tool was developed that aggregates information about the producer, data lineage, compliance with standards, existence of quality information, user's feedback, expert reviews and citation information.
e. OSM) has evolved in terms of quality will give insights valuable to other existing VGI projects or projects that will follow in the future, including those in the citizen science domain. Spatial data quality is the cornerstone of every spatial database, map, product or service. Measuring, understanding and documenting the quality of spatial data is of paramount importance for any kind of geodata, including VGI. This chapter will examine OSM quality evolution from a new point of view. In Section 2, quality evaluation procedures, as described in the ISO quality framework, will be discussed. Then, in Section 3, the methodology for understanding the evolution of OSM quality will be introduced. The central focus will not be on the data themselves (as is usually the case in most OSM-based quality studies), but rather on the micro-environment inside which OSM is evolving.
OSM is a prime example of VGI as it has managed to provide free, constantly updated, crowdsourced data for the globe. However, when research focuses on VGI data quality, scholars tend to examine some of the spatial quality elements for a given study area, e. g. cities, urban areas or nationwide  (Antoniou, 2011; Girres and Touya, 2010; Haklay et al. , 2010; Jokar Arsanjani et al. , 2015). The studies usually follow a benchmark evaluation process, which involves creating a copy of what is a continuously changing dataset, and then evaluating this copy as if it were a static dataset. This method gives insight into the data quality at the time when the copy was created; thus, these efforts provide a good understanding of selected quality elements at a given point in time compared with corresponding authoritative datasets.
However, spatial datasets, and especially VGI ones, are not static products and hence time is a critical factor that is not often considered. The starting point for a spatial product is the specifications that will be used to create the dataset. Yet these specifications can change over time for both authoritative and VGI datasets. In fact, the latter kind of Geographic Information (GI) is more susceptible to changes in specifications since bottom-up processes provide the flexibility for new rules to be established or existing ones deprecated more easily by the community of volunteers. While the path of evolution and change in the specifications of a product is inescapable, there is a fundamental difference in how each source of GI (i. e. authoritative or VGI) handles their dataset life-cycle.
For example, authoritative data, collected by NMAs or Commercial Mapping Companies (CMCs), usually follow a versioning system. Users of such data are notified that a set of updates is available or, more relevant to our case, that a new dataset has been created based on new specifications. The product specifications can also be available to the interested parties. A case in point can be found in the practices of the UK's Ordnance Survey (OS). For the OS Mas-terMap product (OS 2001), for example, OS provides a detailed document that explains how each physical entity is conceived, modelled and stored and thus what accuracy and attributes should be expected. The important point here is that while a new dataset is developed, or during the migration from one form of specification to another, the datasets are not accessible to the users.
This process takes place in-house, and only when the whole process has been concluded are the data available for use. This is in contrast with what takes place with VGI. In a sense, VGI datasets are following one of the main characteristics of Web 2. 0 (O'Reilly, 2007), i. e. perpetual beta. This small phrase is usually applied to software development cycles, and means that there are no versioning cycles but rather a continuous effort of software development so as to match evolving user needs; here this notion spills over to datasets, and OSM is an excellent example for monitoring this. The perpetual editing of and changes to OSM specifications has made OSM evolve from a dataset with a handful of layers and physical features to an extremely detailed dataset, in many cases far more detailed than any NMA or CMC dataset.
The difference between VGI and authoritative data is that in VGI while the evolution of datasets takes place the actual data are available without any guarantees or indications regarding the state or compliance of each feature in relation to a specification's version. It is not difficult to imagine that this process, while it has many advantages, can create a series of inconsistencies and, in fact, deteriorate the overall quality of the data. Thus, while specification improvements might eventually be a necessary step for a better, more inclusive, detailed and meaningful dataset, during the transition time, the dataset is bound to suffer from inconsistencies, mixed feature versions and mixed typologies that exist in former and latter specifications. This is even more likely if there is a perpetual change in specifications without any rigorous provision on how to manage the data transition and compliance.
The Impact of the Contribution Micro-environment on Data Quality 169 Returning the discussion to quality evaluation processes, benchmark comparisons are usually chosen not because they are necessarily the best way to evaluate the data quality of a VGI dataset but because they are the most practical to perform and report. ISO (2005)  explains that benchmark procedures should be based on the establishment of a suitable reporting frequency. Sporadic and non-systematic evaluations, although perfectly acceptable in an academic environment, do not provide a clear view of OSM quality, or of the quality of any other VGI source. To this end, a different approach suggested by the ISO quality framework is to evaluate constantly changing datasets, as is the case of OSM data, using a continuous process.
Methodology To evaluate OSM evolution from a quality point of view, we need to consider what process to use. A way forward is to follow one of the two ISO suggestions. This means that we need to develop a benchmarking method that will be able to examine an instance of the OSM data against an authoritative dataset on a regular basis (e. g. weekly, monthly, etc. ). For a number of reasons, this is not straightforward. First, there is no global-scale authoritative dataset that could play the role of the reference data. Even if such datasets were available for academic research, it is not clear which one would be more detailed and at which places. For example,  Vandecasteele and Devillers (2015)  report that in many places OSM is far more detailed than any authoritative dataset available. Moreover, such an approach would require the implementation of considerable amounts of brute force computing on a regular basis.
General Changes to the Main OSM wiki Page OSM specifications are described in a wiki-based process. The starting point is a MediaWiki 1 web page titled 'Map Features'  (OpenStreetMap, 2016). This page lists all of the physical features that should be included in the OSM database, along with some of the basic attributes that should describe each feature. The OSM community decides what is added or removed from this list through a voting system. In the OSM world, the features are called keys and the attributes values. In the 'Map Features' web page, the physical features are grouped into categories and sub-categories depending on their semantics and nature. For each feature, additional information is available, such as the type of geometry that should be used (i. e. node, way or area), comments on what each feature represents, assisting documentation from Wikipedia, a photograph that shows how the feature appears on the OSM map and a photograph that functions as a photo-interpretation key.
For example, in the wiki-forums it is explicitly stated that the OSM community might introduce best practices, guidelines or even deprecated features and attributes and that nothing is banned. Contributors are free to add whatever they believe will better describe the physical world. Thus, inconsistencies and mismatches in the keys and values used can come from both a 'formal' change in the specifications and the free key/tag combination choice available to users. Interestingly, in the case when changes in the specification are introduced, automatic correction of the existing features is highly discouraged; the rules state: 'Under no circumstances should you automatically (or semi-automatically) change "deprecated" tags to something else in the database on a large scale without conforming to the Automated Edits code of conduct.
The Impact of the Contribution Micro-environment on Data Quality 189 The variety and the large number of OSM editors currently in use indicates the degree of interest in the OSM project. However, this wide range of OSM editors diversifies the data sources and can possibly affect the coherence and homogeneity of the contributions. Indeed, at the time of writing (i. e. May 2016), there were 27 editors available for the OSM community to choose from. This freedom, while in line with the ideology of a crowdsourced project, might undermine the overall effort for a usable dataset of high quality. However, the flip side of this observation might reside in the penetration that selected editors have in the OSM community. Indeed, by examining the statistics from the OSM wiki pages 6 regarding the most popular editors, a more encouraging picture is painted.
Notes 1 Introduction Quality visualisation of geospatial data is as important as the data themselves  (Pang, 2001). The recent development in VGI projects, such as OpenStreetMap (OSM) and Geonames, makes this topic even more critical and challenging, as novice users now access, use and create geographic information. The novice user does not question the quality of VGI data, as he/she is either unaware of the quality issue or erroneously believes that quality problems do not exist in the dataset. The source of geographic data (i. e. VGI vs. proprietary/authoritative) is not perceived as an important factor when determining the credibility of a map  (Parker, 2014). A nicely designed map in terms of cartography and an operational map environment, e. g. OSM, is considered as a reliable source.
Data Indicators In recent studies, a number of data indicators have been proposed and visualised. Two different approaches are observed: indicators can be computed and visualised at the feature level or using grid cells that cover the study area. In the first approach, nodes, points and lines are used. For example,  Trame and Keßler (2011)  visualised the number of versions for OSM POIs (Points of Interest) by using a colour spectral scheme (heat map 2 ) and overlaid the representation onto OSM. In another study  (van Exel, 2011a) , contour lines were used to visualise the average number of version (updates) of any node in the OSM database. Contours of different values were visualised with different hues. Van Exel (2011b) also proposed a combined visualisation of two metrics for the linear OSM features: (i) the time passed since a feature has last been updated by the community is visualised using a hue colour scheme and (ii) the number of versions, indicating how many updates a feature has received since its creation, is visualised using the width of the linear symbol.
VGI Data Quality The nature of VGI datasets -see Chapter 2 by  See et al. (2017)  and Chapter 3 by  Mooney and Minghini (2017)  -and their quality aspects play an important role in the choices regarding visualisation. Past research  (Buttenfield and Beard, 1994; Buttenfield and Weibel, 1988; MacEachren, 1992; MacEachren, 1995)  has proved that the selection of a visualisation method should be related to the quality element represented and the measure/indicator used. The main information that users need about VGI quality focuses on fitness-foruse. Since fitness-for-use depends on a number of quality elements (such as positional accuracy, completeness, currency, etc. ) and on criteria related to the planned use of the data, users may need to be presented with visualisations for a number of data quality measures and indicators in order to reach a decision on the suitability of a dataset.
g. sequential colour schemes, diverging colour schemes, and qualitative colour schemes  (Brewer, 1994; Harrower and Brewer, 2003). Such schemes can be applied in bivariate representations, which depict data and quality together, treating quality as a second variable  (Kunz et al. , 2011; MacEachren et al. , 2005). All intrinsic approaches have in common the fact that slight changes in uncertainty can be difficult to identify, especially for datasets with great variability  (Kunz et al. , 2011). However, this can be mitigated with the help of interactive functionality. Extrinsic techniques  (Howard and MacEachren, 1996) , which introduce new objects to depict quality, e. g. glyphs, grids, etc. , that work independently of the existing symbols for data values, are presented in Table  2. These new objects portray quality using appropriate visual variables such as size, colour value, texture, etc.
Specific attention should be paid to the structure and complexity of such guidelines, best practices and protocols; in particular, they should not discourage citizens from contributing, while simultaneously ensuring that the collected data are of an acceptable quality for the purpose of the specific VGI project. Not secondarily, they should ease or facilitate the reuse of VGI for projects and applications other than the one(s) it was originally collected for. The relevance of establishing protocols in VGI projects and the potential problems for communities and society that arise when these protocols are absent have been highlighted by many authors, including  Sui (2007) , Johnson and Sieber (2013) and  See et al. (2016). In Europe, only a few NMAs have experience with using or integrating VGI in their authoritative datasets  (Olteanu-Raimond et al.
, 2017a) , while protocols for VGI within NMAs, governments or Commercial Mapping Companies (CMCs) are lacking  (Johnson and Sieber, 2013). Conversely, as mentioned above, many authors have developed methodologies to study the quality of VGI (after it has been collected) and have undertaken VGI comparison, integration or conflation with data from NMAs and CMCs to build more up-to-date, accurate and complete datasets  (Girres and Touya, 2010; Haklay, 2010; Ludwig et al. , 2011; Al-Bakri and Fairbairn, 2012; Du et al. , 2012; Pourabdollah et al. , 2013; Touya et al. , 2013; Gao et al. , 2014; Jokar Arsanjani et al. , 2015b; Brovelli et al. , 2016a; Fan et al. , 2016). To instruct users in the production of data that are fit-for-purpose, some VGI projects provide detailed guidelines instead of defining a real protocol.
Open-StreetMap (OSM) 2 is the most popular VGI project and one of the most studied in the literature (Jokar Arsanjani et al. , 2015c); it is extensively described in Chapter 3  (Mooney and Minghini, 2017). Over its more than ten years of life, there has been a progressive development of guidelines about the types of geographic features that users can create and the attributes (or tags) that can be attached to them. The updated version of these guidelines is maintained in a page 3 on the OpenStreetMap Wiki, while their development and enrichment over time is discussed in Chapter 8  (Antoniou and Skopeliti, 2017). It is worth mentioning that, although a real, strict protocol for creating OSM data does not exist and indeed there is considerable freedom left to the contributors, several studies have documented the high quality of OSM crowdsourced datasets (see e.
Similarly, an extensive and detailed protocol for digitising old French maps was created and enriched through user collaboration on a dedicated platform 5 , which allowed for consistent data records to be maintained  (Perret et al. , 2015). In the same way, the GéoPeuple project used protocols to create topographic vector datasets from old French maps for analysing population growth  (Ruas et al. , 2014). The Degree Confluence Project 6 is an example of a project applying a protocol to collect photographs of the landscape from all the intersection points (or confluences) of one degree latitude-longitude around the globe. Volunteers are asked to take either photographs in the four cardinal compass directions (north, south, east, west) or one or more panoramic views from the intersection, one general photograph taken within 100 metres of the confluence, and one photograph of the GPS used.