dataesr/scientific-paragraphs-categorization huggingface dataset:
Reference Land Use Cover Data Reference data is required to train supervised remote sensing classifiers and to validate LUC maps. Reference LUC datasets consist of a series of geographically distributed sample points with LUC information. Each point contains information about the specific land use or cover in the pixel or polygon of the Earth's surface represented by the point. The reference datasets are subject to the same spatial abstraction required in LUC maps. Reference points are associated with a specific pixel or polygon. The level of abstraction required varies depending on the size of these points. The uncertainty of the reference information will also vary accordingly. The fact that a single land use or cover is assigned to a whole pixel or polygon, even though they may contain other land uses or covers, can also produce uncertainty.
In addition, there is always a degree of subjectivity in the decision to assign a pixel or polygon to a particular category, especially in borderline cases that are not clear-cut. This can create an additional source of uncertainty. Relatively few general LUC reference datasets are currently available. This is because many reference datasets were created ad hoc every time a new LUC map was validated or reference data was required to train a remote sensing classifier, and it was therefore unnecessary to have a ready supply of general LUC reference datasets. These datasets are also affected by some degree of thematic generalization, as is any LUC map. LUC information must conform to a specific classification system or legend. Given the ad hoc nature of many reference datasets, the classification or legend used to classify the land uses and covers was normally also case-specific.
However, the recent emergence of standard LUC reference datasets aimed at a wide range of users and research fields has extended the use of standard legends and classification systems, such as the FAO LCCS, when drawing up these datasets. One of the most renowned LUC reference datasets is the Land Use Cover Area frame Sample (LUCAS), produced by EUROSTAT every 3 years since 2006. It is made up of more than 330,000 survey points across the EU. 3  An increasing number of countries have taken part in every new version of the survey. Of all the LUC reference datasets available, this is the most comprehensive. For each point, experts collect information about land uses, land covers and other relevant environmental parameters. LUCAS also includes four photographs for each surveyed point. It is the only LUC reference dataset reviewed that provides a coherent time series of data for different years.
In recent years, various reference datasets used to validate and train classifiers of global LUC maps have been made available online, so enabling them to be used for other purposes rather than just in the production of one specific map. The work done by the team from the GOFC-GOLD Land Cover Office is of special note. They collected and improved the reference datasets from six different LUC products (GLC2000, GlobCover 2005, STEP, VIIRS, GLCNMO and the urban dataset from the University of Tokyo). Samples of these datasets (with up to 70% of all the available reference points) are freely available for download on the project website. 4 here is a growing trend to gather reference data through crowdsourcing and volunteering initiatives. Information gathered in this way is often referred to as Volunteered Geographic Information (VGI) and is part of citizen science.
5 It produced a global LUC reference dataset over four campaigns  (Fritz et al. 2017). Sahariah et al. (2017)  also produced a global LUC reference dataset for cropland land covers using Geo-Wiki and crowdsourcing. Both datasets are available online for any user interested in the PANGEA repository. 6 he Australian Terrestrial Ecosystem Research Network (TERN) has developed a specific Geo-Wiki application to validate Australian LUC maps: AusCover. 7  Also associated with Geo-Wiki, the LACO-wiki platform provides another tool for the collection of LUC reference datasets. 8 Users can easily validate their own LUC maps on this platform, which includes a repository of reference data created or hosted by the community. It is a very comprehensive, user-friendly tool for LUC reference data production and LUC map validation, which has outperformed the capabilities of Geo-Wiki for this specific task.
Many other tools and platforms have been developed in recent years with similar purposes: Collect Earth, GLFC LT, VIEW-IT…  (Bey et al. 2016). However, although these platforms offer the tools required to create LUC reference datasets through crowdsourcing, many of these datasets are not made available online. Even in the platforms based on crowdsourced information, the LUC reference data remains very case-specific and is not disseminated, so preventing its reuse in other situations. Although they cannot be considered LUC data as such, volunteered geo-referenced photographs may be useful for obtaining reference LUC datasets. They provide a fixed picture of a landscape at a given point in time. By analysing the picture, users can identify the dominant land cover or land use, so obtaining LUC reference data.
Further Reading Fonte CC, Bastin L,  See L, et al. (2015)  Usability of VGI for validation of land cover maps. Int J Geogr Inf Sci 29:1269-1291. https://doi. org/10. 1080/13658816. 2015. 1018266 This paper reviews the main platforms and sources available for volunteer-based collection of LUC reference data and other information that may be useful for producing datasets of this kind. It also discusses the pros and cons of this approach for obtaining reference LUC data. This paper summarizes the state of the art on global LUC mapping. It identifies the areas where most progress has been made in the field, referring in particular to the products with greater spatial detail and more frequent temporal information; the increasing importance of validation; the progressive implementation of the FAO Land Cover Classification System (LCCS) framework as the standard LUC classification method; and the increasing interest in citizen engagement.
(2015)  Harnessing the power of volunteers, the internet and Google Earth to collect and validate global spatial information using Geo-Wiki. Technol Forecast Soc Change 98:324-335. https://doi. org/10. 1016/j. techfore. 2015. 03. 002  Good description of the Geo-Wiki platform, its history, evolution and current capabilities. It also reviews some of the LUC reference datasets based on information collected through the platform. Tsendbazar NE, de Bruin S, Herold M (2015) Assessing global land cover reference datasets for different user communities. ISPRS J Photogramm Remote Sens 103:93-114. https://doi. org/10. 1016/j. isprsjprs. 2014. 02. 008 The paper compares and analyses 12 LUC reference datasets in detail. These datasets are used in the production and validation of global LUC maps. This is one of the most comprehensive reviews of the LUC reference datasets currently available.
As a result, they tend to conceive the MMU in terms of pixel size. From this perspective, it is generally accepted that the smallest observable feature in the final map, i. e. the MMU, should comprise at least four contiguous pixels  (NOAA 2011). When it comes to determining the MMU of LUC maps, it is important to differentiate between databases and maps. Patches that might be a suitable size for data analysis could be completely inappropriate for map publishing. Single pixels or small groups of pixels forming small areas below the MMU threshold might be considered in data analysis, but would not appear on the map. Three intrinsic characteristics of LUC mapping must be taken into consideration when deciding the most appropriate MMU: (i) Confusion between use and coverage, (ii) Definition of land use categories and associated land size, and (iii) High sensitivity of LUC maps to the interrelations between MMU and scale.
Abstract We present an approach that is widely used in the field of remote sensing for the validation of single LUC maps. Unlike other chapters in this book, where maps are validated by comparison with other maps with better resolution and/or quality, this approach requires a ground sample dataset, i. e. a set of sites where LUC can be observed in the field or interpreted from high-resolution imagery. Map error is assessed using techniques based on statistical sampling. In general terms, in this approach, the accuracy of single LUC maps is assessed by comparing the thematic map against the reference data and measuring the agreement between the two. When assessing thematic accuracy, three stages can be identified: the design of the sample, the design of the response, and the estimation and analysis protocols.
An editable attribute table is also created to classify the samples. In this exercise, we will be using MapAccurAssess, a plugin developed by the authors of this chapter, which includes several of the suggestions proposed by  Olofsson et al. (2013) and Finegold et al. (2016). It is available at https://doi. org/10. 5281/zenodo. 5419130 with its associated documentation. For more information on the plugin, readers are referred to Chapter "About This Book". This plugin provides several functions for calculating sample size in a stratified random design, using Neyman's optimal allocation to calculate the number of sampling sites in each thematic class. If, after that, any class has less than 50 sampling sites, it must be assigned between 50 and 100 sites depending on the complexity of the map.
8 ). Step 8 If you have completed Steps 1-8 successfully, you can now start photointerpreting high-resolution satellite images. The exercise involves identifying the predominant land-cover type in each sampling unit and recording the corresponding code in the "Refer_data" column of the attribute table (Fig. 9 ). The meanings of the codes are described in the auxiliary data distributed with the Marqués de Comillas LUC map, available at https://doi. org/10. 5281/ zenodo. 5418318. Photointerpreting all the spatial units of the sample can be a lengthy process, so we suggest that you try to photointerpret at least 10 to 20 spatial units and then compare your results with the "Photo-interpreted reference dataset-Marqués de Comillas 2019", a reference dataset was prepared by the authors. It is available, together with all book's data, at https://doi.
References Cochran  WG (1977)   The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. In this chapter, we describe the fundamental principles and the normal procedure followed when cross-tabulating two datasets. Cross-Tabulation analysis (Sect. 1) is usually the first step in the validation of Land Use Cover (LUC) data. It compares two datasets to observe their spatial relationship, i. e. their degree of spatial (dis) agreement. Results are usually displayed in the form of maps, tables and other statistical measures.
Multiple-Resolution Cross-Tabulation (Sect. 2) compares two raster datasets at multiple spatial resolutions. Basic Cross-Tabulation can compare raster and vector data, while Multiple-Resolution Cross-Tabulation only works with raster data, which is what we use in the exercises provided as examples. In the exercises, raster data were obtained from vector data previously rasterized at different spatial resolutions. As a reference we use LUC maps, although ground points could also be used as reference data for these analyses. Examples of Cross-Tabulation analyses at one and multiple resolutions are presented for four different cases: to validate single LUC maps, to validate the soft maps produced by a model, to validate a simulation exercise and to validate and study land change in a series of LUC maps.
In the example exercises, we used CORINE and SIOSE maps from the Asturias Central Area database, as well as maps from the modelling exercises carried out with this database. In the Chapters "Metrics Based on a Cross-Tabulation Matrix to Validate Land Use Cover Maps" and "Pontius Jr. Methods Based on a Cross-Tabulation Matrix to Validate Land Use Cover Maps", we focus on specific analyses that can be carried out on the basis of Cross-Tabulation analyses, such as Land Use Cover Changes (LUCC) Budget or Quantity and Allocation disagreement. These help unleash the full potential of Cross-Tabulation analysis.
Keywords Cross-Tabulation Á Multiple-Resolution Á Land Use Cover data Á Validation 1 Basic Cross-Tabulation Description Cross-Tabulation is a primary analysis that crosses two datasets, either raster or vector, to analyse their spatial relation. This analysis combines the datasets in spatial terms. It produces a map or table that shows how the values of one dataset spatially relate with the values in the other, thereby informing us as to whether the two datasets share the same values at a given location and, if not, with which other values they have established a relation.
QGIS Exercises The methods and techniques presented in Chapter "Pontius Jr. Methods Based on a Cross-Tabulation Matrix to Validate Land Use Cover Maps" (e. g. LUCC Budget, Intensity Analysis, Quantity and Allocation disagreement…) are based on this basic Cross-Tabulation analysis. In this chapter, we will therefore only be describing the fundamental principles and the normal procedure followed when performing a Cross-Tabulation between two datasets. QGIS includes many tools for cross-tabulating spatial data through the associated GRASS and SAGA models. The "Semi-Automatic Classification Plugin" also includes tools to cross-tabulate datasets for different purposes. Table  1  includes a review of the available Cross-Tabulation tools in QGIS. It provides information of the input and output parameters in each tool.
3  illustrates the general pattern of disagreement areas, it does not provide much information about the particular characteristics of the disagreement between the two datasets. For a better understanding of how similar/different CORINE is from SIOSE, other maps must be drawn up. With the obtained raster, we can for example represent where the urban fabric of CORINE (  2 ) confuses with other classes in SIOSE. We can even detail which classes of SIOSE are affected. To do so, we must first identify the codes (ErrMa-trixCode) for the combinations we are looking for, i. e. pixels which are urban fabric in CORINE (Classified is 2) and which belong to any other category in SIOSE (Reference is not 2). These are codes  3, 16, 42, 55, 68, 81, 94, 107, 120, 133, 146 and 159. We can also represent the pixels that both CORINE and SIOSE label as urban fabric (code 29).
The resulting map can be seen in Fig. 4. This map shows the city of Oviedo and its immediate surrounding area. Most of the city is identified as urban fabric in both SIOSE and CORINE. However, CORINE also labels as urban fabric many small patches that SIOSE identifies, for example, as industrial or commercial areas or artificial green urban areas. This disagreement is to be expected given the different Minimum Mapping Units (MMU) and Minimum Mapping Widths (MMW) of both databases. The MMU used in CORINE is 25 ha, whereas in SIOSE it is only 0. 5-2 ha. The result is that many of the small patches inside the city that SIOSE identifies as other classes are classified as urban fabric in CORINE because of the scale at which this map was made. CORINE offers a much more generalized picture of the landscape to be mapped out.
In this way, these techniques can provide useful, complementary information on this question. However, users can compare the dataset at as many different resolutions as they deem fit. These must always be coarser than the original spatial resolution. The concept of spatial resolution refers to the level of spatial detail available in the spatial data. It applies to data in raster format, where the spatial resolution is defined by the pixel size. This means that, unlike basic Cross-Tabulation, this analysis can only be performed with raster data.
We do this via the option "Pre-initiate the output image with value [optional]", available under the "Advanced parameters" options (Fig. 20 ). Our background value (12) will also be the nodata value of our raster. We can assign a nodata value for the raster we are going to create using the option "Assign a specified nodata value to output bands [optional]" (Fig. 19 ). Step 2 Once we have finished the first rasterization, we must repeat the same procedure for the other three spatial resolutions that we need for the SIOSE dataset. Then, we must repeat the whole workflow for the CORINE map. Once all these tasks have been completed, we will have 8 different maps (4 SIOSE and 4 CORINE) at 4 different spatial resolutions (25, 50, 75 and 100 m). Step 3 Once all the maps have been created, we can start the Cross-Tabulation.
Section 3 introduces the Kappa indices, which allow us to assess the agreement between two datasets, given the agreement expected by random coincidence. We used the indices to compare observed or simulated maps with a reference map. In Sect. 4 we evaluate the agreement between maps at a global level (the entire map) by focusing on a specific feature such as a smaller area or a particular category (stratum level). Finally, in Sect. where A 1 and A 2 are the category areas in question at dates 1 and 2, respectively. AC can be divided by the number of years between the two dates to obtain the average annual change area over the study period. Relative change (RC) is obtained by normalizing the absolute change value by the category area at date 1. RC ¼ A 2 À A 1 ð Þ =A 1 This formula expresses the proportion of the category area that changed over the study period.
Execution Step 1 Install the MapAccurAssess plugin. Should you need help, please see Chapter "About This Book" and the plugin's documentation. Step 2 If the plugin has been successfully installed, an icon should appear in the main graphics panel. To start the exercise, click on this icon. Alternatively, go to the Complements menu, select Accuracy Assessment and then Accuracy Assessment again. Step 3 Select the shapefile with the reference samples (Photo-interpreted reference dataset-Marqués de Comillas 2019) 2 and indicate the column with the reference data and the column with the values for the thematic classes used in the map. After that, select the land cover map you want to assess (Marqués de Comillas Land Use Land Cover Map 2019). If the map is in vector format, indicate the column containing the thematic class values.
Aim To compare and validate the change simulated by two models. For this purpose, we overlay three maps that show change versus non-change over the same interval. The initial map in all cases is the CORINE dataset for 2005. The changes from 2005 to 2011 are calculated for the simulation from model 1, for the simulation from model 2 and for the CORINE dataset as reference. WRONG HITS are not evaluated in this exercise.
We use the Ariège database to validate the different soft maps (change potential and suitability maps) produced by the model by comparing them with real land use maps of the Ariège Valley for two dates  (CORINE 2012 and 2018). All these validation techniques are carried out using raster data. As commented earlier, the soft maps produced by the model are continuous, ranked variables. We designed exercises using this original format. In other chapters of this book, the soft maps produced by the model are validated after reclassification of the original maps.
Values closer to 1 indicate simple geometries, more closely resembling a square, whereas values closer to 2 indicate more complex geometries, which are less like the simple shape of a square. Contrary to what might be expected, and with the exception of the port areas (Category 9 after reclassification), patch shapes were more complex in CORINE than SIOSE. This seems illogical given that SIOSE is made at a finer scale (1:25,000) than CORINE (1:100,000) and delimits land use areas more accurately. In our case, CORINE has more complex patch shapes than SIOSE because of the rasterization of the CORINE and SIOSE vector databases, which reduced the complexity of the SIOSE polygons, resulting in more regular shapes. Finally, "Like adjacencies" and the "Patch cohesion index" (Table  4 ) provide information about the compactness of the categories in a map.
Values closer to 0 mean than the patches belonging to a particular category are very scattered. Values closer to 1 (Like adjacencies) or to 10 (Patch cohesion index) indicate that they are tightly clustered. The "Like adjacencies" metric is based on the number of adjacencies between pixels, whereas the "Patch cohesion index" is obtained by calculating the ratio between the area and the perimeter of the patches. This means that although they provide information on a similar subject (compactness), they complement each other. These metrics show that land uses are represented in a more compact (more clustered) manner in the CORINE database. This makes sense because of the lower degree of fragmentation and the greater generalization of CORINE compared to SIOSE. All in all, even if important differences between the two maps could be identified in terms of landscape configuration, most of these are due to the different criteria used in the drawing of each map.
2 focuses specifically on the changes that take place on pattern borders. This enables changes to be divided into those that take place on the borders of existing features and those that form new, disconnected features. Bringing this chapter on landscape patterns to a close, Sect. 3 presents a technique for quantifying allocation errors in simulation maps and more precisely on the minimum distance between the allocation errors in simulation maps and the nearest patch belonging to the same category on the reference map. The comparison between a raster-based and a vector-based approach brings us back to the differences in measurement inherent in the representation of entities in raster and vector mode. These techniques are applied to two datasets. Section 1 uses the Asturias Central Area database, where CORINE maps are compared to SIOSE maps and simulation outputs.
Map Curves Description This is a quantitative method proposed by  Hargrove et al. (2006)  to evaluate the spatial concordance between different categorical raster or vector datasets. It calculates the Goodness of Fit (GOF) (Fig. 1 ), a standard metric that evaluates the spatial concordance between the patches of two or more rasters or the polygons of two or more vectors. Unlike other methods, it does not evaluate spatial agreement at cell level, and instead focuses on agreement at patch level in rasters or at polygon level in vectors. Consequently, this method is independent of spatial resolution. GOF values range from 0 to 1. Maximum GOF (  1 ) is obtained when there is full overlap between two polygons or patches. If there is no overlap, GOF is 0. If overlap affects half the area of the polygons or patches, GOF will be 0.
5. When comparing pairs of maps, the GOF value may vary depending on whether the assessed map is evaluated against the reference map or the reference map is evaluated against the assessed map. Map Curves calculates the GOF values for both these operations. It then uses the highest of these two GOF values in the comparison. GOF values may be obtained either for the whole dataset or for the set of patches or polygons that make up each category on the map. Although it is technically possible to calculate a GOF for each individual polygon or patch, it is computationally very demanding and is not normally done. Based on the GOF metrics at the category level, the results of the map comparison may be expressed in a graph, which shows the percentage of the categories in the map that have a specific GOF value.