User interface for evaluating the quality of geospatial data

Graphical user interfaces (GUIs) for reviewing and evaluating geospatial data are provided. A GUI may allow a user to configure a geospatial data quality assessment system by grouping feature classes into one or more categories and selecting target quality scores for the overall data set and/or one or more categories. A GUI may display one or more review areas for a user to review and allow a user to generate one or more error calls in the review areas. A GUI may display one or more scores associated with a geospatial data set. A GUI may allow a user to modify to which category one or more feature classes are assigned. In response to an adjustment of one or more feature classes between categories, the user interfaces may display one or more updated quality scores associated with one or more categories.

FIELD OF THE INVENTION

The present disclosure relates generally to a graphical user interface (GUI) for evaluating the quality of a geospatial data set based on an intended use of the data.

BACKGROUND OF THE INVENTION

Geospatial data is typically generated by identifying features in images captured by aerial sensors, such as cameras in satellites, airplanes, helicopters, unmanned aerial systems, or other aerial platforms. Photogrammetrists or spatial analysts typically view these images and identify desired features. A cartographer or photogrammetrist may identify a feature in an overhead image, such as a building, road, river, or lake, and create the feature in a geospatial data set, often using a software based tool. Several types of errors may arise when geospatial data is generated. For example, features in the image may be overlooked and are not placed in the geospatial data set (omission), or features may be erroneously included in the data set when no such feature exists in the real world (commission). Features placed in the geospatial data set may be subject to other types of error. A feature may be properly identified but placed in the wrong location in the data set (positional accuracy), or a feature may be properly identified and placed in the data set, but may be characterized incorrectly. For example, a feature may be characterized based on an incorrect theme (e.g. a road is incorrectly characterized as a river), or a feature may be characterized as having an incorrect attribute (e.g. a feature may be properly identified as a building, but the analyst may incorrectly classify the building as a school instead of a church).

The quality of a geospatial data set—that is, the quantity and type of errors in the data set—is typically important to creators and consumers of geospatial data sets. However, there are currently no graphical tools that enable users of geospatial data to accurately and objectively characterize the quality of a geospatial data set and assess the quality of the data set against an intended use.

SUMMARY OF THE INVENTION

In some embodiments, a first method, for reviewing quality of displayed geospatial data, is provided, the first method comprising: at an electronic system with a display: displaying a map based on a geospatial data set corresponding to a geographical area; overlaying one or more selectable review areas on the map, wherein the review areas are determined by the electronic system based on probabilities associated with a density of geospatial features associated with the geospatial data set; receiving a selection of a first selectable review area from a user; in response to receiving a selection of a first selectable review area from a user, displaying the first selectable review area; receiving an input from a user indicating that the first selectable review area has been reviewed; and in response to receiving an input from a user indicating that the first selectable review area has been reviewed, displaying the map.

In some embodiments of the first method, the one or more selectable review areas are indicated as unreviewed.

In some embodiments, the first method comprises in response to receiving an input from a user indicating that the first selectable review area has been reviewed, indicating that the first selectable review area has been reviewed.

In some embodiments, the first method comprises: receiving a selection of a second selectable review area from a user; and in response to receiving a selection of a second selectable review area from a user, displaying the second selectable review area.

In some embodiments, the first method comprises receiving an error marker input from a user at a location on the map, wherein an error marker indicates an error associated with a geospatial feature.

In some embodiments, the first method comprises: in response to receiving an error marker input, prompting a user for a description of the error marker.

In some embodiments of the first method, the description comprises feature class information.

In some embodiments of the first method, the description comprises error type information.

In some embodiments, the first method comprises displaying the error marker on the map, wherein the error marker is selectable.

In some embodiments, the first method comprises: receiving a selection of a selectable error marker from a user; and in response to receiving a selection of a selectable error marker, displaying information associated with the error marker.

In some embodiments of the first method, the displayed information comprises feature class information.

In some embodiments of the first method, the displayed information comprises error type information.

In some embodiments, a second method, for reviewing quality of displayed geospatial data, is provided, the second method comprising: at an electronic system with a display: displaying a first review area, wherein the first review area is a region of a map based on a geospatial data set corresponding to a geographical area, and the first review area is determined by the electronic system based on probabilities associated with a density of geospatial features associated with the geospatial data set; receiving an error marker input from a user at a location in the first review area, wherein the error marker indicates an error associated with a geospatial feature; and receiving an input from a user indicating that the first selectable review area has been reviewed.

In some embodiments, the second method comprises: in response to receiving an input from a user indicating that the first selectable review area has been reviewed, displaying the map; and overlaying the first review area and a second selectable review area on the map.

In some embodiments, the second method comprises: receiving a selection of the second selectable review area from a user; and in response to receiving a selection of a second selectable review area from a user, displaying the second selectable review area.

In some embodiments, the second method comprises: in response to receiving an error marker input, prompting a user for a description of the error marker.

In some embodiments of the second method, the description comprises feature class information.

In some embodiments of the second method, the description comprises error type information.

In some embodiments, the second method comprises displaying the error marker on the first review area, wherein the error marker is selectable.

In some embodiments, the second method comprises: receiving a selection of the selectable error marker from a user; and in response to receiving a selection of the selectable error marker, displaying information associated with the error marker.

In some embodiments of the second method, the displayed information comprises feature class information.

In some embodiments of the second method, the displayed information comprises error type information.

In some embodiments, a third method, for evaluating a geospatial data set, is provided, the third method comprising: at an electronic system with a display: displaying a quality score for a geospatial data set, wherein the quality score for the geospatial data set corresponds to an error rate of the data set; displaying one or more categories of selectable feature classes; displaying a quality score for each of the one or more categories of selectable feature classes, wherein the quality score a category of selectable feature classes corresponds to an error rate of feature classes associated with a category; receiving from a user a selection of a selectable feature class associated with a first category to be moved to a second category; and displaying an updated quality score for the first category and the second category.

In some embodiments, the third method comprises: receiving from a user a target quality score for the geospatial data set; and indicating whether the quality score for the geospatial data set is greater than or equal to the target quality score for the geospatial data set.

In some embodiments, the third method comprises: receiving from a user one or more target quality scores for one or more categories of selectable feature classes; and for each category of selectable feature classes, indicating whether the quality score for a category of selectable feature classes is greater than or equal to the target quality score associated with category.

In some embodiments, the third method comprises: displaying one or more categories of error types; and displaying, for each of the one or more categories of selectable feature classes, an amount of errors associated with each of the one or more categories of error types.

In some embodiments, the third method comprises: displaying an updated amount of errors associated with each of the one or more categories of error types for the first category and the second category.

In some embodiments of the third method, each of the one or more categories corresponds to a level of relevance to an intended use of the geospatial data set.

In some embodiments, a first system, for reviewing quality of displayed geospatial data, is provided, the system comprising: a display; one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: displaying a map based on a geospatial data set corresponding to a geographical area; overlaying one or more selectable review areas on the map, wherein the review areas are determined by the electronic system based on probabilities associated with a density of geospatial features associated with the geospatial data set; receiving a selection of a first selectable review area from a user; in response to receiving a selection of a first selectable review area from a user, displaying the first selectable review area; receiving an input from a user indicating that the first selectable review area has been reviewed; and in response to receiving an input from a user indicating that the first selectable review area has been reviewed, displaying the map.

In some embodiments, a second system, for reviewing quality of displayed geospatial data, is provided, the second system comprising: a display; one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: displaying a first review area, wherein the first review area is a region of a map based on a geospatial data set corresponding to a geographical area, and the first review area is determined by the electronic system based on probabilities associated with a density of geospatial features associated with the geospatial data set; receiving an error marker input from a user at a location in the first review area, wherein the error marker indicates an error associated with a geospatial feature; and receiving an input from a user indicating that the first selectable review area has been reviewed.

In some embodiments, a third system, for evaluating a geospatial data set, is provided, the third system comprising: a display; one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: displaying a quality score for a geospatial data set, wherein the quality for the geospatial data set corresponds to an error rate of the data set; displaying one or more categories of selectable feature classes; displaying a quality score for each of the one or more categories of selectable feature classes, wherein the quality score a category of selectable feature classes corresponds to an error rate of feature classes associated with a category; receiving from a user a selection of a selectable feature class associated with a first category to be moved to a second category; and displaying an updated quality score for the first category and the second category.

In some embodiments, a first non-transitory computer-readable storage medium, storing instructions for reviewing quality of displayed geospatial data, is provided, the instructions configured to be executed by one or more processors of a system comprising a display, memory, and one or more processors, the instructions configured to cause the system to: display a map based on a geospatial data set corresponding to a geographical area; overlay one or more selectable review areas on the map, wherein the review areas are determined by the electronic system based on probabilities associated with a density of geospatial features associated with the geospatial data set; receive a selection of a first selectable review area from a user; in response to receiving a selection of a first selectable review area from a user, display the first selectable review area; receiving an input from a user indicating that the first selectable review area has been reviewed; and in response to receiving an input from a user indicating that the first selectable review area has been reviewed, display the map.

In some embodiments, a second non-transitory computer-readable storage medium, storing instructions for reviewing quality of displayed geospatial data, is provided, the instructions configured to be executed by one or more processors of a system comprising a display, memory, and one or more processors, the instructions configured to cause the system to: display a first review area, wherein the first review area is a region of a map based on a geospatial data set corresponding to a geographical area, and the first review area is determined by the electronic system based on probabilities associated with a density of geospatial features associated with the geospatial data set; receive an error marker input from a user at a location in the first review area, wherein the error marker indicates an error associated with a geospatial feature; and receive an input from a user indicating that the first selectable review area has been reviewed.

In some embodiments, a third non-transitory computer-readable storage medium, storing instructions for evaluating a geospatial data set, is provided, the instructions configured to be executed by one or more processors of a system comprising a display, memory, and one or more processors, the instructions configured to cause the system to: display a quality score for a geospatial data set, wherein the quality for the geospatial data set corresponds to an error rate of the data set; display one or more categories of selectable feature classes; display a quality score for each of the one or more categories of selectable feature classes, wherein the quality score a category of selectable feature classes corresponds to an error rate of feature classes associated with a category; receive from a user a selection of a selectable feature class associated with a first category to be moved to a second category; and display an updated quality score for the first category and the second category.

DETAILED DESCRIPTION OF THE INVENTION

As explained above, there is a need for a graphical user interface (GUI) that enables a user to accurately and easily review a geospatial data set and evaluate the quality of the data set based on an intended use of the data. Currently, geospatial data quality is typically determined by a reviewer receiving a data set and looking for errors. If the reviewer finds a certain number of errors, the data set may be rejected. Several problems arise with current methods; the review is inherently biased and reviewers typically do not take into account the intended purpose of the data when performing a review. For example, a data set that has many errors associated with fire hydrants may nonetheless be suitable for a delivery truck driver who only cares about roads. Under existing methods, a geospatial data set may be rejected based on errors that are not relevant to the intended use of the data set.

Current graphical user interfaces for reviewing and evaluating geospatial data sets result in inefficient review and inconsistent results. Geospatial data sets are often too large for a reviewer to review every feature for accuracy. Current methods often do not provide guidance as to how much of a data set should be reviewed in order to accurately characterize the data set as a whole and do not provide guidance to reviewers as how to organize their review. Thus, different reviewers may prioritize their review in different ways, leading to different conclusions about the quality of the data set and causing reviewers to waste time reviewing unnecessary amounts of data and/or unnecessary regions of the data set. For example, one reviewer may focus on urban areas that include a higher density of features and possible errors. Another reviewer may review a dataset based on high interest feature classes (e.g., roads or rivers) and arrive at a different conclusion about the quality of the data set.

These problems may be addressed by the GUIs described herein for reviewing and evaluating a geospatial data set. The GUIs described herein allow for accurate, consistent, and objective review and assessment of a geospatial data set by enabling a user to configure a geospatial data quality assessment system, review selected regions of a geospatial data set, and evaluate the quality of a geospatial data set based on user selectable parameters.

In one or more embodiments, the graphical user interfaces described herein may allow a user to select a geospatial data set for review. The geospatial data set may correspond to a geographic area and may include a plurality of features corresponding to real world objects or locations. Features may be grouped into feature classes. For example, features corresponding to roads may be grouped into one feature class. In one or more embodiments, the user interfaces described herein may enable a user to configure a data quality assessment system by grouping feature classes into one or more categories corresponding to the relevance of a feature class to an intended use of the data set, and selecting one or more target quality scores for the overall data set and/or one or more categories. By assigning feature classes to different categories of importance based on the feature classes' relative importance to an intended use of a geospatial data set, generated quality scores for each category may be applied toward evaluating the viability of the geospatial data set against an intended use.

Based on a statistical analysis of a distribution of features in a geospatial data set, a data quality assessment system may determine one or more regions of the data set to be reviewed for accuracy by a user. In one or more embodiments, the user interfaces described herein may display one or more review areas for a user to review. A review area may be a region of the data set being assessed. In some embodiments, the user interfaces may prompt a user to review each review area. In other embodiments, the user interfaces may allow a user to select which review areas to review and in what order. In some embodiments, the user interfaces may indicate which review areas have been reviewed and which review areas have not been reviewed. The user interfaces may enable a user to generate one or more error calls in the review areas. An error call may correspond to an error in the data set, such as omission, commission, or other error type. The user interfaces may enable a user to select a location associated with an error and input other information associated with an error, such as a feature type, feature ID, and/or other information.

Based on errors identified by a user during review, a data quality assessment system may determine one or more quality scores associated with a geospatial data set. For example, a system may determine an overall quality score for a data set and one or more quality scores for one or more categories of feature classes. The user interfaces described herein may display one or more scores associated with a geospatial data set. The user interfaces may enable a user to modify to which category one or more feature classes are assigned. In response to an adjustment of one or more feature classes between categories, the user interfaces may display one or more updated quality scores associated with one or more categories.

FIG. 1is a diagram of a system100for displaying a graphical user interface for reviewing and evaluating the quality of a geospatial data set, according to some embodiments. In some embodiments, system100can be any suitable type of microprocessor-based device, such as a personal computer, workstation, server, videogame console, or handheld computing device, such as a phone or tablet. The system can include, for example, one or more of processor110, input device120, output device130, storage140, and communication device150. Input device120and output device130can generally correspond to those described above and can either be connectable or integrated with the computer.

In some embodiments, system100can be a computer. System100can be a local computer or a cloud computing platform. System100can include multiple computers. Computers composing system100may be co-located or may be located in different locations. System100can be a host computer connected to a network. In other embodiments, system100can be a client computer or a server.

Input device120can be any suitable device that provides input, such as a touch screen or monitor, keyboard, mouse, or voice-recognition device. Output device130can be any suitable device that provides output, such as a touch screen, monitor, printer, disk drive, or speaker.

Storage140can be any suitable device that provides storage, such as an electrical, magnetic, or optical memory, including a RAM, cache, hard drive, CD-ROM drive, tape drive, or removable storage disk.

Communication device150can include any suitable device capable of transmitting and receiving signals over a network, such as a network interface chip or card. The components of the computer can be connected in any suitable manner, such as via a physical bus or wirelessly.

Storage140can be a non-transitory computer-readable storage medium comprising one or more programs, which, when executed by one or more processors, such as processor110, cause the one or more processors to execute methods described herein, such as method700and/or800described with respect toFIGS. 7 and 8below.

Software160, which can be stored in storage140and executed by processor110, can include, for example, the programming that embodies the functionality of the present disclosure (e.g., as embodied in the systems, computers, servers, and/or devices as described above). In some embodiments, software160can be implemented and executed on a combination of servers such as application servers and database servers.

Software160, or part thereof, can also be stored and/or transported within any computer-readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above, that can fetch and execute instructions associated with the software from the instruction execution system, apparatus, or device. In the context of this disclosure, a computer-readable storage medium can be any medium, such as storage140, that can contain or store programming for use by or in connection with an instruction execution system, apparatus, or device.

Software160can also be propagated within any transport medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above, that can fetch and execute instructions associated with the software from the instruction execution system, apparatus, or device. In the context of this disclosure, a transport medium can be any medium that can communicate, propagate, or transport programming for use by or in connection with an instruction execution system, apparatus, or device. The transport-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, or infrared wired or wireless propagation medium.

Below, graphical user interfaces200-600are discussed with respect toFIGS. 2-6in greater detail. GUIs200-600may, in some embodiments, be interrelated interfaces of a single program or application configured to be used in connection with one another. For example, each of the GUIs200-600may be different screens that are selectively accessible from an application or other computer program configured to work in conjunction with one or more electronic devices in order to review and evaluate a geospatial data set.

FIG. 2is an example diagram that illustrates a GUI200for configuring a geospatial data quality assessment system, according to some embodiments. GUI200may enable a user to select a geospatial data set for review and configure a data quality assessment system. GUI200may enable a user to configure a data quality assessment system by grouping feature class into one or more categories corresponding to the relevance of a feature class to an intended use of the data set and selecting one or more target quality scores for the overall data set and/or one or more categories. By assigning feature classes to different categories of importance based on the feature classes' relative importance to an intended use of a geospatial data set, generated quality scores for each category may be applied toward evaluating the viability of the geospatial data set against an intended use. GUI200may also enable a user to upload a reviewed data set that includes one or more review areas for analysis.

GUI200may include an overall score field210, and one or more feature class categories220a-220c. Overall score field210may include an overall target score field212and an overall actual score field214. Overall target score field210may be an input, such as a text field, drop down menu, or other input, that allows a user to enter a target quality score for a geospatial data set to be evaluated. Actual score field214may be an output configured to display a quality score of a geospatial data set after a geospatial data set has been analyzed.

Feature class categories may correspond to levels of importance of feature classes to an intended use of a geospatial data set. A geospatial data quality assessment system may generate quality scores based on each category. By assigning feature classes to different categories of importance based on the feature classes' relative importance to an intended use of a geospatial data set, generated quality scores for each category may be applied toward evaluating the viability of the geospatial data set against an intended use.

A feature class category, such as feature class category220c, may include a list222of selectable feature classes224a-224g, a category target score field226, and a category actual score field228. Category target score field226may be an input, such as a text field, drop down menu, or other input, that allows a user to enter a target quality score for feature classes included in list222. Category actual score field228may be an output configured to display a quality score for feature classes in list222after a geospatial data set has been analyzed.

In some embodiments, overall actual score fields and category actual score fields may initially be blank. The fields may be populated with quality scores after a geospatial data set has been analyzed, such as by GUI600discussed below with respect toFIG. 6.

Selectable feature classes224a-224gmay be interactive affordances, in that they may be clicked, tapped, pressed, or otherwise selected in order to activate functionality. Once selected, a selectable feature class may be moved between feature class categories, such as by dragging a selected feature class from a first feature class category to second feature class category. A user may move more than one feature class between categories simultaneously, such as by selecting multiple feature classes in a first category and dragging the selected feature classes to a second category.

GUI200may also include a use case field230. Use case field230may be an input, such as a text field, drop-down menu, wheel menu, check-boxes, radio buttons, or other input that allows a user to select a use case from a pre-defined set of use cases. A use case may correspond to a predefined assignment of selectable feature classes to feature class categories and/or a predefined selection of target scores.

In response to receiving a use case input from a user, GUI200may update feature class categories220a-220cbased on the user selection. GUI200may automatically move one or more selectable feature classes from a first feature class category to a second feature class category in accordance with a predefined assignment associated with the selected use case.

In response to receiving a use case input from a user, GUI200may also update one or more target score fields, such as overall target score field210and/or one or more category score fields226. GUI200may automatically display in one or more target score fields a target quality score in accordance with predefined scores associated with the selected use case.

GUI200may also include one or more attributes240-246, such as a company attribute240, user attribute242, a project attribute244, and a data set attribute246. In some embodiments, GUI200may prompt a user to enter and username and/or password. The GUI may then display information associated with the username entered by the user in one or more of attributes240-246. For example, company attribute240may indicate a company name associated with the user's account. User attribute242may indicate a user's name associated with the user account, or other information uniquely identifying a user. Data set attribute246may indicate a geospatial data set to be analyzed. Project attribute244may indicate a project with which a geospatial data set is associated, such as a geospatial data set indicated in data set attribute246. Multiple geospatial data sets may be associated with a project.

GUI200may include a load button250that upon a user's selection causes the GUI to prompt the user to select a first geospatial data set to be analyzed. The GUI may display a second user interface enabling a user to navigate to a storage location containing one or more data files comprising a geospatial data set to be analyzed. In response to the user selection, GUI200may update data set attribute236to indicate the geospatial data set selected by the user.

GUI200may include a download button260that upon a user's selection causes the GUI to prompt the user to select a storage location to which to download a second geospatial data set for review. The GUI may then download the second geospatial data set. In some embodiments, the second geospatial data set may comprise one or more selected areas of the first geospatial data set to be reviewed. In other embodiments, the second geospatial data set may comprise one or more polygons that represent one or more review areas of the first geospatial data set.

GUI200may also include an upload button270that upon a user's selection causes the GUI to prompt the user to upload a reviewed geospatial data set. For example, after a user has used the second geospatial data set to review the first geospatial data set for errors, a user may select upload button270to upload the reviewed data set for analysis. The data set to be uploaded may include one or more errors created by a user during review.

GUI200may also include a save button280that upon a user's selection causes the GUI to prompt the user to select a storage location to which to save a data file storing configuration entered by the user. Stored configuration information may define a new use case that may be selected later by a user. The GUI may then save a file storing configuration information entered by the user and/or the GUI may store configuration information as an entry in a database. For example, the GUI may store one or more target scores entered by the user, such as in overall target score210and/or one or more category target scores226. The GUI may store information indicating to which categories a user has assigned one or more feature classes. Configuration information may be stored in a library of use cases that may be used to automatically configure GUI200, such as by selecting stored configuration information via use case field230.

The stored information may be used later to automatically configure GUI200. For example, the predefined set of use cases associated with use case field230may be updated based on the stored configuration information. The stored information may be selected during future sessions to automatically populate GUI200based on the stored configuration information. Stored use cases may be shared between users within the same organization or in separate organizations to increase consistency of quality assessment between users.

FIG. 3is an example diagram that illustrates a GUI300for reviewing geospatial data, according to some embodiments. Based on a statistical analysis of a distribution of features in a geospatial data set, a data quality assessment system may determine one or more regions of the data set to be reviewed for accuracy by a user. In one or more embodiments, GUI300may display one or more review areas for a user to review. A review area may be a region of the data set being assessed. In some embodiments, GUI300may prompt a user to review each review area. In other embodiments, GUI300may allow a user to select which review areas to review and in what order. In some embodiments, GUI300may indicate which review areas have been reviewed and which review areas have not been reviewed. GUI300may enable a user to generate one or more error calls in the review areas. An error call may correspond to an error in the data set, such as omission, commission, or other error type. GUI300may enable a user to select a location associated with an error and input other information associated with an error, such as a feature type, feature ID, and/or other information.

A user may navigate to one or more review areas and create error calls at locations where errors are identified.

GUI300may include a map310and one or more selectable review areas320a-320c. Map310may correspond to a geographic area. Map310may be based on a geospatial data set to be reviewed, such as selected in response to a user's selection of load button240.The one or more selectable review areas may be overlaid on map310.

GUI300may include a feature class field330. Feature class field330may be an input, such as a text field, drop-down menu, wheel menu, check-boxes, radio buttons, or other input that allows a user to select a feature class from a pre-defined set of feature classes. In response to a user selection of a feature class, GUI300may update map310to display feature associated with the selected feature class.

GUI300may also include an error type field340. Error type field340may be an input, such as a text field, drop-down menu, wheel menu, check-boxes, radio buttons, or other input that allows a user to select a feature class from a pre-defined set of error types. In response to a user selection of an error type, GUI300may update map310to display error calls associated with the selected error type.

GUI300may also include a pause button350. In some embodiments, a data quality assessment system may track an amount of time a user spends reviewing selected review areas of a geospatial data set and/or an amount of time a user spends reviewing each selected review of a geospatial data set. In response to a user selection of pause button350, a system may pause a timer tracking the elapsed review time of a user. In response to a second user selection of pause button350, the system may resume tracking elapsed review time.

GUI300may also include a zoom in button360and a zoom out button362. In response to a user selection of zoom in button360, GUI300may zoom in to a selected region of map310. A zoomed in region may correspond to a review area, or other area of map310. In response to a user selection of zoom out button362, GUI300may zoom out, such as by shrinking a first region displayed in map310and displaying adjacent regions. By zooming out, GUI300may cause one or more review areas to be displayed in map310.

Selectable review areas320a-320cmay be interactive affordances, in that they may be clicked, tapped, pressed, or otherwise selected in order to cause GUI300to display a larger version of the selected review area. In some embodiments, in response to a user selection of a review area, GUI300may display a larger version of the selected review area by zooming in on the selected review area. In some embodiments, in response to a user selection of a review area, GUI300may update map310such that the selected review fills all of map310. In other embodiments, in response to a user selection of a review area, GUI300may update map310such that the selected review fills a portion of map310.

In some embodiments, the one or more selectable review areas may be indicated as unreviewed. For example, unreviewed review areas may be indicated by boundaries having a first color. In some embodiments, after a user has selected a review area, the selected review area may be indicated as reviewed. For example, reviewed review areas may be indicated by boundaries having a second color. In other embodiments, a review area may be indicated as reviewed after it has been displayed by GUI300, such as by filling a majority of map310. In still other embodiments, a review area may be indicated as reviewed in response to a user selection, such as of review complete button370.

In some embodiments, rather than initially displaying substantially all of a geographic area corresponding to a geospatial data set to be reviewed, GUI300may initially display a first review area, such that the first review area fills a majority of map310. In some embodiments, after a first review area has been displayed, the review area may be indicated as reviewed by GUI300.

GUI300may also include a review complete button370. A user may select review complete button370to indicate that the user has finished reviewing a displayed review area. In response to a user selection of review complete button370, GUI300may automatically display a second review area. Alternatively, after a user has reviewed a first review area, a user may view the entire geographic area represented by the geospatial data set and all review areas, such as by selecting zoom out button362. A user may then select a second review area.

FIG. 4is an example diagram that illustrates a GUI400for navigating between review areas of a geospatial data set, according to some embodiments. In some embodiments, GUI400may be overlaid on GUI300.

GUI400includes a frame field410and a plurality of navigation buttons412-418. GUI400includes a first frame button412, a last frame button414, a previous frame button416and a next frame button418. Frame field410may be an input, such as a text field, drop-down menu, wheel menu, check-boxes, radio buttons, or other input that allows a user to select a review area. In response to a user selection, GUI400may update frame field410to indicate the review area currently selected by a user. Review areas may be stored in a predefined order. Navigation buttons412-418may enable a user to select a new review area based on the predefined order of review areas. First frame button412may be an input that, when selected by a user, causes GUI400to display in frame field410an indication that a first review area in the predefined order is currently selected. Similarly, last frame button418may be an input that, when selected by a user, causes GUI400to display in frame field410an indication that a last review area in the predefined order is currently selected. Previous frame button414may be an input that, when selected by a user, causes GUI400to display in frame field410an indication that a review area immediately preceding a current review area in the predefined order is selected. Finally, next frame button416may be an input that, when selected by a user, causes GUI400to display in frame field410an indication that a review area immediately subsequent to a current review area in the predefined order is selected.

GUI400also includes a zoom current button420. In response to a user selection of zoom current button420, GUI300may update map310to display a review area currently selected in frame field410. GUI400also includes a zoom all button430. In response to a user selection of zoom all button420, GUI300may update map310to display a wide view of the geographic area represented by the geospatial data set being reviewed, such that all review areas are displayed in map310.

GUI400also includes a zoom to frame field440. Zoom to frame field440may be an input, such as a check box or radio button. If zoom to frame field440is selected, GUI300may display a selected review frame such that the review area fills all of map310upon a user selection of zoom current button420. If zoom to frame field440is not selected, GUI300may display a selected review frame such that the review area fills a majority, but not all, of map310upon a user selection of zoom current button420.

FIG. 5is an example diagram that illustrates a GUI500for creating an error call in a geospatial data set, according to some embodiments. An error call may indicate an error associated with a feature. In some embodiments, GUI500may be overlaid on GUI300. In other embodiments, GUI500may be displayed in response to receiving an error call input from a user at a location in GUI300, such as on map310. For example, in some embodiments, a user may click, double click, tap, press, or otherwise select a location, such as on map310, to indicate that the user wishes to create an error call at the selected location. In response to a user selection of an error location, GUI500may be displayed to enable the user to create an error call.

GUI500may prompt a user for a description of the error call. GUI500may include a plurality of fields for receiving information about an error call, such as a placement method field510, a feature class field520, an error type field530, a feature ID field540, and a description field550.

Placement method field510may be an input, such as check boxes, radio buttons, or other input that allow a user to select a placement method from a predefined list of placement methods. For example, a placement method may be manual or based on feature selection. When a manual placement is selected, a user may generate an error call at a location in the data set by selecting the location, such as by clicking, tapping, or otherwise selecting the location. When a feature selection placement is selected, a user may generate an error call by selecting a feature, such as by clicking, tapping, or otherwise selecting the feature. In response to selection of a first feature at a location by a user, a user interface, such as GUI300, may highlight the selected feature. A user may confirm the selected feature, such as by left clicking the feature, or reject the feature, such as by right clicking the feature. In response to receiving confirmation of a feature from a user, a user interface, such as GUI300, may display an interface for creating an error call, such as GUI500. In response to receiving a rejection of a feature from a user, a user interface, such as GUI300, may de-highlight the first feature and highlight a second feature at the location, which may be confirmed or rejected by a user. If no additional features exist at the location after a feature has been rejected, all features may be de-selected and de-highlighted.

Feature class field520may be an input, such as a text field, drop down menu, or other input, that allows a user to select a feature class type from a predefined list of feature classes to be associated with an error call.

Similarly, error type field530may be an input, such as a text field, drop down menu, or other input, that allows a user to select an error type from a predefined list of error types to be associated with an error call. For example, a user may select an error of omission, commission, classification correctness, attribute correctness, positional accuracy, or other error type.

Feature ID field540may be an input, such as a text field, drop down menu, or other input, that allows a user to select a unique ID for a feature with which an error is associated. In some embodiments, the ID may correspond to a feature class associated with the error.

Description field550may be an input, such as a text field, that allows a user to input additional information regarding an error call to be created. For example, a user may input a justification for creating an error call or providing additional information, such as linking to online resources.

GUI500may also include a place call button560that when selected by a user causes GUI300to place an error call on map310based on the parameters selected by a user in GUI500.

In some embodiments, placed error calls may be selectable, such as by a user. In response to receiving a selection of a selectable error call by a user, GUI300may display information associated with the selectable error call. For example, GUI300may display feature class information, error type information, feature ID information, or other information associated with a selected error call.

FIG. 6is an example diagram that illustrates a GUI600for evaluating geospatial data quality, according to some embodiments. Based on errors identified by a user during review, a data quality assessment system may determine one or more quality scores associated with a geospatial data set. For example, a system may determine an overall quality score for a data set and one or more quality scores for one or more categories of feature classes. GUI600may display one or more scores associated with a geospatial data set. GUI600may enable a user to modify to which category one or more feature classes are assigned. In response to an adjustment of one or more feature classes between categories, GUI600may display one or more updated quality scores associated with one or more categories. GUI600may also display information indicating an amount of one or more error types are associated with one or more categories of feature classes.

GUI600may correspond to GUI200for configuring a geospatial data quality assessment system. GUI600may be an updated version of GUI200based on results of an analysis of a geospatial data set.

GUI600may include substantially similar features as GUI200. GUI600may include an overall score field610, and one or more feature class categories620a-620c.

Overall score field610may include an overall target score field612and an overall actual score field614. Overall target score field610may be an input, such as a text field, drop down menu, or other input, that allows a user to enter a target quality score for a geospatial data set to be evaluated. Actual score field614may be an output configured to display a quality score associated with a geospatial data set that has been analyzed.

GUI600may be configured to indicate whether an actual score is greater than or equal to an associated target score. For example, if an actual score is greater than or equal to an associated target score, GUI600may display the actual score and/or the associated target score as a first color. If an actual score is less than an associated target score, GUI600may display the actual score and/or the target score as a second color. In response to receiving an updated target score input from a user, GUI600may update an indication associated with the target score and associated actual score. For example, if a user modifies a target score that was less than an actual score to be greater than the actual score, GUI600may so indicate, such as by changing a color associated with the target score and/or actual score.

A feature class category, such as feature class category620c, may include a list622of selectable feature classes624a-224g, a category target score field626, and a category actual score field628. Category target score field626may be an input, such as a text field, drop down menu, or other input, that allows a user to enter a target quality score for feature classes included in list622. Category actual score field628may be an output configured to display a quality score associated with feature classes in list622for a geospatial data set that has been analyzed.

Selectable feature classes624a-624gmay be interactive affordances, in that they may be clicked, tapped, pressed, or otherwise selected in order to activate functionality. Once selected, a selectable feature class may be moved between feature class categories, such as by dragging a selected feature class from a first feature class category to second feature class category. A user may move more than one feature class between categories simultaneously, such as by selecting multiple feature classes in a first category and dragging the selected feature classes to a second category.

In some embodiments, GUI600may update one or more quality score fields in response to a user moving one or more feature classes between categories. For example, in response to a user moving a feature class from a first category to a second category, GUI600may update actual quality scores associated with the first and second category. GUI600may also update one or more indications of whether one or more actual scores is greater than or equal to one or more associated target scores based on the feature class adjustment. For example, if removing a feature class from a first category causes the category's actual score to become greater than the category's target score, GUI600may so indicate, such as by updating a color associated with the actual score and/or the target score.

GUI600may also include a use case field630. Use case field630may be an input, such as a text field, drop-down menu, wheel menu, check-boxes, radio buttons, or other input that allows a user to select a use case from a pre-defined set of use cases. In response to receiving a use case input from a user, GUI600may update feature class categories620a-620cbased on the user selection. GUI600may automatically move one or more selectable feature classes from a first feature class category to a second feature class category in accordance with a predefined assignment associated with the selected use case. In response to receiving a use case selection from a user, GUI600may update one or more quality score fields if the use case selection causes one or more feature classes to be moved from a first category to a second category. GUI600may update one or more quality score fields based on the feature class adjustment. GUI600may also update one or more indications of whether one or more actual scores is greater than or equal to one or more associated target scores in response to receiving a use case selection from a user. For example, if removing a feature class from a first category causes the category's actual score to become greater than the category's target score, GUI600may so indicate, such as by updating a color associated with the actual score and/or the target score.

In response to receiving a use case input from a user, GUI600may also update one or more target score fields, such as overall target score field610and/or one or more category score fields626. GUI600may automatically display in one or more target score fields a target quality score in accordance with predefined scores associated with the selected use case. GUI600may update indications associated with one or more target scores and corresponding actual scores based on the target score adjustments. For example, GUI600may update one or more target scores and associated actual scores to indicate whether the adjusted target score is greater than or equal to an associated actual score.

GUI600may include a report button630that upon a user's selection causes the GUI to prompt the user to select a storage location to which to download an output based on the displayed assessment. The GUI may then download the output. The output may include information corresponding to the data quality assessment. For example, the output may include one or more quality scores and associated target scores, one or more lists of error types found during review of the geospatial data set, and/or one or more items indicating whether one or more quality scores is greater than or less than a corresponding target score.

The output may also include information regarding the review of the selected regions. For example, the output may include a list of errors identified during review and corresponding information, such as error type, location, and/or information regarding the corresponding feature, such as feature class, attribute, or other characteristic. The output may also indicate the identity of the reviewer, the total time spent reviewing, the time spent reviewing each selected region, whether the reviewer visited each selected region, or other information. The output may include the number of errors corresponding to each feature class.

GUI600may also include a load button640that upon a user's selection causes the GUI to prompt the user to select a new geospatial data set for review. In response to receiving a selection of a new geospatial data set to be reviewed, GUI600may reinitiate the data assessment system, such as by removing values from quality score and/or actual score fields, and/or updating one or more feature class categories and associated selectable feature classes, such as based on the selected data set or based on a default setting. In response to receiving a selection of a new geospatial data set to be reviewed, GUI200may be displayed to configure a second analysis.

GUI600may also include an error type field650. Error type field650may include a plurality of error types652a-652eand a plurality of error fields654a-654c. Error types may include omission, commission, thematic accuracy, attribute consistency, positional accuracy, or other error type. Each error field may correspond to an error type and a feature class category. GUI600may display in each error field an amount of errors associated with the error type and feature class category to which the error field corresponds. For example, one or more error fields may be an output configured to display a number of errors associated with a feature class category of the corresponding error type. Additionally or alternatively, one or more error fields may be displayed as a color corresponding to an amount of errors associated with a feature class category of the corresponding error type. In this way, error type field650may visually indicate which error types contribute most to an amount of errors associated with each feature class category.

FIG. 7Ais a first flowchart700that illustrates a method for reviewing quality of displayed geospatial data, according to some embodiments.

At step702, a map based on a geospatial data set may be displayed. The map may correspond to a geographic area represented by the geospatial data set.

At step704, one or more selectable review areas may be overlaid on the map.

At step706, a selection of a first selectable review area may be received.

At step708, in response to receiving a selection of a first selectable review area, the first selectable review area may be displayed.

At step710, an input indicating that the first selectable review area has been reviewed.

At step712, the map may be displayed.

FIG. 7Bis a second flowchart750that illustrates a method for reviewing quality of displayed geospatial data, according to some embodiments.

At step752, a first review area may be displayed.

At step754, an error marker at a location in the first review area may be received.

At step756, an input may be received indicating that the first review area has been reviewed.

FIG. 8is a flowchart that illustrates a method for evaluating a geospatial data set, according to some embodiments.

At step802, a quality score for a geospatial data set may be displayed.

At step804, one or more categories of selectable feature classes may be displayed.

At step806, a quality score may be displayed for each category of selectable feature classes.

At step808, a selection of feature classes to be moved from a first category to a second category may be received.

At step810, an updated quality score for the first and second categories may be displayed.

The foregoing description sets forth exemplary methods, parameters and the like. It should be recognized, however, that such description is not intended as a limitation on the scope of the present disclosure but is instead provided as a description of exemplary embodiments. The illustrative embodiments described above are not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described to best explain the principles of the disclosed techniques and their practical applications. Others skilled in the art are thereby enabled to best utilize the techniques and various embodiments with various modifications as are suited to the particular use contemplated.

Although the disclosure and examples have been fully described with reference to the accompanying figures, it is to be noted that various changes and modifications will become apparent to those skilled in the art. Such changes and modifications are to be understood as being included within the scope of the disclosure and examples as defined by the claims. In the foregoing description of the disclosure and embodiments, reference is made to the accompanying drawings, in which are shown, by way of illustration, specific embodiments that can be practiced. It is to be understood that other embodiments and examples can be practiced, and changes can be made without departing from the scope of the present disclosure.

Although the foregoing description uses terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. In addition, it is also to be understood that the singular forms “a,” “an,” and “the” used in the foregoing description are intended to include the plural forms as well, unless the context clearly indicates otherwise. It is also to be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It is further to be understood that the terms “includes, “including,” “comprises,” and/or “comprising,” when used herein, specify the presence of stated features, integers, steps, operations, elements, components, and/or units but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, units, and/or groups thereof.

The present disclosure also relates to a device for performing the operations herein. This device may be specially constructed for the required purposes, or it may include a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory, computer readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus. Furthermore, the computers referenced in this disclosure may include a single processor or may be architectures employing multiple processor designs for increased computing capability.