# Heuristic Evaluation of a Publicly Accessible Geoprocessing Platform: Categories of Usability Issues

Bruna Santana Capeleti, Juliana França, André Pimenta Freire

► To cite this version:

Bruna Santana Capeleti, Juliana França, André Pimenta Freire. Heuristic Evaluation of a Publicly Accessible Geoprocessing Platform: Categories of Usability Issues. 19th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2023, York, United Kingdom. pp.123-131, 10.1007/978-3-031-61698-3_12. hal-05305347

HAL Id: hal-05305347

https://inria.hal.science/hal-05305347v1

Submitted on 9 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Heuristic Evaluation of a Publicly Accessible Geoprocessing Platform: Categories of Usability Issues

Bruna Santana Capeleti<fnref n="1" />(✉), Juliana Baptista dos Santos França<fnref n="2" />(✉), and André Pimenta Freire<fnref n="1" />(✉)

**Abstract.** The engagement with geoprocessing data has witnessed a substantial and continuous growth. As a response to this expansion, diverse sectors have commenced presenting their information through publicly accessible platforms that cater to a wide range of user profiles, encompassing experts and non-experts alike. In light of this prevailing trend, the objective of this research is to conduct a heuristic evaluation of a publicly accessible application by the Brazilian government, with a particular focus on geoprocessing. The primary aim of this evaluation is to identify and categorize the issues encountered by specialists during the assessment process. Consequently, this study aims to perform a heuristic evaluation of a Brazilian government's public application that is openly accessible. The study's findings will be used to indicate the various categories of issues encountered in the evaluations. This research constitutes a crucial component of a broader investigative effort that explores diverse approaches to identifying common categories of issues among various geospatial applications. Ultimately, the results derived from this heuristic evaluation, alongside the broader research, are expected to make significant contributions to the improvement of geospatial applications.

**Keywords:** Human-data interaction · Geoprocessing · Usability Evaluation

## 1 Introduction

The increasing frequency of data interaction across diverse contexts highlights a pressing need to assess the quality of user interactions and interpretations when engaging with data. In addition, geoprocessing has witnessed a notable upsurge in prominence in recent years owing to its remarkable capabilities in representing distances on maps, facilitating route finding, and enabling various related functionalities. To ensure effective data analysis within geoprocessing systems, the implementation of appropriate processes to enhance interactive technologies

<footnote n="1">¹ Universidade Federal de Lavras, Lavras, Brazil</footnote>

<footnote>bruna.capeleti2@estudante.ufla.br, apfreire@ufla.br</footnote>

<footnote n="2">² Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil</footnote>

<footnote>julianabsf@ic.ufrj.br</footnote>
becomes imperative. This necessitates a comprehensive consideration of user-centered approaches and strategies designed to address the inherent complexities involved.

According to the research conducted by Ziegler et al. [11], geospatial data assumes an increasingly crucial role in the endeavors of Earth and climate scientists, social scientists, and data journalists, as they strive to analyze spatiotemporal changes in our environment and societies. The authors further argue that technologies involving geoprocessing present significant challenges, given their inherent difficulty in usage and the substantial effort required for proficiency.

The approach employed in this study entails conducting heuristic evaluation with the participation of three experts in human-computer interaction. Existing heuristics will be applied to evaluate a public application that is specifically focused on the planning of mineral research and production.

Based on the results obtained from the heuristic evaluation, the identified usability issues will be analyzed and correlated with some of the heuristics present in the literature. This process will culminate in the development of categories to illustrate the usability problems encountered within this context.

This study constitutes an integral component of a broader master's research, with the primary aim of discerning the limitations of existing approaches within the heuristics already documented in the literature concerning the field of geo-processing. By doing so, this research seeks to contribute valuable insights to the field and foster advancements in the usability and efficiency of geoprocessing systems.

# 2 Theoretical Background

The purpose of this section is to provide the references pertaining to the research theme, along with presenting related works that employ similar approaches.

## 2.1 Human-Data Interaction

Recently, the definition of the concept of human-data interaction has been revised. In the past, when databases were not widely used, and new tools for manipulating data had not yet emerged, the concept of human-data interaction was limited to the data itself, without taking into account the user who interacted with it. However, with the advent of new data manipulation tools and their evolution, the study of human-data interaction has shifted its focus towards user interaction and interpretation. This has resulted in a departure from the notion that data is the sole central element of the interaction [10].

As stated by Knaflic [2], in the realm of human-data interaction, it is crucial to establish the intended audience for a presentation of data. This serves as the foundation for creating a context in which the data will be presented. Furthermore, it is imperative to provide a clear explanation of the data being presented, while avoiding the overloading of information to prevent user confusion. This information is of great significance in the context of evaluated data since users will typically access it without any form of assistance or supervision.
The interaction between humans and data enables the acquisition of knowl-
edge on a variety of topics through the use of specialized tools that support the
complete lifecycle of data, allowing for user access and comprehension [7].

## 2.2 Usability in Geoprocessing Context

The study conducted by Ziegler et al. [11] aimed to gain insights into the chal-
lenges confronted by users of geospatial data. To achieve this, a contextual
inquiry study was carried out, involving 25 participants from various sectors
such as academia, industry, newsrooms, and the public sector. These partici-
pants were actively engaged in utilizing geospatial data in their respective work.
The primary objective of the study was to identify the unfulfilled computing
requirements of the diverse and expanding community of geospatial data users.
Based on their findings, the authors presented six design opportunities.

According to Nakic et al. [5], "visualization of geographic data is part of many widely used solutions that aim to communicate the information to the end user". The authors additionally note that in the context of visualization, data is frequently represented through color schemes or ranges. Furthermore, within the Internet of Things context, the Geographic Information System (GIS) is increasingly significant.

Kubiček et al. [3] make reference in their publication to “new data sources
as well new visualization methods are re-opening the discussion about the user’s
ability to read the map and interpret its content in a coherent way”. The authors
also discuss the challenges inherent in representing and comprehending data in
the context of geographic information.

According to Holodinsky et al. [1], “Geographic visualizations have been used
to understand disease since the nineteenth century”. Therefore, the study aimed
to evaluate the usability of software that creates simple visualizations to sup-
port decision-making, with 18 participants. Only one participant was unable
to correctly interpret the maps, and three topics emerged from the evaluation:
improvement suggestions, user interface comments, and visualization tool per-
ceptions.

The aim of the study conducted by Manson et al. [4] was to investigate
the efficacy of using eye tracking as a tool for measuring the usability of web
mapping applications. The authors assert that the study was instrumental in
demonstrating how eye tracking technology can be employed to enhance the
design and usability of such applications. The study was executed through two
phases of evaluation. In the first phase, participants were recruited to complete
an initial survey, undergo a brief training session, engage in web mapping tasks,
and then respond to follow-up survey questions. In the second phase, a smaller
group of participants completed another round of web mapping tasks, followed
by a second survey.
## 2.3 Related Work

The study conducted by Wei et al. [9] aimed to analyze two research categories: Traditional Cartography and Geo-visualization. The objective was to identify the issues and questions when these tools are used, focusing on comparing aspects related to cognition, adaptability, and usability. The authors found that the category used depends on the context, as the second one evolved from the first. Depending on the purpose of use, the benefits and the aspects mentioned before may vary.

The study conducted by Wei et al. [9] aimed to analyze two research categories: Traditional Cartography and Geo-visualization. The objective was to identify the issues and questions when these tools are used, focusing on comparing aspects related to cognition, adaptability, and usability. The authors found that the category used depends on the context, as the second one evolved from the first. Depending on the purpose of use, the benefits and the aspects mentioned before may vary.

In their study, Victorelli and Reis [8] suggest a collection of heuristics that pertain to the design of components utilizing human-data interaction, specifically:

1. Human-data interaction design guidelines for visualization systems

1.1. Self evidence in coordinated views

1.2. Consistency between coordinated visualizations

1.3. Reversible operations in visualizations

2. Use smooth animated transitions between visualizations states when they can help the user to notice the difference between the data

3. Immediately provide visual feedback on the interaction

4. Maximize direct manipulation with data

5. Minimize information overload

5.1. Show information context

5.2. Avoid requiring data memorization

6. Semantically enrich the interaction

6.1. Semantically enrich search interaction

6.2. Enriched feedback from human incorporated into system

6.3. Refine and train models through user feedback

# 3 Methodology

The aim of this chapter is to outline the methodology that was employed to conduct the research.

## 3.1 Systems Selected for Evaluation

The application selected for the evaluation is “P3M”, designed to support the planning of mineral research and production in Brazil, enabling the visualization of various information presented in layers. Figure 1 presents the map screen.
<figure><img src="image_3.png" /><figcaption>Fig. 1. P3M Environment</figcaption></figure>

The Research and Mineral Production Planning Support Platform (P3M) has been conceived and is being implemented by a team of experts from the Directorate of Geology and Mineral Resources (DGM) of the Geological Survey of Brazil (SGB-CPRM). The process of conceptualization and implementation of P3M is supported and involves the participation of the Secretariat of Geology, Mining, and Mineral Transformation (SGM) of the Ministry of Mines and Energy (MME) and the National Mining Agency (ANM). All data used in this version of P3M are public, open, and freely accessible on the internet through their respective institutions.

## 3.2 Heuristic Evaluation

The evaluation aimed to assess the performance of the data presented by the map and its filters. Interaction with the application was performed and contrasted with usability heuristics proposed by Molich and Nielsen [6] and specific heuristics for human-data interaction proposed by Victorelli and Reis [8].

In order to assess the usability of the application selected, a collaborative heuristic evaluation was conducted, which involved three professionals from the field of user-computer interaction. Additionally, the identified heuristics were categorized into different levels of severity, which are presented below:

- A score of 0 was assigned when the evaluator did not consider the item to be a problem;

- A score of 1 was assigned to cosmetic problems that do not significantly hinder the user's interaction;

- A score of 2 was assigned to simple problems that have a low impact on the user's interaction;
- A score of 3 was assigned to serious problems that have a high impact on the user's interaction;

- A score of 4 was assigned to catastrophic problems that prevent the user from continuing the interaction.

## 3.3 Analysis

Following the completion of the heuristic evaluations, the identified usability issues were associated to existing heuristics in the literature, specifically those related to human-computer interaction [6] and human-data interaction [8]. Additionally, as previously mentioned, the primary objective of this research is to contribute to a broader master's research project. Consequently, the final phase of the analysis focused on ascertaining whether the identified issues could be classified into common categories, aiming to facilitate a more comprehensive visualization of the types of problems encountered in a case study involving heuristic evaluation within a publicly accessible geoprocessing platform.

In order to gain a deeper understanding of the identified usability issues and their potential implications, the categorization was performed by grouping the problems based on their shared characteristics and underlying causes.

This systematic categorization enhances the interpretability of the results and lays the foundation for future research and iterative improvements in the platform's design and usability.

# 4 Results

As stated in the methodology section, the current study aims to assess the usability of a publicly accessible geoprocessing application, wherein the identified issues will be categorized. Furthermore, the identified usability issues were linked to existing heuristics in the literature, as documented by Molich and Nielsen [6] and Victorelli and Reis [8]. This section is intended to present the obtained results.

## 4.1 Results Overview

The P3M application had as result 66 usability problems, related to several items, such as a lack of sequential steps, causing the management of information types, layers, and maps to become disjointed. Consequently, in order for the utilization to align with expectations, the user's interaction model would need to differ from that of other existing geoprocessing applications, even though those applications may not have a particularly straightforward interaction design either.

Regarding the heuristics to which the identified problems were related, there is a direct correlation between them. In Molich and Nielsen's heuristics [6], the heuristic with the highest number of associated problems was number 6 - Recognition rather than recall. Conversely, in Victorelli and Reis's heuristics [8], the heuristic most linked to the identified problems was also number 6 - Semantic enrichment of interaction. Both heuristics address issues concerning the lack of
steps and patterns for correctly performing actions within the platform. Consequently, enhancing the quality of interaction with the data would be achieved through the implementation of interaction patterns, clear steps to be followed for task execution, and more comprehensive descriptions of the platform's functionalities.

The second heuristic with the highest number of associated problems, for the Molich and Nielsen's heuristics [6], was number 1 - Visibility of System Status. Similarly, for Victorelli and Reis's heuristics, it was number 3 - Immediately provide visual feedback on the interaction. These findings underscore the importance of providing appropriate feedback to users, allowing them to orient themselves within the application, understand their progress, and maintain a clear and concise user journey. Properly implemented visual feedback enhances user experience and promotes a sense of control and understanding while interacting with the system.

## 4.2 Categories

The categories of issues identified in the usability evaluation are as follows:

1. **Bugs and spelling errors:** Instances of technical malfunctions and inaccuracies in written content were detected within the platform, potentially impacting the user experience and credibility;

2. **Confusion during platform interaction:** In the evaluation were identified instances of confusion and disorientation while engaging with the platform, indicating a necessity for improved user guidance and intuitively designed interfaces;

3. **Color and map navigation issues:** Problems related to color usage and difficulties in navigating the map interface were identified, highlighting the need for enhanced visual design and improved map interaction mechanisms;

4. **Lack of clarity in item usage:** In the evaluation were encountered challenges in comprehending how to use certain items or features within the platform, emphasizing the requirement for clearer instructions and more intuitive interface designs;

5. **Inadequate comprehension of data significance:** Were identified difficulties in grasping the meaning and context of the presented data, underscoring the importance of improved data visualization and the inclusion of explanatory aids;

6. **Quality of data interpretation:** The interpretation to derive meaningful insights from the presented data was hindered by certain issues, further accentuating the significance of presenting data in a user-friendly and easily understandable manner.

Addressing these usability issues is crucial for enhancing the overall user experience, improving data interpretation, and cultivating a more efficient and satisfying interaction with the geoprocessing platform.
# 5 Conclusion

The objective of this research was to conduct a heuristic evaluation of a publicly accessible geoprocessing application developed by the Brazilian government. The primary aim was to identify usability issues within the platform and subsequently categorize them to discern common patterns, thereby enabling a visualization of areas where improvements can be made.

During the heuristic evaluation, a total of 66 usability problems were identified. Notably, the heuristic with the highest number of associated problems from Molich and Nielsen [6] was number 6 - "Recognition rather than recall". Similarly, in Victorelli and Reis's heuristics [8], the most prevalent heuristic linked to the identified problems was also number 6 - "Semantic enrichment of interaction". Both heuristics specifically address issues pertaining to the lack of clear steps and patterns for correctly executing actions within the platform.

As a result of the evaluation, six distinct categories that represent the types of problems were identified: bugs and spelling errors, confusion during platform interaction, color and map navigation issues, lack of clarity in item usage, inadequate comprehension of data significance, and quality of data interpretation. These categories provide a structured and comprehensive overview of the various usability challenges observed within the geoprocessing application evaluated.

For future works, additional case studies will be conducted with the primary objective of identifying which categories are consistently related to usability problems in the context of geoprocessing. These subsequent studies will contribute to further understanding the usability issues in geoprocessing applications and potentially reveal new patterns and insights. The findings of this research will be presented in a master's dissertation for the computer science master's program.

# References

1. Holodinsky, J.K., Francis, M.J., Goyal, M., Hill, M.D., Kamal, N.: Testing the usability of a software for geospatial and transport modeling in acute stroke service planning. Front. Neurol. **10**, 694 (2019)

2. Knaflic, C.N.: Storytelling with Data: A Data Visualization Guide for Business Professionals. Wiley, Hoboken (2015)

3. Kubíček, P., Báčová, R., Stachoň, Z.: Multivariate data visualization and usability: preliminary notes. In: Brus, J., Vondrakova, A., Vozenilek, V. (eds.) Modern Trends in Cartography. LNGC, pp. 385–394. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-07926-4_29

4. Manson, S.M., Kne, L., Dyke, K.R., Shannon, J., Eria, S.: Using eye-tracking and mouse metrics to test usability of web mapping navigation. Cartogr. Geogr. Inf. Sci. **39**(1), 48–60 (2012)

5. Nakić, J., Kosović, I.N., Franić, A.: User-centered design as a method for engaging users in the development of geovisualization: a use case of temperature visualization. Appl. Sci. **12**(17), 8754 (2022)
6. Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 249–256 (1990)

7. Victorelli, E.Z., Dos Reis, J.C., Hornung, H., Prado, A.B.: Understanding human-data interaction: literature review and recommendations for design. Int. J. Hum. Comput. Stud. **134**, 13–32 (2020)

8. Victorelli, E.Z., Reis, J.C.D.: Human-data interaction design guidelines for visualization systems. In: Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems, pp. 1–10 (2020)

9. Wei, L.L.Y., Ibrahim, A.A.A., Nisar, K., Ismail, Z.I.A., Welch, I.: Survey on geographic visual display techniques in epidemiology: taxonomy and characterization. J. Ind. Inf. Integr. **18**, 100139 (2020)

10. Werman, T.: Human data interaction (HDI): the new information frontier (2021). encurtador.com.br/puEK3

11. Ziegler, P., Chasins, S.E.: A need-finding study with users of geospatial data. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–16 (2023)