# Design of an Attention Tool Using HCI and Work-Related Variables

Patrícia Gamboa, Cláudia Quaresma, Rui Varandas, Helena Canhão, Rute Sousa, Ana Rodrigues, Sofia Jacinto, João Rodrigues, Cátia Cepeda, Hugo Gamboa

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Patrícia Gamboa, Cláudia Quaresma, Rui Varandas, Helena Canhão, Rute Sousa, et al. Design of an Attention Tool Using HCI and Work-Related Variables. 12th Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), Jul 2021, Costa de Caparica, Portugal. pp.262-269, 10.1007/978-3-030-78288-7_25. hal-03685917

HAL Id: hal-03685917

https://inria.hal.science/hal-03685917

Submitted on 2 Jun 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Design of an Attention Tool Using HCI and Work-Related Variables

Patricia Gamboa<fnref n="1" />,<fnref n="2" />, Cláudia Quaresma<fnref n="1" />, Rui Varandas<fnref n="1" />,<fnref n="2" />, Helena Canhão<fnref n="3" />,<fnref n="4" />,
Rute Dinis de Sousa<fnref n="3" />,<fnref n="4" />, Ana Rodrigues<fnref n="3" />,<fnref n="4" />, Sofia Jacinto<fnref n="3" />,<fnref n="4" />,<fnref n="5" />, João Rodrigues<fnref n="1" />,
Cátia Cepeda<fnref n="1" /> and Hugo Gamboa<fnref n="1" />,<fnref n="2" />

{pg.neves@campus.fct.unl.pt, q.claudia@fct.unl.pt, r.varandas@campus.fct.unl.pt, helena.canhao@nms.unl.pt, anamfrodrigues@gmail.com, sofia.jacintobraga@nms.unl.pt, jmd.rodrigues@fct.unl.pt, c.cepeda@campus.fct.unl.pt, hgamboa@fct.unl.pt}

**Abstract.** The project Prevention of Occupational Disorders in Public Administrations based on Artificial Intelligence (PrevOccupAI) aims to identify and characterize profiles of work-related disorders (WRD) and daily working activities profiles. WRD have major impacts on the well-being and quality of life of individuals, as on productivity and absenteeism. Thus, to increase individuals' quality of life and productivity, a tool focusing on human attention is being developed, integrating the insights of workers of AT (Autoridade Tributária) and the literature on attention and time management. By inputting Human-Computer Interaction (HCI) and work-related variables into an Artificial Intelligence (AI) layer, a dashboard system will provide workers' information on causes of loss of focus they may not be aware of. Additionally, another layer will provide Recommendations, such as mindfulness-based tips, to assist in the management of feelings, emotions or work-related concerns, possibly increasing awareness and focus on the present. This manuscript presents the preliminary design of this tool.

**Keywords:** Artificial Intelligence, HCI, attention, work-related disorders, occupational health

## 1 Introduction

Mental health and musculoskeletal disorders are currently the most prevalent work-related disorders (WRD) [1], impacting the quality of life of workers, and the organizational productivity and absenteeism [2]. Occupational hazards continue to be a cause of disorders, with an effect on mortality worldwide, even with the significant improvements due to technologies and measures introduced over the last decades in

<footnote n="1">¹ Departamento de Física, LIBPhys, FCT- Universidade Nova de Lisboa, Portugal</footnote>

<footnote n="2">² PLUX Wireless Biosignals S.A., Portugal</footnote>

<footnote n="3">³ Comprehensive Health Research Centre, NOVA Medical School, Portugal</footnote>

<footnote n="4">⁴ EpiDoC Unit, CEDOC, NOVA Medical School, Portugal</footnote>

<footnote n="5">⁵ ISCTE, CIS, IUL, Portugal</footnote>
workplaces [2]. Presently, cognitive demands of working tasks are high due to constant interruptions, ambient distractions, information overload, etc. [3]. The literature available on these topics has underlined that providing meaningful feedback to workers based on their personal data and delivering recommendations on ways to improve work efficacy and well-being contribute to reducing risk factors of WRD related to mental health difficulties and disorders. To the best of our knowledge, there is a lack of scientifically validated tools able to collect variables related to attention management at office environments, coupled with the delivery of personalised feedback and recommendations.

This work is part of the project Prevention of Occupational Disorders in Public Administrations based on Artificial Intelligence (PrevOccupAI) that aims to identify and characterize profiles of work-related disorders (WRD) and daily working activities profiles of individuals working in the Public Administration, in particular AT (*Autoridade Tributária*). In this project, both ergonomic and mental health risk factors will be assessed by the group of researchers, though solely the latter will be discussed in this manuscript.

The present paper is structured as follows: a brief literature review (Section 2) that includes the role of attention and time management and links to work performance and well-being; and the role of feedback and recommendations in the enhancement of those variables; the framing of the current work with AI systems (Section 3); the presentation of the concept of an attention promotion tool to assist public office workers and its development roadmap (Section 4) through: (i) the selection of measures to gather personal data from office workers and definition of variables; (ii) a *focus group* to explore with the workers, the measures selected so the feedback provided is meaningful and personalised, promoting workers’ self-awareness and self-reflection on their work-related tasks; and to explore the display of personalised tips to improve well-being and work performance; (iii) the validation of the tool; (4) a brief conclusion and orientations regarding future work (Section 5).

# 2 Literature Review

## 2.1 Attention and Time Management at Work

Working implies to rely on executive functions, a set of mental processes that include three core functions: inhibition control, working memory and cognitive flexibility [4]. Selective or focused attention (one of the mechanisms of inhibitory control), in particular, enables individuals to attend to selected stimuli while suppressing attention to other stimuli [4]. Some salient stimuli attract attention in an involuntary way, such as loud noises, while others in a more voluntary way, based on personal goals or intention [4]. In work performance, attention enables individuals to pursue goals without getting distracted by alternative stimuli available in physical and virtual environments [5]. However, constant interruptions, ambient distractions (e.g., doorbell rings in home office, nowadays more common due to the COVID-19 pandemic [6]), and information overload, are factors that contribute to create
cognitive straining conditions, impairing task performance and diminishing the well-being at work [3]. Indeed, work stress is at the top of the most commonly reported causes of WRD and loss of work performance [7].

WRD, also named occupational disorders, are caused primarily as a consequence of exposure to risk factors related to a work activity or job [1]. They have multiple causes, with work environment variables (e.g., physical and psychosocial) playing a role, together with other risk factors, such as personal determinants related to demographic variables and medical history [1]. The most prevalent WRD worldwide are musculoskeletal and mental health disorders (e.g., [8]) which, in recent studies [9],[10], accounted for around 30% and 50% of occupational reports, respectively. Moreover, work content, work relationships and work pace represented 40% of the top risks identified [9].

Therefore, it is essential to manage cognitively straining factors and reduce their harmful repercussions for individual workers, organizations, and society [3]. The promotion of a healthy working environment must include temporal, spatial and mental separation of work and private life components [11]. This can be done by implementing time management strategies such as organizing the working time, including breaks and quiet moments [12], and through the creation of an ergonomic workplace and a working environment with reduced distractions [11]. In order to foster attention and time management, solutions such as browser extensions and mobile apps are available. For instance, TogglTrack [13] allows individuals to track the time spent in activities such as emails, websites, etc. and block predefined websites such as social media. Other solutions set limits to websites (e.g., RescueTime [14]) or remind individuals to take breaks (e.g., Time Out [15]). Within the context of our project, these time management strategies will be explored and coupled with feedback and recommendations so that self-knowledge and self-awareness are enhanced in a more comprehensive and personalised manner, providing a novel tool to support office workers.

## 2.2 Feedback and Recommendations

Enhancing self-knowledge in the work context can be achieved through the use of digital solutions that track personal data, analyse it and provide feedback [16]. In fact, receiving meaningful feedback enhances self-awareness regarding work performance, which also impacts job motivation, as suggested by Hackman and Oldham (1975, cit. by [16]). A recent study with factory workers has combined well-being metrics and work performance-related metrics and presented them to workers via a Worker Feedback Dashboard web application [16]. Moreover, this study highlighted three key-design features feedback applications should provide: meaningful personal overviews, guidelines to act on based on those overviews and consideration for unquantified aspects relevant to work (e.g., social support).

Providing recommendations on how to act on (based on the feedback gathered) is also key to support workers' well-being and productivity [16]. Adopting strategies towards attentional processes such as mindfulness [17] can be included in the recommendations. Mindfulness is beneficial to help workers improving their attention towards working tasks, to disconnect from work when they need to [6] and to assist in the management of work-related concerns, while also enhancing self-awareness and
positive emotional states [18]. Other tips that elicit behaviour change can also be
useful to promote healthy actions related to work aspects and well-being.

3 Relation to Applied Artificial Intelligence Systems

Work-related disorders are a major concern in working environments and solutions that try to prevent them are limited by the variables that researchers are able to monitor/extract and by the knowledge available (i.e., variables highlighted in the literature review). By embedding AI into our tool, we will understand the variables that are better appropriate to assess attention and the AI models will expand on the previously programmed event detection by learning new sources of inattention/strain that might not have been defined previously. Thus, AI will play a major role in extending the prevention of WRD and in monitoring attention and cognitive constraints in the workplace.

4 Tool Development

4.1 Concept

Our tool aims to combine variables from HCI with work-related variables that can be
associated with attention and known to have an effect on work performance and well-
being. The variables will be chosen taken into consideration both the literature review
and the *focus group* with a group of workers from AT (the number of participants and
departments as the protocol for data acquisition is not yet decided). The data collected
will be displayed into a Dashboard System and analysed in order to deliver
meaningful easy-to-interpret personal outcomes that facilitate self-awareness and self-
reflection regarding working aspects. The tool concept, illustrated in Figure 1,
consists of 4 main layers: 1 - interaction; 2 – acquisition; 3 - analysis and 4 – feedback
and recommendations. The interactive layer will gather data from the use of tools
(HCI) and from questionnaires to deliver data to layer 2. On layer 3, the data gathered
will be integrated and analysed (AI models will also be used) and subsequently
displayed into a Dashboard System in order to deliver meaningful easy-to-interpret
feedback on personal outcomes that facilitate self-awareness and self-reflection
regarding working aspects. Additionally, this integration will allow us to introduce
recommendations based on sound knowledge regarding the identified factors,
promoting targeted actions in working environments.
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Overview of the Tool Concept and its 4 layers - 1. Interaction, 2. Acquisition, 3. Analysis and 4. Feedback and Recommendation.</figcaption></figure>

## 4.2 Development Roadmap

**Variables definition.** As office workers have to rely on the use of a computer to perform most of their work, their interaction with it is an important source of information, enabling us to retrieve personal data related to work processes and tasks. In that sense, HCI tools can be used to assess focused attention and human performance by providing variables such as the time and success in task completion, among others [19]. HCI techniques have also been successfully used to measure human behaviour based on their browser activity, namely by means of mouse pointer, keyboard, among other measures. Understanding these variables can provide insights on workers’ mental state and task engagement, being therefore relevant to assess focused attention [20]. Variables contributing to cognitive strain and closely related to attention include those that directly disrupt the primary task the worker is attending to, such as: interruptions from colleagues [3], notifications [5] or environmental constraints [6]. Through HCI these interruptions can be collected (e.g., the number of interruptions resulting from chats or notifications). Regarding environmental constraints, this input may be directly introduced by the worker into the dashboard tool, for example.

On the other hand, variables that can promote a balanced working life and attention management, such as quiet working hours [12] and break times [11], can also be tracked through HCI. Though the frequency and duration of breaks is not consensual in the literature, we propose to collect this data in order to have a broad set of variables and to meet this gap in the literature.

For other variables, we will develop specific measures based on questions eliciting qualitative assessments by the workers. Important questions might be related to leisure times during work hours (e.g., activities during breaks and social interactions), cognitive information, overload perception, etc. Portuguese versions of scales for measuring particular aspects of work productivity can also be used.

We will define a list of pertinent variables to track based on the literature available and also use the *Focus group* with the workers to gather information on more pertinent ones to add or remove.
**Focus Group.** *Focus group* will be used as part of our research tools to capture information that will assist the definition of the work-related variables and HCI to be collected (e.g., detailed qualitative information on disruptions at work). This method favours the acquisition of broader information, in a more economical way, while also benefiting from insights on group interaction and non-verbal communication [21]. Understanding the cognitively demanding tasks and working conditions of these office workers will be important to comprehend which personal data can be integrated into the system and is perceived as useful, while also enabling insights regarding demands at an organisational level.

The meaningful participation of workers has been linked to the effectiveness of safety and health programs and workers are often the ones who know most about potential hazards associated with their jobs [22] so, eliciting their insights is an essential part of creating this tool. Moreover, we will add questions related to user acceptance and experience in order to gather information relevant to consider for the design of the dashboard system.

**Validation.** We will focus our research efforts on testing: (1) if the tool is able to infer attention focus from the HCI and work-related variables chosen? (2) if the tool is perceived as effective and generating positive impacts? (3) if using the tool produces work performance enhancement? (4) if using the tool has an impact on well-being.

To answer these questions, we will need to operationalise variables, some of which have to take into consideration work-related variables that are not measurable. In such cases, questionnaires with rating scales can be useful tools. To assess the proposed tool, we will develop a user experience questionnaire and gather information on users' adherence (e.g., tool metrics use such as days of use). To operationalize work performance, we will collect input from workers about how their tasks are usually evaluated. This information might also be gathered through *focus group* and interviews with managers. Regarding the well-being assessment we will select, from previously validated measures, the instruments that better capture key variables defined in our project.

# 5 Conclusion and Future Work

With the rise of prevalence of WRD (e.g.,[1]) and highly cognitive demanding environments, particularly office workplaces (e.g., [3]), attention tools that promote workers' well-being and work performance are needed.

The present work focuses on the development of an innovative attention tool for office workers. The proposed concept combines HCI and work-related variables embedded into a dashboard system that, by using AI models, provides feedback on meaningful personal data, enhancing self-awareness and self-reflection towards work performance. Eliciting feedback has been found to impact well-being and work performance, which in turn can lead to enhanced productivity [16]. On top of this, our tool will display personalised recommendations, that will be automatically determined in the AI layer, which has not been combined in other studies [16].

Promoting workers' well-being should be a top priority regarding occupational health promotion since it benefits the worker and the organisation: by increasing workers' health and reducing absenteeism, also enhancing organizational performance [23]. We
believe that by integrating different dimensions of the workers' experience and presenting recommendations, such as mindfulness-based tips, we will be able to increase office workers' well-being and work performance.

As highlighted in the previous section, future work on the development of the tool will focus on the selection of variables that will be tracked as well as operationalizing them and establishing how to measure them. The dashboard will be designed taking into account the most relevant variables from both sources, i.e., HCI and work-related parameters, together with workers' insights (since involving them as co-constructors will likely empower them). Finally, the tool will undergo a validation step with evaluation regarding acceptance towards the tool and user experience in order to enhance it, as well as measures pertaining to the validity and reliability of the tool.

Some of the expected limitations of the system relate to the collection of data regarding important constructs to infer when investigating work and attention processes that are not easy to measure using existing metrics or methods, such as motivation and work relationships. This limitation can be overcome by including information retrieved from other sources such as the COPSOQ (Copenhagen Psychosocial Questionnaire) [24], a questionnaire that assesses psychosocial variables and risk factors at workplaces.

Questions regarding privacy concerns and the use of the application will also have to be elicit during the tool development. Furthermore, we need to have a careful approach in defining certain metrics as though users value positive feedback from the application [16] if feedback confronts aspects of the self, such as self-image (e.g., lower work performance measures than expected), feelings of self-competence and self-esteem may decrease (Stiglbauer et al. 2019 cit. by [16]).

The presented concept is still an idea on its early stages of development, but it is our contention that it will allow the collection of relevant data and its integration into a system that has the worker at the core will allow a better understanding of variables playing a role in attentional processes at work and ways to foster occupational health promotion.

**Acknowledgments.** This work was partly supported by Fundação para a Ciência e Tecnologia, under project PREVOCUPAI (DSAIPA/AI/0105/2019) and PhD grants PD/BDE/142816/2018, PD/BDE/150304/2019, PD/BDE/150672/2020.

## References

1. World Health Organization. Occupational and work-related diseases, https://www.who.int/activities/occupational-and-work--related-diseases

2. Rushton, L.: The Global Burden of Occupational Disease. Curr Environ Health Rep. 4 (3), 340-348 (2017). https://doi.org/10.1007/s40572-017-0151-2

3. Kalakoski, V., Selinheimo, S., Valtonen, T. et al.: Effects of a cognitive ergonomics workplace intervention (CogErg) on cognitive strain and well-being: a cluster-randomized controlled trial. A study protocol. BMC Psychol 8, 1 (2020). https://doi.org/10.1186/s40359-019-0349-1

4. Diamond, A.: Executive Functions. Annual Review of Psychology, 64 (1), 135--168 (2013). https://doi.org/10.1146/annurev-psych-113011-143750

5. Roca, C.: Human attention and its implications for human computer interaction. (2011)
6. Toniolo-Barrios, M., Pitt, L.: Mindfulness and the challenges of working from home in times of crisis. Bus Horiz (2020) https://doi.org/10.1016/j.bushor.2020.09.004

7. Page, K.M., Milner, A.J., Martin, A., Turrell, G., Giles-Corti, B., LaMontagne, A.D.: Workplace stress: what is the role of positive mental health? J Occup Environ Med. 56 (8), 814--9 (2014) https://doi.org/10.1097/JOM.0000000000000230

8. Luger, T., Maher, C.G., Rieger, M.A., Steinhilber, B.: Work-break schedules for preventing musculoskeletal symptoms and disorders in healthy workers. Cochrane Database Syst Rev. 23, 7 (2019). https://doi.org/10.1002/14651858.CD012886.pub2

9. Netherlands Center for Occupational Diseases. Statistics on Occupational Diseases 2018, https://www.occupationaldiseases.nl/content/statistics-occupational-diseases-2018

10. Health and Safety Executive. Health and safety at work Summary statistics for Great Britain 2020, https://www.hse.gov.uk/statistics/over

11. Mojtahedzadeh, N., Rohwer, E., Lengen, J., Harth, V., Mache, S.: Gesundheitsfördernde Arbeitsgestaltung im Homeoffice im Kontext der COVID-19-Pandemie [Health-promoting work design for telework in the context of the COVID-19 pandemic]. Zentralbl Arbeitsmed Arbeitsschutz Ergon. 7, 1--6 (2021). https://doi.org/10.1007/s40664-020-00419-1.

12. König,C.J., Kleinmann, M., Höhmann, W.: A field test of the quiet hour as a time management technique. European Review of Applied Psychology, 63 (3), 137--145 (2013). https://doi.org/10.1016/j.erap.2012.12.003

13. TogglTrack, https://toggl.com/

14. RescueTime, https://www.rescuetime.com/

15. Time Out, https://www.dejal.com/timeout/

16. Heikkilä, P., Honka, A., Kaasinen, E. et al.: Quantified factory worker: field study of a web application supporting work well-being and productivity. Cogn Tech Work (2021). https://doi.org/10.1007/s10111-021-00671-2

17. Bakker, A.B., Demerouti, E., De Boer, E., Schaufeli, W.B.: Job demands and job resources as predictors of absence duration and frequency. Journal of Vocational Behavior, 62, 341--56 (2003).

18. Brown, K.W., Ryan, R.M.: The benefits of being present: mindfulness and its role in psychological well-being. J Pers Soc Psychol. 84(4), 822--48 (2003). https://doi.org/10.1037/0022-3514.84.4.822

19. Lazar, J., Feng, J.H., Hochheiser, H.: Research Methods in Human Computer Interaction (Second Edition). Morgan Kaufmann, San Francisco (2017).

20. Cepeda, C., Tonet, R., Osorio, D. N., Silva, H. P., Battegay, E., Cheetham, M., Gamboa, H. (2019). Latent: A Flexible Data Collection Tool to Research Human Behavior in the Context of Web Navigation. In: IEEE Access, 7, 77659-77673. (2019) https://doi.org/10.1109/ACCESS.2019.2916996

21. Nagle, B., Williams, N. (n.d.). Methodology brief: Introduction to focus groups, http://www.mmgconnect.com/projects/userfiles/file/focusgroupbrief.pdf

22. United States Department of Labor. Occupational Safety and Health Administration: Recommended Practices for Safety and Health Programs, https://www.osha.gov/shpguidelines/worker-participation.html

23. Danna, K.D., Griffin, R.W.: Health and Well-Being in the Workplace: A Review and Synthesis of the Literature. 25, 357--384 (1999)

24. Burr, H., Berthelsen, H., Moncada, S., Nübling, M., Dupret, E. Demiral, Y. et al: The Third Version of the Copenhagen Psychosocial Questionnaire. Safety and Health at Work, 10, 4, 482--503 (2019). https://doi.org/10.1016/j.shaw.2019.10.002