# Gazealytics: A Unified and Flexible Visual Toolkit for Exploratory and Comparative Gaze Analysis

Kun-Ting Chen, Arnaud Prouzeau, Joshua Langmead, Ryan Whitelock-Jones,

Lee Lawrence, Tim Dwyer, Christophe Hurter, Daniel Weiskopf, Sarah

Goodwin

► To cite this version:

Kun-Ting Chen, Arnaud Prouzeau, Joshua Langmead, Ryan Whitelock-Jones, Lee Lawrence, et al. Gazealytics: A Unified and Flexible Visual Toolkit for Exploratory and Comparative Gaze Analysis.

ETRA 2023 - ACM Symposium on Eye Tracking Research and Applications, May 2023, Tubingen,

Germany. 69, ACM, pp.1-7, 2023, 10.1145/3588015.3589844 . hal-04144751

HAL Id: hal-04144751

https://enac.hal.science/hal-04144751v1

Submitted on 9 Oct 2024

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Gazealytics: A Unified and Flexible Visual Toolkit for Exploratory and Comparative Gaze Analysis

Kun-Ting Chen

University of Adelaide
Australia

University of Stuttgart
Germany

Kun-Ting.Chen@adelaide.edu.au

Arnaud Prouzeau

Inria & LaBRI (University of
Bordeaux, CNRS, Bordeaux-INP)
France

Arnaud.Prouzeau@inria.fr

Joshua Langmead

Monash University
Australia

jlan0025@student.monash.edu

Ryan T Whitelock-Jones

Monash University
Australia

ryanwhitelockjones@gmail.com

Lee Lawrence

Monash University
Australia

Lee.Lawrence@monash.edu

Tim Dwyer

Monash University
Australia

Tim.Dwyer@monash.edu

Christophe Hurter

ENAC, Université de Toulouse
France

Christophe.Hurter@enac.fr

Daniel Weiskopf

University of Stuttgart
Germany

Daniel.Weiskopf@visus.uni-
stuttgart.de

Sarah Goodwin

Monash University
Australia

Sarah.Goodwin@monash.edu

## ABSTRACT

We present a novel, web-based visual eye-tracking analytics tool called GAZEALYTICS. Our open-source toolkit features a unified combination of gaze analytics features that support flexible exploratory analysis, along with annotation of areas of interest (AOI) and filter options based on multiple criteria to visually analyse eye tracking data across time and space. GAZEALYTICS features coordinated views unifying spatiotemporal exploration of fixations and扫描 paths for various analytical tasks. A novel matrix representation allows analysis of relationships between such spatial or temporal features. Data can be grouped across samples, user-defined AOIs or time windows of interest (TWIs) to support aggregate or filtered analysis of gaze activity. This approach exceeds the capabilities of existing systems by supporting flexible comparison between and within subjects, hypothesis generation, data analysis and communication of insights. We demonstrate in a walkthrough that Gazealytics supports multiple types of eye tracking datasets and analytical tasks.

## CCS CONCEPTS

* Human-centered computing → Visualization toolkits; Empirical studies in visualization.

## KEYWORDS

Eye tracking, visual analytics, area of interest, time window of interest, matrix-based overview, group-level visualisation

## ACM Reference Format:

Kun-Ting Chen, Arnaud Prouzeau, Joshua Langmead, Ryan T Whitelock-Jones, Lee Lawrence, Tim Dwyer, Christophe Hurter, Daniel Weiskopf, and Sarah Goodwin. 2023. Gazealytics: A Unified and Flexible Visual Toolkit for Exploratory and Comparative Gaze Analysis. In *2023 Symposium on Eye Tracking Research and Applications (ETRA '23), May 30-June 2, 2023, Tubingen, Germany*. ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/3588015.3589844

## 1 INTRODUCTION

Analysis of eye tracking data collected in observational studies is a complex process that aims to make sense of large amounts of gaze data involving high-frequency gaze samples, collected from a number of individuals [Afzal et al. 2022; Pozdniakov et al. 2023]. Even in controlled experiments, there may be many samples collected, each involving multiple activities, which can result in long recordings requiring complicated multivariate data analysis [Burch 2022; Chang et al. 2018; Servais et al. 2022]. Therefore, to make analysis tractable, researchers often identify key areas of interest (AOIs) within the field of view, as well as time windows of interest (TWIs) throughout the period of participant activity [Kwok et al. 2019; Menges et al. 2020].

AOIs are typically pre-identified in the experimental setup and statistical methods can be used to test hypotheses concerning expected activity within and across AOIs. However, the comprehensive space-time data recorded from gaze tracking is not only useful for hypothesis testing but may also be used for building hypothesis, i.e., supported by data-driven exploratory eye tracking analysis [Kurzhals et al. 2016b].

<footnote>Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.</footnote>

<footnote>ETRA '23, May 30-June 2, 2023, Tubingen, Germany</footnote>

<footnote>© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.</footnote>

<footnote>ACM ISBN 979-8-4007-0150-4/23/05...$15.00</footnote>

<footnote>https://doi.org/10.1145/3588015.3589844</footnote>
<figure><img src="image_2.png" /><figcaption>**Figure 1:** An example of multi-way visual exploration of a controlled experiment. An exploration can begin at any stage of (a–f) and move between them as shown by arrows (arrows between the leftmost and the rightmost columns are ignored for clarity). Colours indicate 6 AOIs and the relationships between AOIs and sample groups (b). Sample groups were determined by their pairwise similarity comparison based on the number of AOI transitions (a).</figcaption></figure>

Many of the current tools are inflexible in the degree to which they support exploratory data analysis [Burch et al. 2019; Menges et al. 2020]. Existing non-visual tools allow analysis with a pre-defined hypothesis, but there is little in the way of tools that allow for the exploration or generation of hypothesis [Blascheck et al. 2016; Panetta et al. 2020]. To our knowledge, there is no single tool that provides a highly dynamic visual exploration with filtering options based on multiple criteria over fixations, saccades and scanpaths across many ways of grouping the data, e.g., by samples, AOIs (spatial filtering), or TWIs (temporal filtering).

Our contribution is twofold. First, we provide a multi-way visual exploratory approach for data from eye tracking studies, extensively using matrix-based overview (see Figure 1). As we summarise in section 2, a unified general matrix relationship approach has not been previously presented for eye tracking analysis. Our second contribution is an interactive group-level exploration tool for gaze data analysis. GAZEALYTICS provides flexible visual data aggregation for identifying gaze patterns at various data dimensions: samples, AOIs, and TWIs. We show with eye tracking examples that GAZEALYTICS supports flexible exploratory and comparative analysis.

We make GAZEALYTICS publicly available, as an open-source gaze analytics toolkit<fnref n="1" />. The toolkit provides an easy-to-use interface to integrate into analysts’ existing workflow. Several users have already used the web application for their real-world eye tracking data analysis problems (see, e.g., [Cai et al. 2022; Chen et al. 2023; Pozdniakov et al. 2023]).

## 2 RELATED WORK

Blascheck et al. [2017] survey techniques for interactive visualisation and visual analytics for eye tracking. Their taxonomy covers various facets of gaze analysis. In particular, they distinguish between static and dynamic stimuli, between passive and active stimulus content, between temporal, spatial, and spatiotemporal visualisations, and between point-based and AOI-based representations of gaze.

Our approach primarily targets static stimuli, and mostly passive stimulus content. GAZEALYTICS provides a rich set of visualisation views, thus supporting a range of techniques to cover temporal, spatial, and spatiotemporal analysis. Finally, we make heavy use of AOIs to drive the analysis, but still support point-based visualisation as well.

Brehmer and Munzner [2013] list 11 categories to characterise task typology. Kurzhals et al. [2017] present six eye tracking visualisation tasks to describe multivariate gaze data analysis and their derived analytical tasks. To support the elasticity of GAZEALYTICS compared with state-of-the-art, we aim to support the full range of eye tracking visualisation tasks. Using Brehmer and Munzner [2013]'s typology and Blascheck et al. [2017]'s taxonomy, Table 1 compares GAZEALYTICS with other visual eye tracking analytics tools—both research and commercial ones. It should be noted that the selection of tools for this comparison is based on literature review, and author's collaborations with eye-tracking practitioners actively using the tools. The criteria focus on matrix-based visualisation and interactive group-level exploration. It is not meant to be

<footnote n="1">¹https://github.com/gazealytics/gazealytics-master</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 2: Coordinated views of GAZEALYTICS: (a) data management panel; (b) spatial panel; (c) parameter control panel; (d) metric panel; (e) timeline panel.</figcaption></figure>

exhaustive by any means; we have rather chosen a set of tools that
cover a range of different functionalities and tasks that is closest to
our work.

While existing visualisation techniques come with a variety of
visualisation views [Blascheck et al. 2014], we want to pick out
matrix-based visualisation as one particularly important one. Ma-
trix visualisations can show AOI transitions of fixations and of
aggregated or concatenated fixations over multiple samples be-
tween AOIs [Wang et al. 2021], scanpath similarity between sam-
ples [Burch et al. 2018; Kumar et al. 2019], relationships of mean
fixation duration on AOIs by tasks [Siirtola et al. 2009], and so on.

Kurzhals et al. [2016a] point out that only little work had been
done in combining traditional numerical-based analysis and human-
in-the-loop gaze analytical reasoning. Therefore, there is still need
for an improved combination of visual-interactive exploration and
advanced visualisations.

In summary, GAZEALYTICS adopts many of the individual visualisation views already known from the literature. In particular, it makes heavy use of matrix visualisation, aggregation, and filtering. However, as shown in Table 1, GAZEALYTICS is unique in its combination of such individual components, the number and extent of supported visualisation views, their integration in the interactive and user-steered process, and the level of exposure of parameters to the user.

# 3 SYSTEM AND DESIGN

In the following, we describe key characteristics of our design. The
main GUI components are illustrated in Figure 2. Data is loaded
from a text file with three tab separated fields: time, gaze x- and y-
coordinates, easily obtained from most eye-tracking equipment and
therefore generalisable to many eye-tracking applications [Kurzhals
et al. 2017]. Additionally, GAZEALYTICS adopts an optional fourth

input field where a user can label the timestamp and gaze coordinate
with an associated experimental condition which can be mapped
to the visual gaze analysis.

## 3.1 Design overview

GAZEALYTICS consists of five GUI panels. The data management panel in Figure 2(a) allows for managing input data across multiple dimensions: *Who* (samples), *Where* (AOIs), and *When* (TWIs), described in Kurzhals et al. [2017]. Furthermore, this panel allows visual aggregation, configuration, and manipulation of grouping across these dimensions. Export of various results of the current sessions such as gaze features, AOI mapping of fixations, and visualisations are available in multiple buttons. Other tabs of the data management panel support text annotation, customising colour schemes using analysts’ existing definitions, and export configuration for visual elements such as axis labels or charts. A parameter control panel in Figure 2(c), including the control bar at the top of Figure 2(b) allows for toggling and showing current state of data filtering by individual, group level, or overall in each data dimension. This panel also includes an interactive legend for filtering out unselected sample groups. For example, Figure 2(c)-middle shows a legend of 3 sample groups (green, ogange, purple), where a mouse click on green legend resulted in visually filtering out the scanpath and density maps of organe and purple groups in the spatial view, as seen in Figure 2(b).

Three main visualisation panels provide coordinated views for
flexible exploration: the metric panel in Figure 2(d), the spatial panel
in Figure 2(b), and the timeline panel in Figure 2(e). The metric
panel in Figure 2(d) provides flexible multivariate data exploration
and comparison (Relate [Kurzhals et al. 2017]). The metric panel
(d) can be switched between a matrix and a histogram view (next
to the matrix tab) of fixation and saccade distributions. A video
view, embedded to the metric panel (left of the matrix), as seen in
<table><caption>Table 1: An overview of GAZEALYTICS and existing analytical tools' support for eye tracking visualisation taxonomy (visualisation techniques) and 11 interactive visualisation task categories (grouped into Encode, Manipulate, Introduce). The comparison criteria (3rd column) centres on metric visualisations and interactive visual grouping and exploration as a focus of this work.</caption><thead><tr><th colspan="2">Name</th><th>Gazealitics</th><th>Burch et al. 2018</th><th>Kurzhals et al. 2018</th><th>Kumar et al. 2019</th><th>Nezel et al. 2019</th><th>Merges et al. 2019</th><th>Kurzhals et al. 2019</th><th>Vogelsperger et al. 2020</th><th>Kruger et al. 2014</th><th>Goodwin et al. 2019</th><th>Blickner et al. 2022</th><th>Pulitano et al. 2019</th><th>Taino Pro Lab</th></tr></thead><tbody><tr><td>Tool type</td><td>R: research tools; C: commercial tools</td><td>R</td><td>R</td><td>R</td><td>R</td><td>R</td><td>R</td><td>R</td><td>R</td><td>R</td><td>R</td><td>R</td><td>R</td><td>C</td><td>C</td><td>C</td></tr><tr><td rowspan="2">Stimuli</td><td>Stimuli content</td><td>A: active; P: passive</td><td>P</td><td>P</td><td>A</td><td>P</td><td>P</td><td>A</td><td>A</td><td>P</td><td>P</td><td>P</td><td>P</td><td>A</td><td>A</td><td>A</td></tr><tr><td>Stimuli dynamics</td><td>S: stationary; D: dynamic</td><td>S</td><td>S</td><td>D</td><td>S</td><td>S</td><td>D</td><td>S</td><td>D</td><td>S</td><td>S</td><td>S</td><td>S</td><td>D</td><td>D</td><td>D</td></tr><tr><td rowspan="10">Visualisation techniques</td><td rowspan="4">Visual metrics (matrix)</td><td>fixation distribution / saccade transitions</td><td>X</td><td>X</td><td></td><td>X</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>similarity between samples</td><td>X</td><td>X</td><td></td><td>X</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>group-level metrics relationships</td><td>X</td><td>X</td><td></td><td>X</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>small multiples (minmaps)</td><td>X</td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="2">Visual metrics (histogram)</td><td>saccade length / fixation duration by samples / AOIs</td><td>X</td><td></td><td>X</td><td></td><td></td><td></td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td></td><td></td></tr><tr><td>saccade length / fixation duration by groups</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">Space</td><td>group-level density map</td><td>X</td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td></tr><tr><td>fixation context before / after visiting / glancing a</td><td>X</td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td><td>X</td><td>X</td><td></td><td></td><td></td><td></td></tr><tr><td>directional saccade bundling</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="2">Timeline</td><td>AOI sequence / scarf plot by samples</td><td>X</td><td></td><td>X</td><td></td><td></td><td></td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td></td></tr><tr><td>AOI group sequence / scarf plot by AOI groups</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td>X</td><td></td></tr><tr><td>Video</td><td>gain additional context / linked with TWI</td><td>X</td><td></td><td>X</td><td></td><td>X</td><td>X</td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td rowspan="2">Encode</td><td>Encode</td><td>A: AOI-based; P: Point-based; B: Both</td><td>B</td><td>P</td><td>B</td><td>P</td><td>B</td><td>P</td><td>B</td><td>B</td><td>B</td><td>B</td><td>A</td><td>B</td><td>B</td><td>B</td></tr><tr><td>Select</td><td>focus-and-context / linking / brushing by sample</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td></td><td></td></tr><tr><td rowspan="8">Manipulate</td><td>Select</td><td>focus-and-context / linking / brushing by group</td><td>X</td><td></td><td>X</td><td></td><td>X</td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Navigate</td><td>between overview / group / individual view of</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>Arrange</td><td>layout of multiple coordinated views</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td>X</td><td></td><td></td><td></td><td></td></tr><tr><td>Arrange</td><td>order of data representation by layout optimisation</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">Change</td><td>A: AOIs; T: TWIs; B: both</td><td>B</td><td>T</td><td>B</td><td></td><td>A</td><td>B</td><td>B</td><td>A</td><td>B</td><td>T</td><td>B</td><td>B</td><td>B</td><td>B</td></tr><tr><td>visual data granularity</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>visual elements: colour scheme, size, transparency</td><td>X</td><td>X</td><td></td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>Filter</td><td>hide and show visuals of sample/AOI/TWI</td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td rowspan="5">Introduce</td><td>Aggregate</td><td>visuals / metrics for A: AOIs, S: samples, T: TWIs; a: all</td><td>a</td><td>ST</td><td>ST</td><td>S</td><td></td><td>S</td><td>S</td><td>ST</td><td>S</td><td>AS</td><td>AS</td><td>AS</td><td>AS</td><td></td></tr><tr><td>Annotate</td><td>fixation labelling with AOIs</td><td>X</td><td></td><td>X</td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>Annotate</td><td>visualisations with notes</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td>X</td><td></td><td></td><td>X</td><td></td></tr><tr><td rowspan="2">Import</td><td>A: AOIs; S: samples; T: TWIs; a: all of them</td><td>a</td><td>S</td><td>SA</td><td>S</td><td>S</td><td>S</td><td>S</td><td>SA</td><td>S</td><td>a</td><td>SA</td><td>S</td><td></td><td></td></tr><tr><td>saved project files to restore previous state of analysis</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td>X</td><td></td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>Derive</td><td>pre-identified AOI/TWI/sample grouping</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="2">Record/Produce</td><td>new attributes (metrics) from data grouping</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td></td><td>X</td><td></td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>export AOIs/TWIs/samples; metrics extraction</td><td>X</td><td></td><td></td><td></td><td></td><td>X</td><td></td><td>X</td><td></td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td></td><td>save visualisations</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr></tbody></table>

Figure 2(d), allows a user to attach a video associated with each sample. When a TWI is selected, it navigates the video of a currently selected sample to the corresponding timespan of the TWI for gaining context about eye movement data. Furthermore, coordinated views showing the spatial panel (b) can be linked with a matrix or histogram view in Panel (d), which present group or individual attention distribution numerically and across the spatial view field via density maps, with fixations and saccades showing spatial context. A scanpath showing temporal eye movement sequence is shown on-demand in Figure 2(b) on mouse hovering over the scarfplot of AOI sequences in Figure 2(e).

## 3.2 Multi-way visual exploration with matrix-based overview

In the metric panel in Figure 2(d), a matrix shows view metrics between pairs of features grouped by either sample, TWI, or AOI. The matrix supports a common range of fixation, saccade and scanpath metrics by Poole and Ball [2006]. To support both exploratory

and comparative analytical tasks, the matrix shows similarity comparison (Compare [Kurzhals et al. 2017]) in terms of visual metrics within each of the data dimension: samples, AOIs, and TWIs. Examples include AOI (group) sequence scores, based on encoding each sample's AOI visiting sequence, and string alignment according to the Needleman-Wunsch algorithm [Needleman and Wunsch 1970]. Similarity of AOI transitions between sample (group) and attention distribution (density overlap of AOIs or density maps).

An example of a controlled experiment dataset using metric panel and spatial panel of 16 samples is shown in Figure 1. To take an overview of the entire dataset, an analyst can directly jump into any multivariate data exploration at individual (a,b,c) or group level (b,d,e,f). Temporal filter is applied where metrics are aggregated for all repeated measures within TWI Group 2. To gain spatial context, Figure 1(f) shows results of brushing and linking over matrix cell (SankeyDiagram1, SankeyDiagram2), denoted as (SD1, SD2) of AOI transition matrix in Figure 1(c), highlighting AOI transitions in Group 4 has higher frequency of “looking out” from SD1 to SD2
than Group 11, supported by fixation context (red, green, yellow dots) where the brushed saccades connect to.

To visually identify gaze patterns (Detect [Kurzhals et al. 2017]) from visual exploration, visualisation algorithms such as matrix reordering [Behrisch et al. 2016], directional and spatial saccade bundling (in the spatial view) [Hurter et al. 2011] are applied to optimise the layout and reveal high-level structures or patterns for better capturing the relationships within the data [Chen 2022]. Figure 1(a) shows an example of applying the “optimal leaf” ordering to reveal three potential visual blocks (using AOI transition similarity criteria) along the diagonal [Behrisch et al. 2016]. Samples can then be grouped, while moving into various matrix relationship visualisations to cross-verify the grouping and metrics results.

## 3.3 Interactive visual grouping and exploration in multiple data dimensions

GAZEALYTICS supports a comprehensive set of tools for group-level analysis. A user can aggregate metrics of fixations, visitations, density distribution, saccade transition counts, etc, from two or more samples, with spatial aggregation of these metrics in AOI groups, and temporal aggregation of them in TWI groups, identified by unique logical group IDs (GIDs). Figure 1 shows a temporal aggregation of saccade transition similarity by a selected TWI Group: 2 (defined in section 4), across samples (Figure 1(a)), or samples and sample groups (Figure 1(b)). In the spatial panel, visual grouping and data granularity also applies to a focus-and-context interaction which allows selecting an AOI and gaining an overview of surrounding spatial context of fixations by samples, spatial, and temporal filtering. Figure 1(f) shows fixations context: before/after visiting/glancing a selected (focus) AOI (SD1, denoted by bolder boundary). If a different TWI group or a different sample filtering is applied, the visuals will show different results; This also applies to brushing & linking where coordinated views highlight selected group's fixation, saccades as one brushes over matrix cells or complementary views. Timeline panel displays AOI group sequence, or scarf plots coloured by AOI groups. A change of grouping triggers metric re-calculation using data aggregation and the resulting visualisations are re-generated.

## 3.4 Implementation

The software architecture is an MVC framework using HTML canvas for visuals. Metrics aggregation is achieved via identifying data elements with the same logical group ID (GID). GAZEALYTICS has native support for having more than one instance, via spawning a new tab within a web browser for a flexible comparison across experimental conditions. It allows insights to be disseminated with peers through exporting and restoring the visualisations.

# 4 EXPLORATORY AND COMPARATIVE ANALYSIS EXAMPLES

The features of GAZEALYTICS were developed through a series of close collaborations with eye-tracking practitioners actively using the software to analyse their collected study data. We now present two examples of actual use. We refer to the task typology in Table 1 when describing the activities performed.

The first example use case is an evaluation of whether information visualisations placed side-by-side could be complementary [Chang et al. 2017]. The experimental design involved 24 participants, and 12 repeated measures across 4 tasks (Figure 1). Chang et al. [2017] used the Tobii Pro software to label AOIs and extracted fixation metrics, and they built custom visualisations to inspect the results before running statistical comparisons. This is a typical approach for the analysis of eye movement data, and here it succeeded in detecting three eye movement strategies between two large predefined AOIs within the stimuli. However, it was difficult to analyse finer-grained gaze behaviours.

In contrast, with GAZEALYTICS, one can flexibly begin analysis with any stage of the multi-way exploration (subsection 3.2) from different perspectives.

**1. Confirm pre-identified hypothesis:** We first created two AOIs using definitions of the left and right side-by-side diagrams, as described in the prior study (Import) [Chang et al. 2017]. Trials were imported as TWIs representing start time and end time of each trial (Import). By selecting a local filtering with a TWI (selecting a specific trial), we aggregated samples to observe within-subject differences across participants in terms of the fixation percentage time on AOIs metrics (Filter, Aggregate). This allowed us to quickly confirm the exclusive and parallel use of the left and right side-by-side views for a representative graph exploration task (i.e., Out-Link task [Chang et al. 2017]).

**2. Data-driven exploration:** To gain further insights into the use of each left and right view, we created new finer-grained AOIs not reported previously by Chang et al. [2017], by labeling three matrix waves and three Sankey Diagrams within the AOI (Annotate), as shown in Figure 1(f). Six AOIs were created and two logical AOI group IDs were assigned to each AOI from the group editor in the data management panel (Aggregate, Change).

Next, we selected a larger granularity to gain an overview (TWI Group 2), i.e., temporal aggregate of data dimension, which aggregated metrics from all the trials belonging to the Out-Link task (Select, Navigate). Using coordinated views of metric, spatial, and timelines, it gave us additional context. In the spatial view in Figure 2(b), we first inspected the gaze points entering (green dots), leaving (red dots), or glancing out (yellow dots) of the leftmost Sankey Diagram AOI (boundary highlighted in bold when selected from the spatial view or AOI tab). Subsequently, we repeated the same exploration by switching between sample groups to allow us to gain insights into each sample group's “gazing out rate” (to the left or right) of the selected leftmost Sankey-diagram (SD1)” (Select, Navigate). As a result, we found that Group 11 has a higher number of red dots than Group 4 (as seen in Figure 1(f)).

**3. Comparative analysis:** To compare group behaviours, we jumped into data-driven exploration of the stage denoted in Figure 1(a). By comparing pairwise AOI transitions similarity and arranging the matrix layout with optimal matrix reordering (Ar-range), three different visual blocks were visually identified along the diagonal. We edited the group ID of each sample according to the block patterns found (Aggregate, Change). We visually identified that Group 11 and Group 4 had similar AOI group fixation percentage times based on new metrics derived at a group-level (Figure 1(f, g)) (Derive). For Group 11, Figure 3 showed it has less frequent AOI transitions than Group 4. We extracted AOI transitions metrics of
<figure><img src="image_4.png" /><figcaption>Figure 3: Visual comparison of AOI transitions between two samples groups: Group 4 and Group 11 identified through multi-way exploration in Figure 1. Data is filtered by temporal aggregation: selected TWI Group 2, but it also can be changed to other TWI groups, all TWIs (for gaining an entire overview), or an individual TWI.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 4: Example of exploring AOI definitions (using the spatial panel) for labelling fixations, linked with the matrix: sample-AOI relationships with the hit-any-AOI-rate (HAAR) metric, and the timeline panel.</figcaption></figure>

each individual participant from GAZEALYTICS (Produce), and executed our statistical test script of the extracted data in R. We found a potential weak trend (*p* = 0.09) that there was an overall difference in AOI transitions across groups (Friedman's non-parametric test). This provided the new insight that Group 4 participants using Sankey Diagram had a weak trend to have smaller number of AOI transitions than Group 11. During exploration, we also spotted the outlier P21, which has less than 10 fixations. It was then excluded (Filter) from the rest of the exploration.

**4. Coordinated views and interactive exploration:** Our second use case is an example of exploratory analysis with data collected from a user study where AOIs are not pre-identified [Chen et al. 2020]. GAZEALYTICS supports dynamically annotating unlabelled gaze-data with AOIs while visually inspecting the impact of AOI definitions on visual metrics (such as AOI uncertainty metrics), to arrive at more robust AOI definitions. An example of visually improving hit-any-AOI-rate (HAAR) is shown in Figure 4, which could lead to higher quality of eye tracking data analysis [Wang
et al. 2022]. Visual exploration reduces the impact of AOI uncertainty by improving HAAR from 77% (left) to 88% (by manipulating the shape of green and pink polygons to be more aligned at the boundaries) (*Annotate, Change*). Further visual checking and manipulation result in an improved HAAR of 93% (right) for a more suitable AOI definition. Inspecting the timeline panel (at the bottom right) shows that fixations at the gray area (bottom left) were mapped to AOI4 and AOI6.

## 5 CONCLUSION

We have presented GAZEALYTICS, a new visual eye tracking analytics toolkit that supports flexible exploratory gaze analysis. It combines matrix-based overview with the ability to aggregate visual metrics at individual or group-level of sample, area-, and time-window of interest, that are dynamically linked with their spatiotemporal visualisations. Our comparative table shows that GAZEALYTICS provides a comprehensive and interactive toolkit for exploratory gaze analysis beyond other current toolkits, in terms of a full range of visualisation task taxonomy supported. Eye tracking examples show that with the unified toolkit and multiple coordinated views, it supports multiple types of eye tracking datasets and a wide range of analytical tasks.

Furthermore, there has not been many extensive user studies on how a flexible and generic visual eye tracking analytics toolkit affects eye tracking analysts' workflow in the real-world analysis context. With this approach, we hope to further elicit new insights through evaluating with eye tracking experts to propose a more human-centric approach to visual eye tracking analytics.

## ACKNOWLEDGMENTS

The authors would like to thank Chunlei Chang for actively using the toolkit and provide us his feedback, Ishwari Bhade for co-developing GAZEALYTICS, Kuno Kurzhals for the discussion of visual eye tracking toolkit comparisons, as well as the anonymous reviewers for their helpful comments on this paper. This research is funded by Monash Faculty of Information Technology and the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Project-ID 251654672 – TRR 161.

## REFERENCES

* Umair Afzal, Arnaud Prouzeau, Lee Lawrence, Tim Dwyer, Saikiranao Bichinepally, Ariel Liebman, and Sarah Goodwin. 2022. Investigating Cognitive Load in Energy Network Control Rooms: Recommendations for Future Designs. *Frontiers in Psychology* 13 (2022).

* Michael Behrisch, Benjamin Bach, Nathalie Henry Riche, Tobias Schreck, and Jean-Daniel Fekete. 2016. Matrix Reordering Methods for Table and Network Visualization. *Computer Graphics Forum* 35, 3 (2016), 693–716.

* T. Blascheck, M. John, K. Kurzhals, S. Koch, and T. Ertl. 2016. VA²; A Visual Analytics Approach for Evaluating Visual Analytics Applications. *IEEE Transactions on Visualization and Computer Graphics* 22, 1 (2016), 61–70.

* T. Blascheck, K. Kurzhals, M. Raschke, M. Burch, D. Weiskopf, and T. Ertl. 2014. State-of-the-Art of Visualization for Eye Tracking Data. In *EuroVis – STARS, R. Borgo, R. Maciejewski, and I. Viola (Eds.). The Eurographics Association*, 63–82.

* T. Blascheck, K. Kurzhals, M. Raschke, M. Burch, D. Weiskopf, and T. Ertl. 2017. Visualization of Eye Tracking Data: A Taxonomy and Survey. *Computer Graphics Forum* 36, 8 (2017), 260–284.

* Matthew Brehmer and Tamara Munzner. 2013. A Multi-Level Typology of Abstract Visualization Tasks. *IEEE Transactions on Visualization and Computer Graphics* 19, 12 (2013), 2376–2385.

* Michael Burch. 2022. *Eye Tracking and Visual Analytics*. CRC Press.

* Michael Burch, Ayush Kumar, and Neil Timmermans. 2019. An Interactive Web-Based Visual Analytics Tool for Detecting Strategic Eye Movement Patterns. In *Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications*. Article 93, 5 pages.

* Michael Burch, Kuno Kurzhals, Niklas Kleinhans, and Daniel Weiskopf. 2018. EyeMSA: Exploring Eye Movement Data with Pairwise and Multiple Sequence Alignment. In *Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications*. Article 52, 5 pages.

* Minghao Cai, Bin Zheng, and Carrie Demmans Epp. 2022. Towards Supporting Adaptive Training of Injection Procedures: Detecting Differences in the Visual Attention of Nursing Students and Experts. In *Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization*. 286–294.

* Chunlei Chang, Benjamin Bach, Tim Dwyer, and Kim Marriott. 2017. Evaluating Perceptually Complementary Views for Network Exploration Tasks. In *Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems*. 1397–1407.

* Chunlei Chang, Tim Dwyer, and Kim Marriott. 2018. An Evaluation of Perceptually Complementary Views for Multivariate Data. In *Proceedings of the 2018 IEEE Pacific Visualization Symposium (PacificVis).* IEEE, 195–204.

* Kun-Ting Chen. 2022. It's a Wrap! Visualisations that Wrap Around Cylindrical, Toroidal, or Spherical Topologies. *arXiv preprint arXiv:2209.13251 (2022).*

* Kun-Ting Chen, Tim Dwyer, Kim Marriott, and Benjamin Bach. 2020. DoughNets: Visualising Networks Using Torus Wrapping. In *Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems*. Article 53, 11 pages.

* Kun-Ting Chen, Quynh Quang Ngo, Kuno Kurzhals, Kim Marriott, Tim Dwyer, Michael Sedlmair, and Daniel Weiskopf. 2023. Reading Strategies for Graph Visualizations that Wrap Around in Torus Topology. *arXiv:2303.17066 [cs.HC]*

* Christophe Hurter, Alexandru Telea, and Ozan Ersoy. 2011. MoleView: An Attribute and Structure-Based Semantic Lens for Large Element-Based Plots. *IEEE Transactions on Visualization and Computer Graphics* 17, 12 (2011), 2600–2609.

* Ayush Kumar, Neil Timmermans. Michael Burch, and Klaus Mueller. 2019. Clustered Eye Movement Similarity Matrices. In *Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications*. Article 82, 9 pages.

* Kuno Kurzhals, Michael Burch, Tanja Blascheck, Gennady Andrienko, Natalia Andrienko, and Daniel Weiskopf. 2017. A Task-Based View on the Visual Analysis of Eye-Tracking Data. In *Eye Tracking and Visualization*, Michael Burch, Lewis Chuang, Brian Fisher, Albrecht Schmidt, and Daniel Weiskopf (Eds.). Springer International Publishing, Cham, 3–22.

* Kuno Kurzhals, Brian Fisher, Michael Burch, and Daniel Weiskopf. 2016a. Eye tracking evaluation of visual analytics. *Information Visualization* 15, 4 (2016), 340–358.

* Kuno Kurzhals, Marcel Hlawatsch, Christof Seeger, and Daniel Weiskopf. 2016b. Visual Analytics for Mobile Eye Tracking. *IEEE Transactions on Visualization and Computer Graphics* 23, 1 (2016), 301–310.

* Tiffany C. K. Kwok, Peter Kiefer, Victor R. Schinazi, Benjamin Adams, and Martin Raubal. 2019. Gaze-Guided Narratives: Adapting Audio Guide Content to Gaze in Virtual and Real Environments. In *Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems*. Article 491, 12 pages.

* Raphael Menges, Sophia Kramer, Stefan Hill, Marius Nißlmüller, Chandan Kumar, and Steffen Staab. 2020. A Visualization Tool for Eye Tracking Data Analysis in the Web. In *Proceedings of the ACM Symposium on Eye Tracking Research and Applications (ETRA '20 Short Papers)*. Article 46, 5 pages.

* Saul B. Needleman and Christian D. Wunsch. 1970. A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins. *Journal of Molecular Biology* 48, 3 (1970), 443–453.

* Karen Panetta, Qianwen Wan, Srijith Rajeev, Aleksandra Kaszowska, Aaron L Gardony, Kevin Naranjo, Holly A Taylor, and Sos Agaian. 2020. ISeeColor: Method for Advanced Visual Analytics of Eye Tracking Data. *IEEE Access* 8 (2020), 52278–52287.

* Alex Poole and Linden J. Ball. 2006. Eye Tracking in HCI and Usability Research. In *Encyclopedia of Human Computer Interaction*. IGI global, 211–219.

* Stanislav Pozdnjakov, Roberto Martinez-Maldonado, Yi-Shan Tsai, Vanessa Echeverria, Namrata Srivastava, and Dragan Gasevic. 2023. How Do Teachers Use Dashboards Enhanced with Data Storytelling Elements According to their Data Visualisation Literacy Skills?. In *LAK23: 13th International Learning Analytics and Knowledge Conference*. 89–99.

* Anaïs Servais, Christophe Hurter, and Emmanuel J. Barbeau. 2022. Attentional switch to memory: an early and critical phase of the cognitive cascade allowing autobiographical memory retrieval. https://doi.org/10.31234/osf.io/z32qe PsyArXiv.

* Harri Siirtola, Tuuli Laivo, Tomi Heimonen, and Kari-Jouko Räihä. 2009. Visual Perception of Parallel Coordinate Visualizations. In *Proceedings of the 2009 13th International Conference Information Visualisation. IEEE*, 3–9.

* Yao Wang, Mihai Băce, and Andreas Bulling. 2021. Scanpath Prediction on Information Visualisations. *arXiv preprint arXiv:2112.02340 (2021).*

* Yao Wang, Maurice Koch, Mihai Băce, Daniel Weiskopf, and Andreas Bulling. 2022. Impact of Gaze Uncertainty on AOIs in Information Visualisations. In *Proceedings of the 2022 Symposium on Eye Tracking Research & Applications*. Article 60, 6 pages.