DevLan/vispubs huggingface dataset:
Design and Analysis of Intelligent Text Entry Systems with Function Structure Models and Envelope Analysis
Authors: Per Ola Kristensson, Thomas Müllners
Published at CHI, 2021

Designing intelligent interactive text entry systems often relies on factors that are difficult to estimate or assess using traditional HCI design and evaluation methods. We introduce a complementary approach by adapting function structure models from engineering design. We extend their use by extracting controllable and uncontrollable parameters from function structure models and visualizing their impact using envelope analysis. Function structure models allow designers to understand a system in terms of its functions and flows between functions and decouple functions from function carriers. Envelope analysis allows the designer to further study how parameters affect variables of interest, for example, accuracy, keystroke savings and other dependent variables. We provide examples of function structure models and illustrate a complete envelope analysis by investigating a parameterized function structure model of predictive text entry. We discuss the implications of this design approach for both text entry system design and for critique of system contributions.
Designing CAST: A Computer-Assisted Shadowing Trainer for Self-Regulated Foreign Language Listening Practice
Authors: Mohi Reza, Dongwook Yoon
Published at CHI, 2021

Shadowing, i.e., listening to recorded native speech and simultaneously vocalizing the words, is a popular language-learning technique that is known to improve listening skills. However, despite strong evidence for its efficacy as a listening exercise, existing shadowing systems do not adequately support listening-focused practice, especially in self-regulated learning environments with no external feedback. To bridge this gap, we introduce Computer-Assisted Shadowing Trainer (CAST), a shadowing system that makes self-regulation easy and effective through four novel design elements — (i) in-the-moment highlights for tracking and visualizing progress, (ii) contextual blurring for inducing self-reflection on misheard words, (iii) self-listening comparators for post-practice self-evaluation, and (iv) adjustable pause-handles for self-paced practice. We base CAST on a formative user study (N=15) that provides fresh empirical grounds on the needs and challenges of shadowers. We validate our design through a summative evaluation (N=12) that shows learners can successfully self-regulate their shadowing practice with CAST while retaining focus on listening.
Digital Transformations of Classrooms in Virtual Reality
Authors: Hong Gao 0008, Efe Bozkir, Lisa Hasenbein, Jens-Uwe Hahn, Richard Göllner, Enkelejda Kasneci
Published at CHI, 2021

With rapid developments in consumer-level head-mounted displays and computer graphics, immersive VR has the potential to take online and remote learning closer to real-world settings. However, the effects of such digital transformations on learners, particularly for VR, have not been evaluated in depth. This work investigates the interaction-related effects of sitting positions of learners, visualization styles of peer-learners and teachers, and hand-raising behaviors of virtual peer-learners on learners in an immersive VR classroom, using eye tracking data. Our results indicate that learners sitting in the back of the virtual classroom may have difficulties extracting information. Additionally, we find indications that learners engage with lectures more efficiently if virtual avatars are visualized with realistic styles. Lastly, we find different eye movement behaviors towards different performance levels of virtual peer-learners, which should be investigated further. Our findings present an important baseline for design decisions for VR classrooms.
Does Interaction Improve Bayesian Reasoning with Visualization?
Authors: Abigail Mosca, Alvitta Ottley, Remco Chang
Published at CHI, 2021

Interaction enables users to navigate large amounts of data effectively, supports cognitive processing, and increases data representation methods. However, there have been few attempts to empirically demonstrate whether adding interaction to a static visualization improves its function beyond popular beliefs. In this paper, we address this gap. We use a classic Bayesian reasoning task as a testbed for evaluating whether allowing users to interact with a static visualization can improve their reasoning. Through two crowdsourced studies, we show that adding interaction to a static Bayesian reasoning visualization does not improve participants’ accuracy on a Bayesian reasoning task. In some cases, it can significantly detract from it. Moreover, we demonstrate that underlying visualization design modulates performance and that people with high versus low spatial ability respond differently to different interaction techniques and underlying base visualizations. Our work suggests that interaction is not as unambiguously good as we often believe; a well designed static visualization can be as, if not more, effective than an interactive one.
Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors
Authors: Niels van Berkel, Jorge Gonçalves 0001, Daniel Russo 0002, Simo Hosio, Mikael B. Skov
Published at CHI, 2021

The uptake of artificial intelligence-based applications raises concerns about the fairness and transparency of AI behaviour. Consequently, the Computer Science community calls for the involvement of the general public in the design and evaluation of AI systems. Assessing the fairness of individual predictors is an essential step in the development of equitable algorithms. In this study, we evaluate the effect of two common visualisation techniques (text-based and scatterplot) and the display of the outcome information (i.e., ground-truth) on the perceived fairness of predictors. Our results from an online crowdsourcing study (N = 80) show that the chosen visualisation technique significantly alters people’s fairness perception and that the presented scenario, as well as the participant’s gender and past education, influence perceived fairness. Based on these results we draw recommendations for future work that seeks to involve non-experts in AI fairness evaluations.
Effects of Semantic Segmentation Visualization on Trust, Situation Awareness, and Cognitive Load in Highly Automated Vehicles
Authors: Mark Colley, Benjamin Eder, Jan Ole Rixen, Enrico Rukzio
Published at CHI, 2021

Autonomous vehicles could improve mobility, safety, and inclusion in traffic. While this technology seems within reach, its successful introduction depends on the intended user’s acceptance. A substantial factor for this acceptance is trust in the autonomous vehicle’s capabilities. Visualizing internal information processed by an autonomous vehicle could calibrate this trust by enabling the perception of the vehicle’s detection capabilities (and its failures) while only inducing a low cognitive load. Additionally, the simultaneously raised situation awareness could benefit potential take-overs. We report the results of two comparative online studies on visualizing semantic segmentation information for the human user of autonomous vehicles. Effects on trust, cognitive load, and situation awareness were measured using a simulation (N=32) and state-of-the-art panoptic segmentation on a pre-recorded real-world video (N=41). Results show that the visualization using Augmented Reality increases situation awareness while remaining low cognitive load.
Falx: Synthesis-Powered Visualization Authoring
Authors: Chenglong Wang, Yu Feng 0001, Rastislav Bodík, Isil Dillig, Alvin Cheung, Amy J. Ko
Published at CHI, 2021

Modern visualization tools aim to allow data analysts to easily create exploratory visualizations. When the input data layout conforms to the visualization design, users can easily specify visualizations by mapping data columns to visual channels of the design. However, when there is a mismatch between data layout and the design, users need to spend significant effort on data transformation. We propose Falx, a synthesis-powered visualization tool that allows users to specify visualizations in a similarly simple way but without needing to worry about data layout. In Falx, users specify visualizations using examples of how concrete values in the input are mapped to visual channels, and Falx automatically infers the visualization specification and transforms the data to match the design. In a study with 33 data analysts on four visualization tasks involving data transformation, we found that users can effectively adopt Falx to create visualizations they otherwise cannot implement.
FashionQ: An AI-Driven Creativity Support Tool for Facilitating Ideation in Fashion Design
Authors: Youngseung Jeon, Seungwan Jin, Patrick C. Shih, Kyungsik Han
Published at CHI, 2021

Recent research on creativity support tools (CST) adopts artificial intelligence (AI) that leverages big data and computational capabilities to facilitate creative work. Our work aims to articulate the role of AI in supporting creativity with a case study of an AI-based CST tool in fashion design based on theoretical groundings. We developed AI models by externalizing three cognitive operations (extending, constraining, and blending) that are associated with divergent and convergent thinking. We present FashionQ, an AI-based CST that has three interactive visualization tools (StyleQ, TrendQ, and MergeQ). Through interviews and a user study with 20 fashion design professionals (10 participants for the interviews and 10 for the user study), we demonstrate the effectiveness of FashionQ on facilitating divergent and convergent thinking and identify opportunities and challenges of incorporating AI in the ideation process. Our findings highlight the role and use of AI in each cognitive operation based on professionals’ expertise and suggest future implications of AI-based CST development.
Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop
Authors: Anamaria Crisan, Brittany Fiore-Gartland
Published at CHI, 2021

AutoML systems can speed up routine data science work and make machine learning available to those without expertise in statistics and computer science. These systems have gained traction in enterprise settings where pools of skilled data workers are limited. In this study, we conduct interviews with 29 individuals from organizations of different sizes to characterize how they currently use, or intend to use, AutoML systems in their data science work. Our investigation also captures how data visualization is used in conjunction with AutoML systems. Our findings identify three usage scenarios for AutoML that resulted in a framework summarizing the level of automation desired by data workers with different levels of expertise. We surfaced the tension between speed and human oversight and found that data visualization can do a poor job balancing the two. Our findings have implications for the design and implementation of human-in-the-loop visual analytics approaches.
From Detectables to Inspectables: Understanding Qualitative Analysis of Audiovisual Data
Authors: Krishna Subramanian 0002, Johannes Maas, Jan O. Borchers, James D. Hollan
Published at CHI, 2021

Audiovisual recordings of user studies and interviews provide important data in qualitative HCI research. Even when a textual transcription is available, researchers frequently turn to these recordings due to their rich information content. However, the temporal, unstructured nature of audiovisual recordings makes them less efficient to work with than text. Through interviews and a survey, we explored how HCI researchers work with audiovisual recordings. We investigated researchers’ transcription and annotation practice, their overall analysis workflow, and the prevalence of direct analysis of audiovisual recordings. We found that a key task was locating and analyzing inspectables, interesting segments in recordings. Since locating inspectables can be time consuming, participants look for detectables, visual or auditory cues that indicate the presence of an inspectable. Based on our findings, we discuss the potential for automation in locating detectables in qualitative audiovisual analysis.
GestureMap: Supporting Visual Analytics and Quantitative Analysis of Motion Elicitation Data by Learning 2D Embeddings
Authors: Hai Dang, Daniel Buschek
Published at CHI, 2021

This paper presents GestureMap, a visual analytics tool for gesture elicitation which directly visualises the space of gestures. Concretely, a Variational Autoencoder embeds gestures recorded as 3D skeletons on an interactive 2D map. GestureMap further integrates three computational capabilities to connect exploration to quantitative measures: Leveraging DTW Barycenter Averaging (DBA), we compute average gestures to 1) represent gesture groups at a glance; 2) compute a new consensus measure (variance around average gesture); and 3) cluster gestures with k-means. We evaluate GestureMap and its concepts with eight experts and an in-depth analysis of published data. Our findings show how GestureMap facilitates exploring large datasets and helps researchers to gain a visual understanding of elicited gesture spaces. It further opens new directions, such as comparing elicitations across studies. We discuss implications for elicitation studies and research, and opportunities to extend our approach to additional tasks in gesture elicitation.
Grand Challenges in Immersive Analytics
Authors: Barrett Ens, Benjamin Bach, Maxime Cordeil, Ulrich Engelke, Marcos Serrano, Wesley Willett, Arnaud Prouzeau, Christoph Anthes, Wolfgang Büschel, Cody Dunne, Tim Dwyer, Jens Grubert, Jason H. Haga, Nurit Kirshenbaum, Dylan Kobayashi, Tica Lin, Monsurat Olaosebikan, Fabian Pointecker, David Saffo, Nazmus Saquib, Dieter Schmalstieg, Danielle Albers Szafir, Matt Whitlock, Yalong Yang 0001
Published at CHI, 2021

Immersive Analytics is a quickly evolving field that unites several areas such as visualisation, immersive environments, and human-computer interaction to support human data analysis with emerging technologies. This research has thrived over the past years with multiple workshops, seminars, and a growing body of publications, spanning several conferences. Given the rapid advancement of interaction technologies and novel application domains, this paper aims toward a broader research agenda to enable widespread adoption. We present 17 key research challenges developed over multiple sessions by a diverse group of 24 international experts, initiated from a virtual scientific workshop at ACM CHI 2020. These challenges aim to coordinate future work by providing a systematic roadmap of current directions and impending hurdles to facilitate productive and effective applications for Immersive Analytics.
Haptic and Visual Comprehension of a 2D Graph Layout Through Physicalisation
Authors: Adam Drogemuller, Andrew Cunningham, James A. Walsh, James Baumeister, Ross T. Smith, Bruce H. Thomas
Published at CHI, 2021

Data physicalisations afford people the ability to directly interact with data using their hands, potentially achieving a more comprehensive understanding of a dataset. Due to their complex nature, the representation of graphs and networks could benefit from physicalisation, bringing the dataset from the digital world into the physical one. However, no empirical work exists investigating the effects physicalisations have upon comprehension as they relate to graph representations. In this work, we present initial design considerations for graph physicalisations, as well as an empirical study investigating differences in comprehension between virtual and physical representations. We found that participants perceived themselves as being more accurate via touch and sight (visual-haptic) than the graphical-only modality, and perceived a triangle count task as less difficult in visual-haptic than in the graphical-only modality. Additionally, we found that participants significantly preferred interacting with visual-haptic over other conditions, despite no significant effect on task time or error.
IGScript: An Interaction Grammar for Scientific Data Presentation
Authors: Richen Liu, Min Gao, Shunlong Ye, Jiang Zhang 0002
Published at CHI, 2021

Most of the existing scientific visualizations toward interpretive grammar aim to enhance customizability in either the computation stage or the rendering stage or both, while few approaches focus on the data presentation stage. Besides, most of these approaches leverage the existing components from the general-purpose programming languages (GPLs) instead of developing a standalone compiler, which pose a great challenge about learning curves for the domain experts who have limited knowledge about programming. In this paper, we propose IGScript, a novel script-based interaction grammar tool, to help build scientific data presentation animations for communication. We design a dual-space interface and a compiler which converts natural language-like grammar statements or scripts into a data story animation to make an interactive customization on script-driven data presentations, and then develop a code generator (decompiler) to translate the interactive data exploration animations back into script codes to achieve statement parameters. IGScript makes the presentation animations editable, e.g., it allows to cut, copy, paste, append, or even delete some animation clips. We demonstrate the usability, customizability, and flexibility of IGScript by a user study, four case studies conducted by using four types of commonly-used scientific data, and performance evaluations.
Integrated Visualization Editing via Parameterized Declarative Templates
Authors: Andrew M. McNutt, Ravi Chugh
Published at CHI, 2021

Interfaces for creating visualizations typically embrace one of several common forms. Textual specification enables fine-grained control, shelf building facilitates rapid exploration, while chart choosing promotes immediacy and simplicity. Ideally these approaches could be unified to integrate the user- and usage-dependent benefits found in each modality, yet these forms remain distinct. We propose parameterized declarative templates, a simple abstraction mechanism over JSON-based visualization grammars, as a foundation for multimodal visualization editors. We demonstrate how templates can facilitate organization and reuse by factoring the more than 160 charts that constitute Vega-Lite’s example gallery into approximately 40 templates. We exemplify the pliability of abstracting over charting grammars by implementing—as a template—the functionality of the shelf builder Polestar (a simulacra of Tableau) and a set of templates that emulate the Google Sheets chart chooser. We show how templates support multimodal visualization editing by implementing a prototype and evaluating it through an approachability study.
Interpreting the Effect of Embellishment on Chart Visualizations
Authors: Tiffany Andry, Christophe Hurter, François Lambotte, Pierre Fastrez, Alexandru C. Telea
Published at CHI, 2021

Infographics range from minimalism that aims to convey the raw data to elaborately decorated, or embellished, graphics that aim to engage readers by telling a story. Several studies have shown evidence to negative, but also positive, effects on embellishments. We conducted a set of experiments to gauge more precisely how embellishments affect how people relate to infographics and make sense of the conveyed story. By analyzing questionnaires, interviews, and eye-tracking data simplified by bundling, we show that, within bounds, embellishments have a positive effect on how users get engaged in understanding an infographic, with very limited downside. To our knowledge, our work is the first that fuses the aforementioned three information sources to understand infographics. Our findings can help to design more fine-grained studies to quantify the effects of embellishments and also to design infographics that effectively use the embellishments’ positive aspects identified. I think the contribution does not appear well in the abstract. It’s not just that the visual embellishments are positive. We show a methodology that allows us to see what these effects are (in addition to engagement, memorization and recall) at several levels (scales, interviews, eye tracking) which are therefore physiological and emotional. We could include this idea in the abstract?
Investigating the Impact of Real-World Environments on the Perception of 2D Visualizations in Augmented Reality
Authors: Marc Satkowski, Raimund Dachselt
Published at CHI, 2021

In this work we report on two comprehensive user studies investigating the perception of Augmented Reality (AR) visualizations influenced by real-world backgrounds. Since AR is an emerging technology, it is important to also consider productive use cases, which is why we chose an exemplary and challenging industry 4.0 environment. Our basic perceptual research focuses on both the visual complexity of backgrounds as well as the influence of a secondary task. In contrast to our expectation, data of our 34 study participants indicate that the background has far less influence on the perception of AR visualizations. Moreover, we observed a mismatch between measured and subjectively reported performance. We discuss the importance of the background and recommendations for visual real-world augmentations. Overall, our results suggest that AR can be used in many visually challenging environments without losing the ability to productively work with the visualizations shown.
It's a Wrap: Toroidal Wrapping of Network Visualisations Supports Cluster Understanding Tasks
Authors: Kun-Ting Chen, Tim Dwyer, Benjamin Bach, Kim Marriott
Published at CHI, 2021

We explore network visualisation on a two-dimensional torus topology that continuously wraps when the viewport is panned. That is, links may be “wrapped” across the boundary, allowing additional spreading of node positions to reduce visual clutter. Recent work has investigated such pannable wrapped visualisations, finding them not worse than unwrapped drawings for small networks for path-following tasks. However, they did not evaluate larger networks nor did they consider whether torus-based layout might also better display high-level network structure like clusters. We offer two algorithms for improving toroidal layout that is completely autonomous and automatic panning of the viewport to minimiswe wrapping links. The resulting layouts afford fewer crossings, less stress, and greater cluster separation. In a study of 32 participants comparing performance in cluster understanding tasks, we find that toroidal visualisation offers significant benefits over standard unwrapped visualisation in terms of improvement in error by 62.7% and time by 32.3%.