DevLan/vispubs huggingface dataset:
WYTIWYR: A User Intent-Aware Framework with Multi-modal Inputs for Visualization Retrieval
Authors: Shishi Xiao, Yihan Hou, Cheng Jin, Wei Zeng 0004
Published at EuroVis, 2023

Retrieving charts from a large corpus is a fundamental task that can benefit numerous applications such as visualization recommendations. The retrieved results are expected to conform to both explicit visual attributes (e.g., chart type, colormap) and implicit user intents (e.g., design style, context information) that vary upon application scenarios. However, existing example‐based chart retrieval methods are built upon non‐decoupled and low‐level visual features that are hard to interpret, while definition‐based ones are constrained to pre‐defined attributes that are hard to extend. In this work, we propose a new framework, namely WYTIWYR (What‐You‐Think‐Is‐What‐You‐Retrieve), that integrates user intents into the chart retrieval process. The framework consists of two stages: first, the Annotation stage disentangles the visual attributes within the query chart; and second, the Retrieval stage embeds the user's intent with customized text prompt as well as bitmap query chart, to recall targeted retrieval result. We develop aprototype WYTIWYR system leveraging a contrastive language‐image pre‐training (CLIP) model to achieve zero‐shot classification as well as multi‐modal input encoding, and test the prototype on a large corpus with charts crawled from the Internet. Quantitative experiments, case studies, and qualitative interviews are conducted. The results demonstrate the usability and effectiveness of our proposed framework.
xOpat: eXplainable Open Pathology Analysis Tool
Authors: Jirí Horák, Katarína Furmanová, Barbora Kozlíková, Tomás Brázdil, Petr Holub, Martin Kacenga, Matej Gallo, Rudolf Nenutil, Jan Byska, Vít Rusnák
Published at EuroVis, 2023

Histopathology research quickly evolves thanks to advances in whole slide imaging (WSI) and artificial intelligence (AI). However, existing WSI viewers are tailored either for clinical or research environments, but none suits both. This hinders the adoption of new methods and communication between the researchers and clinicians. The paper presents xOpat, an open‐source, browser‐based WSI viewer that addresses these problems. xOpat supports various data sources, such as tissue images, pathologists' annotations, or additional data produced by AI models. Furthermore, it provides efficient rendering of multiple data layers, their visual representations, and tools for annotating and presenting findings. Thanks to its modular, protocol‐agnostic, and extensible architecture, xOpat can be easily integrated into different environments and thus helps to bridge the gap between research and clinical practice. To demonstrate the utility of xOpat, we present three case studies, one conducted with a developer of AI algorithms for image segmentation and two with a research pathologist.
"Explain What a Treemap is": Exploratory Investigation of Strategies for Explaining Unfamiliar Chart to Blind and Low Vision Users
Authors: Gyeongri Kim, Jiho Kim, Yea-Seul Kim
Published at CHI, 2023

Visualization designers increasingly use diverse types of visualizations, but assistive technologies and education for blind and low vision people often focus on elementary chart types. We explore textual explanation as a more generalizable solution. We define three dimensions of explanation strategies based on education theories: comparing to a familiar chart type, describing how to draw one, and using a concrete example. We develop a prototype system that automatically generates text explanations from a given chart specification. We conduct an exploratory study with 24 legally blind people to observe both the effectiveness and the perceived effectiveness of the strategies. The findings include: description of visual appearance is overall more effective than instructions for drawing, effective strategies differ by each chart type and by each participant, and the user’s perceived effectiveness does not always lead to better performance. We demonstrate the feasibility of an explanation generation system and compile design considerations.
"What else can I do?" Examining the Impact of Community Data on Adaptation and Quality of Reflection in an Educational Game
Authors: Erica Kleinman, Jennifer Villareale, Murtuza N. Shergadwala, Zhaoqing Teng, Andy Bryant, Jichen Zhu, Magy Seif El-Nasr
Published at CHI, 2023

Adaptation, or ability and willingness to consider an alternative approach, is a critical component of learning through reflection, especially in educational games, where there are often multiple avenues to success. As a domain, educational games have shown increased interest in using retrospective visualizations to promote and support reflection. Such visualizations, which can facilitate comparison with peer data, may also have an impact on adaptation in educational games. This has, however, not been empirically examined within the domain. In this work, we examine how comparison with other players’ data influenced adaptation, a part of reflection, in the context of a game that teaches parallel programming. Our results indicate that comparison with peers does significantly impact willingness to try a different approach, but suggest that there may also be other ways. We discuss what these results mean for future use of retrospective visualizations in educational games and present opportunities for future work.
"You Can See the Connections": Facilitating Visualization of Care Priorities in People Living with Multiple Chronic Health Conditions
Authors: Hyeyoung Ryu, Andrew B. L. Berry, Catherine Y. Lim, Andrea L. Hartzler, Tad Hirsch, Juanita I Trejo, Zoë Abigail Bermet, Brandi Crawford-Gallagher, Vi Tran, Dawn M. Ferguson, David J. Cronkite, Brooks Tiffany, John Weeks, James D. Ralston
Published at CHI, 2023

Individuals with multiple chronic health conditions (MCC) often face an overwhelming set of self-management work, resulting in a need to set care priorities. Yet, much self-management work is invisible to healthcare providers. This study aimed to understand how to support the development and sharing of connections between personal values and self-management tasks through the facilitated use of an interactive visualization system: Conversation Canvas. We conducted a field study with 13 participants with MCC, 3 caregivers, and 7 primary care providers in Washington State. Analysis of interviews with MCC participants showed that developing visualizations of connections between personal values, self-management tasks, and health conditions helped individuals make sense of connections relevant to their health and wellbeing, recognize a road map of central issues and their impacts, feel respected and understood, share priorities with providers, and support value-aligned changes. These findings demonstrated potential for the guided process and visualization to support priorities-aligned care.
A Need-Finding Study with Users of Geospatial Data
Authors: Parker Ziegler, Sarah E. Chasins
Published at CHI, 2023

Geospatial data is playing an increasingly critical role in the work of Earth and climate scientists, social scientists, and data journalists exploring spatiotemporal change in our environment and societies. However, existing software and programming tools for geospatial analysis and visualization are challenging to learn and difficult to use. The aim of this work is to identify the unmet computing needs of the diverse and expanding community of geospatial data users. We conducted a contextual inquiry study (n = 25) with domain experts using geospatial data in their current work. Through a thematic analysis, we found that participants struggled to (1) find and transform geospatial data to satisfy spatiotemporal constraints, (2) understand the behavior of geospatial operators, (3) track geospatial data provenance, and (4) explore the cartographic design space. These findings suggest design opportunities for developers and designers of geospatial analysis and visualization systems.
A Review and Collation of Graphical Perception Knowledge for Visualization Recommendation
Authors: Zehua Zeng, Leilani Battle
Published at CHI, 2023

Selecting appropriate visual encodings is critical to designing effective visualization recommendation systems, yet few findings from graphical perception are typically applied within these systems. We observe two significant limitations in translating graphical perception knowledge into actionable visualization recommendation rules/constraints: inconsistent reporting of findings and a lack of shared data across studies. How can we translate the graphical perception literature into a knowledge base for visualization recommendation? We present a review of 59 papers that study user perception and performance across ten visual analysis tasks. Through this study, we contribute a JSON dataset that collates existing theoretical and experimental knowledge and summarizes key study outcomes in graphical perception. We illustrate how this dataset can inform automated encoding decisions with three representative visualization recommendation systems. Based on our findings, we highlight open challenges and opportunities for the community in collating graphical perception knowledge for a range of visualization recommendation scenarios.
Accessible Data Representation with Natural Sound
Authors: Md. Naimul Hoque, Md Ehtesham-Ul-Haque, Niklas Elmqvist, Syed Masum Billah
Published at CHI, 2023

Sonification translates data into non-speech audio. Such auditory representations can make data visualization accessible to people who are blind or have low vision (BLV). This paper presents a sonification method for translating common data visualization into a blend of natural sounds. We hypothesize that people’s familiarity with sounds drawn from nature, such as birds singing in a forest, and their ability to listen to these sounds in parallel, will enable BLV users to perceive multiple data points being sonified at the same time. Informed by an extensive literature review and a preliminary study with 5 BLV participants, we designed an accessible data representation tool, Susurrus, that combines our sonification method with other accessibility features, such as keyboard interaction and text-to-speech feedback. Finally, we conducted a user study with 12 BLV participants and report the potential and application of natural sounds for sonification compared to existing sonification tools.
AeroRigUI: Actuated TUIs for Spatial Interaction using Rigging Swarm Robots on Ceilings in Everyday Space
Authors: Lilith Yu, Chenfeng Gao, David Wu, Ken Nakagaki
Published at CHI, 2023

We present AeroRigUI, an actuated tangible UI for 3D spatial embodied interaction. Using strings controlled by self-propelled swarm robots with a reeling mechanism on ceiling surfaces, our approach enables rigging (control through strings) physical objects’ position and orientation in the air. This can be applied to novel interactions in 3D space, including dynamic physical affordances, 3D information displays, and haptics. Utilizing the ceiling, an often underused room area, AeroRigUI can be applied for a range of applications such as room organization, data physicalization, and animated expressions. We demonstrate the applications based on our proof-of-concept prototype, which includes the hardware design of the rigging robots, named RigBots, and the software design for mid-air object control via interactive string manipulation. We also introduce technical evaluation and analysis of our approach prototype to address the hardware feasibility and safety. Overall, AeroRigUI enables a novel spatial and tangible UI system with great controllability and deployability.
AI Shall Have No Dominion: on How to Measure Technology Dominance in AI-supported Human decision-making
Authors: Federico Cabitza, Andrea Campagner, Riccardo Angius, Chiara Natali, Carlo Reverberi
Published at CHI, 2023

In this article, we propose a conceptual and methodological framework for measuring the impact of the introduction of AI systems in decision settings, based on the concept of technological dominance, i.e. the influence that an AI system can exert on human judgment and decisions. We distinguish between a negative component of dominance (automation bias) and a positive one (algorithm appreciation) by focusing on and systematizing the patterns of interaction between human judgment and AI support, or reliance patterns, and their associated cognitive effects. We then define statistical approaches for measuring these dimensions of dominance, as well as corresponding qualitative visualizations. By reporting about four medical case studies, we illustrate how the proposed methods can be used to inform assessments of dominance and of related cognitive biases in real-world settings. Our study lays the groundwork for future investigations into the effects of introducing AI support into naturalistic and collaborative decision-making.
Angler: Helping Machine Translation Practitioners Prioritize Model Improvements
Authors: Samantha Robertson, Zijie J. Wang, Dominik Moritz, Mary Beth Kery, Fred Hohman
Published at CHI, 2023

Machine learning (ML) models can fail in unexpected ways in the real world, but not all model failures are equal. With finite time and resources, ML practitioners are forced to prioritize their model debugging and improvement efforts. Through interviews with 13 ML practitioners at Apple, we found that practitioners construct small targeted test sets to estimate an error’s nature, scope, and impact on users. We built on this insight in a case study with machine translation models, and developed Angler, an interactive visual analytics tool to help practitioners prioritize model improvements. In a user study with 7 machine translation experts, we used Angler to understand prioritization practices when the input space is infinite, and obtaining reliable signals of model quality is expensive. Our study revealed that participants could form more interesting and user-focused hypotheses for prioritization by analyzing quantitative summary statistics and qualitatively assessing data by reading sentences.
AutoVis: Enabling Mixed-Immersive Analysis of Automotive User Interface Interaction Studies
Authors: Pascal Jansen, Julian Britten, Alexander Häusele, Thilo Segschneider, Mark Colley, Enrico Rukzio
Published at CHI, 2023

Automotive user interface (AUI) evaluation becomes increasingly complex due to novel interaction modalities, driving automation, heterogeneous data, and dynamic environmental contexts. Immersive analytics may enable efficient explorations of the resulting multilayered interplay between humans, vehicles, and the environment. However, no such tool exists for the automotive domain. With AutoVis, we address this gap by combining a non-immersive desktop with a virtual reality view enabling mixed-immersive analysis of AUIs. We identify design requirements based on an analysis of AUI research and domain expert interviews (N=5). AutoVis supports analyzing passenger behavior, physiology, spatial interaction, and events in a replicated study environment using avatars, trajectories, and heatmaps. We apply context portals and driving-path events as automotive-specific visualizations. To validate AutoVis against real-world analysis tasks, we implemented a prototype, conducted heuristic walkthroughs using authentic data from a case study and public datasets, and leveraged a real vehicle in the analysis process.
CALVI: Critical Thinking Assessment for Literacy in Visualizations
Authors: Lily W. Ge, Yuan Cui, Matthew Kay 0001
Published at CHI, 2023

Visualization misinformation is a prevalent problem, and combating it requires understanding people’s ability to read, interpret, and reason about erroneous or potentially misleading visualizations, which lacks a reliable measurement: existing visualization literacy tests focus on well-formed visualizations. We systematically develop an assessment for this ability by: (1) developing a precise definition of misleaders (decisions made in the construction of visualizations that can lead to conclusions not supported by the data), (2) constructing initial test items using a design space of misleaders and chart types, (3) trying out the provisional test on 497 participants, and (4) analyzing the test tryout results and refining the items using Item Response Theory, qualitative analysis, a wrong-due-to-misleader score, and the content validity index. Our final bank of 45 items shows high reliability, and we provide item bank usage recommendations for future tests and different use cases. Related materials are available at: https://osf.io/pv67z/.
Causalvis: Visualizations for Causal Inference
Authors: Grace Guo, Ehud Karavani, Alex Endert, Bum Chul Kwon
Published at CHI, 2023

Causal inference is a statistical paradigm for quantifying causal effects using observational data. It is a complex process, requiring multiple steps, iterations, and collaborations with domain experts. Analysts often rely on visualizations to evaluate the accuracy of each step. However, existing visualization toolkits are not designed to support the entire causal inference process within computational environments familiar to analysts. In this paper, we address this gap with Causalvis, a Python visualization package for causal inference. Working closely with causal inference experts, we adopted an iterative design process to develop four interactive visualization modules to support causal inference analysis tasks. The modules are then presented back to the experts for feedback and evaluation. We found that Causalvis effectively supported the iterative causal inference process. We discuss the implications of our findings for designing visualizations for causal inference, particularly for tasks of communication and collaboration.
Charagraph: Interactive Generation of Charts for Realtime Annotation of Data-Rich Paragraphs
Authors: Damien Masson, Sylvain Malacria, Géry Casiez, Daniel Vogel 0001
Published at CHI, 2023

Documents often have paragraphs packed with numbers that are difficult to extract, compare, and interpret. To help readers make sense of data in text, we introduce the concept of Charagraphs: dynamically generated interactive charts and annotations for in-situ visualization, comparison, and manipulation of numeric data included within text. Three Charagraph characteristics are defined: leveraging related textual information about data; integrating textual and graphical representations; and interacting at different contexts. We contribute a document viewer to select in-text data; generate and customize Charagraphs; merge and refine a Charagraph using other in-text data; and identify, filter, compare, and sort data synchronized between text and visualization. Results of a study show participants can easily create Charagraphs for diverse examples of data-rich text, and when answering questions about data in text, participants were more correct compared to only reading text.
Chart Reader: Accessible Visualization Experiences Designed with Screen Reader Users
Authors: John R. Thompson 0002, Jesse J. Martinez, Alper Sarikaya 0001, Edward Cutrell, Bongshin Lee
Published at CHI, 2023

Even though screen readers are a core accessibility tool for blind and low vision individuals (BLVIs), most visualizations are incompatible with screen readers. To improve accessible visualization experiences, we partnered with 10 BLV screen reader users (SRUs) in an iterative co-design study to design and develop accessible visualization experiences that afford SRUs the autonomy to interactively read and understand visualizations and their underlying data. During the five-month study, we explored accessible visualization prototypes with our design partners for three one-hour sessions. Our results provide feedback on the synthesized design concepts we explored, why (or why not) they aid comprehension and exploration for SRUs, and how differing design concepts can fit into cohesive accessible visualization experiences. We contribute both Chart Reader, a web-based accessibility engine resulting from our design iterations, and our distilled study findings—organized by design dimensions—in the creation of comprehensive accessible visualization experiences.
ChartDetective: Easy and Accurate Interactive Data Extraction from Complex Vector Charts
Authors: Damien Masson, Sylvain Malacria, Daniel Vogel 0001, Edward Lank, Géry Casiez
Published at CHI, 2023

Extracting underlying data from rasterized charts is tedious and inaccurate; values might be partially occluded or hard to distinguish, and the quality of the image limits the precision of the data being recovered. To address these issues, we introduce a semi-automatic system leveraging vector charts to extract the underlying data easily and accurately. The system is designed to make the most of vector information by relying on a drag-and-drop interface combined with selection, filtering, and previsualization features. A user study showed that participants spent less than 4 minutes to accurately recover data from charts published at CHI with diverse styles, thousands of data points, a combination of different encodings, and elements partially or completely occluded. Compared to other approaches relying on raster images, our tool successfully recovered all data, even when hidden, with a 78% lower relative error.
Collaborating Across Realities: Analytical Lenses for Understanding Dyadic Collaboration in Transitional Interfaces
Authors: Jan-Henrik Schröder, Daniel Schacht, Niklas Peper, Anita Marie Hamurculu, Hans-Christian Jetter
Published at CHI, 2023

Transitional Interfaces are a yet underexplored, emerging class of cross-reality user interfaces that enable users to freely move along the reality-virtuality continuum during collaboration. To analyze and understand how such collaboration unfolds, we propose four analytical lenses derived from an exploratory study of transitional collaboration with 15 dyads. While solving a complex spatial optimization task, participants could freely switch between three contexts, each with different displays (desktop screens, tablet-based augmented reality, head-mounted virtual reality), input techniques (mouse, touch, handheld controllers), and visual representations (monoscopic and allocentric 2D/3D maps, stereoscopic egocentric views). Using the rich qualitative and quantitative data from our study, we evaluated participants’ perceptions of transitional collaboration and identified commonalities and differences between dyads. We then derived four lenses including metrics and visualizations to analyze key aspects of transitional collaboration: (1) place and distance, (2) temporal patterns, (3) group use of contexts, (4) individual use of contexts.
Collaboration with Conversational AI Assistants for UX Evaluation: Questions and How to Ask them (Voice vs. Text)
Authors: Emily Kuang, Ehsan Jahangirzadeh Soure, Mingming Fan 0001, Jian Zhao 0010, Kristen Shinohara
Published at CHI, 2023

AI is promising in assisting UX evaluators with analyzing usability tests, but its judgments are typically presented as non-interactive visualizations. Evaluators may have questions about test recordings, but have no way of asking them. Interactive conversational assistants provide a Q&A dynamic that may improve analysis efficiency and evaluator autonomy. To understand the full range of analysis-related questions, we conducted a Wizard-of-Oz design probe study with 20 participants who interacted with simulated AI assistants via text or voice. We found that participants asked for five categories of information: user actions, user mental model, help from the AI assistant, product and task information, and user demographics. Those who used the text assistant asked more questions, but the question lengths were similar. The text assistant was perceived as significantly more efficient, but both were rated equally in satisfaction and trust. We also provide design considerations for future conversational AI assistants for UX evaluation.