DevLan/vispubs huggingface dataset:
Understanding Data Accessibility for People with Intellectual and Developmental Disabilities
Authors: Keke Wu, Emma Petersen, Tahmina Ahmad, David Burlinson, Shea Tanis, Danielle Albers Szafir
Published at CHI, 2021

Using visualization requires people to read abstract visual imagery, estimate statistics, and retain information. However, people with Intellectual and Developmental Disabilities (IDD) often process information differently, which may complicate connecting abstract visual information to real-world quantities. This population has traditionally been excluded from visualization design, and often has limited access to data related to their well being. We explore how visualizations may better serve this population. We identify three visualization design elements that may improve data accessibility: chart type, chart embellishment, and data continuity. We evaluate these elements with populations both with and without IDD, measuring accuracy and efficiency in a web-based online experiment with time series and proportion data. Our study identifies performance patterns and subjective preferences for people with IDD when reading common visualizations. These findings suggest possible solutions that may break the cognitive barriers caused by conventional design guidelines.
Understanding Narrative Linearity for Telling Expressive Time-Oriented Stories
Authors: Xingyu Lan, Xinyue Xu, Nan Cao
Published at CHI, 2021

Creating expressive narrative visualization often requires choosing a well-planned narrative order that invites the audience in. The narrative can either follow the linear order of story events (chronology), or deviate from linearity (anachronies). While evidence exists that anachronies in novels and films can enhance story expressiveness, little is known about how they can be incorporated into narrative visualization. To bridge this gap, this work introduces the idea of narrative linearity to visualization and investigates how different narrative orders affect the expressiveness of time-oriented stories. First, we conducted preliminary interviews with seven experts to clarify the motivations and challenges of manipulating narrative linearity in time-oriented stories. Then, we analyzed a corpus of 80 time-oriented stories and identified six most salient patterns of narrative orders. Next, we conducted a crowdsourcing study with 221 participants. Results indicated that anachronies have the potential to make time-oriented stories more expressive without hindering comprehensibility.
Understanding Trigger-Action Programs Through Novel Visualizations of Program Differences
Authors: Valerie Zhao, Lefan Zhang, Bo Wang, Michael L. Littman, Shan Lu 0001, Blase Ur
Published at CHI, 2021

Trigger-action programming (if-this-then-that rules) empowers non-technical users to automate services and smart devices. As a user’s set of trigger-action programs evolves, the user must reason about behavior differences between similar programs, such as between an original program and several modification candidates, to select programs that meet their goals. To facilitate this process, we co-designed user interfaces and underlying algorithms to highlight differences between trigger-action programs. Our novel approaches leverage formal methods to efficiently identify and visualize differences in program outcomes or abstract properties. We also implemented a traditional interface that shows only syntax differences in the rules themselves. In a between-subjects online experiment with 107 participants, the novel interfaces better enabled participants to select trigger-action programs matching intended goals in complex, yet realistic, situations that proved very difficult when using traditional interfaces showing syntax differences.
Viral Visualizations: How Coronavirus Skeptics Use Orthodox Data Practices to Promote Unorthodox Science Online
Authors: Crystal Lee, Tanya Yang, Gabrielle D. Inchoco, Graham M. Jones, Arvind Satyanarayan
Published at CHI, 2021

Controversial understandings of the coronavirus pandemic have turned data visualizations into a battleground. Defying public health officials, coronavirus skeptics on US social media spent much of 2020 creating data visualizations showing that the government’s pandemic response was excessive and that the crisis was over. This paper investigates how pandemic visualizations circulated on social media, and shows that people who mistrust the scientific establishment often deploy the same rhetorics of data-driven decision-making used by experts, but to advocate for radical policy changes. Using a quantitative analysis of how visualizations spread on Twitter and an ethnographic approach to analyzing conversations about COVID data on Facebook, we document an epistemological gap that leads pro- and anti-mask groups to draw drastically different inferences from similar data. Ultimately, we argue that the deployment of COVID data visualizations reflect a deeper sociopolitical rift regarding the place of science in public life.
Vis Ex Machina: An Analysis of Trust in Human versus Algorithmically Generated Visualization Recommendations
Authors: Rachael Zehrung, Astha Singhal, Michael Correll, Leilani Battle
Published at CHI, 2021

More visualization systems are simplifying the data analysis process by automatically suggesting relevant visualizations. However, little work has been done to understand if users trust these automated recommendations. In this paper, we present the results of a crowd-sourced study exploring preferences and perceived quality of recommendations that have been positioned as either human-curated or algorithmically generated. We observe that while participants initially prefer human recommenders, their actions suggest an indifference for recommendation source when evaluating visualization recommendations. The relevance of presented information (e.g., the presence of certain data fields) was the most critical factor, followed by a belief in the recommender’s ability to create accurate visualizations. Our findings suggest a general indifference towards the provenance of recommendations, and point to idiosyncratic definitions of visualization quality and trustworthiness that may not be captured by simple measures. We suggest that recommendation systems should be tailored to the information-foraging strategies of specific users.
Visualizing Examples of Deep Neural Networks at Scale
Authors: Litao Yan, Elena L. Glassman, Tianyi Zhang 0001
Published at CHI, 2021

Many programmers want to use deep learning due to its superior accuracy in many challenging domains. Yet our formative study with ten programmers indicated that, when constructing their own deep neural networks (DNNs), they often had a difficult time choosing appropriate model structures and hyperparameter values. This paper presents ExampleNet—a novel interactive visualization system for exploring common and uncommon design choices in a large collection of open-source DNN projects. ExampleNet provides a holistic view of the distribution over model structures and hyperparameter settings in the corpus of DNNs, so users can easily filter the corpus down to projects tackling similar tasks and compare design choices made by others. We evaluated ExampleNet in a within-subjects study with sixteen participants. Compared with the control condition (i.e., online search), participants using ExampleNet were able to inspect more online examples, make more data-driven design decisions, and make fewer design mistakes.
What Players Want: Information Needs of Players on Post-Game Visualizations
Authors: Günter Wallner, Marnix van Wijland, Regina Bernhaupt, Simone Kriglstein
Published at CHI, 2021

With the rise of competitive online gaming and esports, players’ ability to review, reflect upon, and improve their in-game performance has become important. Post-play visualizations are key for such improvements. Despite the increased interest in visualizations of gameplay, research specifically informing the design of player-centric visualizations is currently limited. As with all visualizations, their design should, however, be guided by a thorough understanding of the goals to be achieved and which information is important and why. This paper reports on a mixed-methods study exploring the information demands posed by players on post-play visualizations and the goals they pursue with such visualizations. We focused on three genres that enjoy great popularity within the competitive gaming scene. Our results provide useful guideposts on which data to focus on by offering an overview of the relevance of different in-game metrics across genres. Lastly, we outline high-level implications for the design of post-play visualizations.
A Visual Analytics Approach for Ecosystem Dynamics based on Empirical Dynamic Modeling
Authors: Hiroaki Natsukawa, Ethan R. Deyle, Gerald M. Pao, Koji Koyamada, George Sugihara
Published at VAST, 2020

An important approach for scientific inquiry across many disciplines involves using observational time series data to understand the relationships between key variables to gain mechanistic insights into the underlying rules that govern the given system. In real systems, such as those found in ecology, the relationships between time series variables are generally not static; instead, these relationships are dynamical and change in a nonlinear or state-dependent manner. To further understand such systems, we investigate integrating methods that appropriately characterize these dynamics (i.e., methods that measure interactions as they change with time-varying system states) with visualization techniques that can help analyze the behavior of the system. Here, we focus on empirical dynamic modeling (EDM) as a state-of-the-art method that specifically identifies causal variables and measures changing state-dependent relationships between time series variables. Instead of using approaches centered on parametric equations, EDM is an equation-free approach that studies systems based on their dynamic attractors. We propose a visual analytics system to support the identification and mechanistic interpretation of system states using an EDM-constructed dynamic graph. This work, as detailed in four analysis tasks and demonstrated with a GUI, provides a novel synthesis of EDM and visualization techniques such as brush-link visualization and visual summarization to interpret dynamic graphs representing ecosystem dynamics. We applied our proposed system to ecological simulation data and real data from a marine mesocosm study as two key use cases. Our case studies show that our visual analytics tools support the identification and interpretation of the system state by the user, and enable us to discover both confirmatory and new findings in ecosystem dynamics. Overall, we demonstrated that our system can facilitate an understanding of how systems function beyond the intuitive analysis of high-dimensional information based on specific domain knowledge.
A Visual Analytics Approach for Exploratory Causal Analysis: Exploration, Validation, and Applications
Authors: Xiao Xie, Fan Du, Yingcai Wu
Published at VAST, 2020

Using causal relations to guide decision making has become an essential analytical task across various domains, from marketing and medicine to education and social science. While powerful statistical models have been developed for inferring causal relations from data, domain practitioners still lack effective visual interface for interpreting the causal relations and applying them in their decision-making process. Through interview studies with domain experts, we characterize their current decision-making workflows, challenges, and needs. Through an iterative design process, we developed a visualization tool that allows analysts to explore, validate, and apply causal relations in real-world decision-making scenarios. The tool provides an uncertainty-aware causal graph visualization for presenting a large set of causal relations inferred from high-dimensional data. On top of the causal graph, it supports a set of intuitive user controls for performing what-if analyses and making action plans. We report on two case studies in marketing and student advising to demonstrate that users can effectively explore causal relations and design action plans for reaching their goals.
A Visual Analytics Approach to Debugging Cooperative, Autonomous Multi-Robot Systems’ Worldviews
Authors: Sandra Bae, Federico Rossi 0001, Joshua Vander Hook, Scott Davidoff, Kwan-Liu Ma
Published at VAST, 2020

Autonomous multi-robot systems, where a team of robots shares information to perform tasks that are beyond an individual robot’s abilities, hold great promise for a number of applications, such as planetary exploration missions. Each robot in a multi-robot system that uses the shared-world coordination paradigm autonomously schedules which robot should perform a given task, and when, using its worldview–the robot’s internal representation of its belief about both its own state, and other robots’ states. A key problem for operators is that robots’ worldviews can fall out of sync (often due to weak communication links), leading to desynchronization of the robots’ scheduling decisions and inconsistent emergent behavior (e.g., tasks not performed, or performed by multiple robots). Operators face the time-consuming and difficult task of making sense of the robots’ scheduling decisions, detecting de-synchronizations, and pinpointing the cause by comparing every robot’s worldview. To address these challenges, we introduce MOSAIC Viewer, a visual analytics system that helps operators (i) make sense of the robots’ schedules and (ii) detect and conduct a root cause analysis of the robots’ desynchronized worldviews. Over a year-long partnership with roboticists at the NASA Jet Propulsion Laboratory, we conduct a formative study to identify the necessary system design requirements and a qualitative evaluation with 12 roboticists. We find that MOSAIC Viewer is faster- and easier-to-use than the users’ current approaches, and it allows them to stitch low-level details to formulate a high-level understanding of the robots’ schedules and detect and pinpoint the cause of the desynchronized worldviews.
A Visual Analytics Framework for Contrastive Network Analysis
Authors: Takanori Fujiwara, Jian Zhao 0010, Francine Chen 0001, Kwan-Liu Ma
Published at VAST, 2020

A common network analysis task is comparison of two networks to identify unique characteristics in one network with respect to the other. For example, when comparing protein interaction networks derived from normal and cancer tissues, one essential task is to discover protein-protein interactions unique to cancer tissues. However, this task is challenging when the networks contain complex structural (and semantic) relations. To address this problem, we design ContraNA, a visual analytics framework leveraging both the power of machine learning for uncovering unique characteristics in networks and also the effectiveness of visualization for understanding such uniqueness. The basis of ContraNA is cNRL, which integrates two machine learning schemes, network representation learning (NRL) and contrastive learning (CL), to generate a low-dimensional embedding that reveals the uniqueness of one network when compared to another. ContraNA provides an interactive visualization interface to help analyze the uniqueness by relating embedding results and network structures as well as explaining the learned features by cNRL. We demonstrate the usefulness of ContraNA with two case studies using real-world datasets. We also evaluate ContraNA through a controlled user study with 12 participants on network comparison tasks. The results show that participants were able to both effectively identify unique characteristics from complex networks and interpret the results obtained from cNRL.
A Visual Analytics Framework for Explaining and Diagnosing Transfer Learning Processes
Authors: Yuxin Ma, Arlen Fan, Jingrui He, Arun Reddy Nelakurthi, Ross Maciejewski
Published at VAST, 2020

Many statistical learning models hold an assumption that the training data and the future unlabeled data are drawn from the same distribution. However, this assumption is difficult to fulfill in real-world scenarios and creates barriers in reusing existing labels from similar application domains. Transfer Learning is intended to relax this assumption by modeling relationships between domains, and is often applied in deep learning applications to reduce the demand for labeled data and training time. Despite recent advances in exploring deep learning models with visual analytics tools, little work has explored the issue of explaining and diagnosing the knowledge transfer process between deep learning models. In this paper, we present a visual analytics framework for the multi-level exploration of the transfer learning processes when training deep neural networks. Our framework establishes a multi-aspect design to explain how the learned knowledge from the existing model is transferred into the new learning task when training deep neural networks. Based on a comprehensive requirement and task analysis, we employ descriptive visualization with performance measures and detailed inspections of model behaviors from the statistical, instance, feature, and model structure levels. We demonstrate our framework through two case studies on image classification by fine-tuning AlexNets to illustrate how analysts can utilize our framework.
A Visual Analytics Framework for Reviewing Multivariate Time-Series Data with Dimensionality Reduction
Authors: Takanori Fujiwara, Shilpika, Naohisa Sakamoto, Jorji Nonaka, Keiji Yamamoto, Kwan-Liu Ma
Published at VAST, 2020

Data-driven problem solving in many real-world applications involves analysis of time-dependent multivariate data, for which dimensionality reduction (DR) methods are often used to uncover the intrinsic structure and features of the data. However, DR is usually applied to a subset of data that is either single-time-point multivariate or univariate time-series, resulting in the need to manually examine and correlate the DR results out of different data subsets. When the number of dimensions is large either in terms of the number of time points or attributes, this manual task becomes too tedious and infeasible. In this paper, we present MulTiDR, a new DR framework that enables processing of time-dependent multivariate data as a whole to provide a comprehensive overview of the data. With the framework, we employ DR in two steps. When treating the instances, time points, and attributes of the data as a 3D array, the first DR step reduces the three axes of the array to two, and the second DR step visualizes the data in a lower-dimensional space. In addition, by coupling with a contrastive learning method and interactive visualizations, our framework enhances analysts' ability to interpret DR results. We demonstrate the effectiveness of our framework with four case studies using real-world datasets.
An Examination of Grouping and Spatial Organization Tasks for High-Dimensional Data Exploration
Authors: John E. Wenskovitch, Chris North 0001
Published at VAST, 2020

How do analysts think about grouping and spatial operations? This overarching research question incorporates a number of points for investigation, including understanding how analysts begin to explore a dataset, the types of grouping/spatial structures created and the operations performed on them, the relationship between grouping and spatial structures, the decisions analysts make when exploring individual observations, and the role of external information. This work contributes the design and results of such a study, in which a group of participants are asked to organize the data contained within an unfamiliar quantitative dataset. We identify several overarching approaches taken by participants to design their organizational space, discuss the interactions performed by the participants, and propose design recommendations to improve the usability of future high-dimensional data exploration tools that make use of grouping (clustering) and spatial (dimension reduction) operations.
Argus: Interactive a priori Power Analysis
Authors: Xiaoyi Wang, Alexander Eiselmayer, Wendy E. Mackay, Kasper Hornbæk, Chat Wacharamanotham
Published at VAST, 2020

A key challenge HCl researchers face when designing a controlled experiment is choosing the appropriate number of participants, or sample size. A priori power analysis examines the relationships among multiple parameters, including the complexity associated with human participants, e.g., order and fatigue effects, to calculate the statistical power of a given experiment design. We created Argus, a tool that supports interactive exploration of statistical power: Researchers specify experiment design scenarios with varying confounds and effect sizes. Argus then simulates data and visualizes statistical power across these scenarios, which lets researchers interactively weigh various trade-offs and make informed decisions about sample size. We describe the design and implementation of Argus, a usage scenario designing a visualization experiment, and a think-aloud study.
Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models
Authors: Joseph F. DeRose, Jiayao Wang, Matthew Berger
Published at VAST, 2020

Advances in language modeling have led to the development of deep attention-based models that are performant across a wide variety of natural language processing (NLP) problems. These language models are typified by a pre-training process on large unlabeled text corpora and subsequently fine-tuned for specific tasks. Although considerable work has been devoted to understanding the attention mechanisms of pre-trained models, it is less understood how a model's attention mechanisms change when trained for a target NLP task. In this paper, we propose a visual analytics approach to understanding fine-tuning in attention-based language models. Our visualization, Attention Flows, is designed to support users in querying, tracing, and comparing attention within layers, across layers, and amongst attention heads in Transformer-based language models. To help users gain insight on how a classification decision is made, our design is centered on depicting classification-based attention at the deepest layer and how attention from prior layers flows throughout words in the input. Attention Flows supports the analysis of a single model, as well as the visual comparison between pre-trained and fine-tuned models via their similarities and differences. We use Attention Flows to study attention mechanisms in various sentence understanding tasks and highlight how attention evolves to address the nuances of solving these tasks.
Auditing the Sensitivity of Graph-based Ranking with Visual Analytics
Authors: Tiankai Xie, Yuxin Ma, Hanghang Tong, My T. Thai, Ross Maciejewski
Published at VAST, 2020

Graph mining plays a pivotal role across a number of disciplines, and a variety of algorithms have been developed to answer who/what type questions. For example, what items shall we recommend to a given user on an e-commerce platform? The answers to such questions are typically returned in the form of a ranked list, and graph-based ranking methods are widely used in industrial information retrieval settings. However, these ranking algorithms have a variety of sensitivities, and even small changes in rank can lead to vast reductions in product sales and page hits. As such, there is a need for tools and methods that can help model developers and analysts explore the sensitivities of graph ranking algorithms with respect to perturbations within the graph structure. In this paper, we present a visual analytics framework for explaining and exploring the sensitivity of any graph-based ranking algorithm by performing perturbation-based what-if analysis. We demonstrate our framework through three case studies inspecting the sensitivity of two classic graph-based ranking algorithms (PageRank and HITS) as applied to rankings in political news media and social networks.