DevLan/vispubs huggingface dataset:
MRAT: The Mixed Reality Analytics Toolkit
Authors: Michael Nebeling, Maximilian Speicher, Xizi Wang, Shwetha Rajaram, Brian D. Hall, Zijian Xie, Alexander R. E. Raistrick, Michelle Aebersold, Edward G. Happ, Jiayin Wang, Yanan Sun, Lotus Zhang, Leah E. Ramsier, Rhea Kulkarni
Published at CHI, 2020

Significant tool support exists for the development of mixed reality (MR) applications; however, there is a lack of tools for analyzing MR experiences. We elicit requirements for future tools through interviews with 8 university research, instructional, and media teams using AR/VR in a variety of domains. While we find a common need for capturing how users perform tasks in MR, the primary differences were in terms of heuristics and metrics relevant to each project. Particularly in the early project stages, teams were uncertain about what data should, and even could, be collected with MR technologies. We designed the Mixed Reality Analytics Toolkit (MRAT) to instrument MR apps via visual editors without programming and enable rapid data collection and filtering for visualizations of MR user sessions. With MRAT, we contribute flexible interaction tracking and task definition concepts, an extensible set of heuristic techniques and metrics to measure task success, and visual inspection tools with in-situ visualizations in MR. Focusing on a multi-user, cross-device MR crisis simulation and triage training app as a case study, we then show the benefits of using MRAT, not only for user testing of MR apps, but also performance tuning throughout the design process.
Paths Explored, Paths Omitted, Paths Obscured: Decision Points & Selective Reporting in End-to-End Data Analysis
Authors: Yang Liu 0136, Tim Althoff, Jeffrey Heer
Published at CHI, 2020

Drawing reliable inferences from data involves many, sometimes arbitrary, decisions across phases of data collection, wrangling, and modeling. As different choices can lead to diverging conclusions, understanding how researchers make analytic decisions is important for supporting robust and replicable analysis. In this study, we pore over nine published research studies and conduct semi-structured interviews with their authors. We observe that researchers often base their decisions on methodological or theoretical concerns, but subject to constraints arising from the data, expertise, or perceived interpretability. We confirm that researchers may experiment with choices in search of desirable results, but also identify other reasons why researchers explore alternatives yet omit findings. In concert with our interviews, we also contribute visualizations for communicating decision processes throughout an analysis. Based on our results, we identify design opportunities for strengthening end-to-end analysis, for instance via tracking and meta-analysis of multiple decision paths.
Prior Setting in Practice: Strategies and Rationales Used in Choosing Prior Distributions for Bayesian Analysis
Authors: Abhraneel Sarma, Matthew Kay 0001
Published at CHI, 2020

Bayesian statistical analysis is steadily growing in popularity and use. Choosing priors is an integral part of Bayesian inference. While there exist extensive normative recommendations for prior setting, little is known about how priors are chosen in practice. We conducted a survey (N = 50) and interviews (N = 9) where we used interactive visualizations to elicit prior distributions from researchers experienced withBayesian statistics and asked them for rationales for those priors. We found that participants' experience and philosophy influence how much and what information they are willing to incorporate into their priors, manifesting as different levels of informativeness and skepticism. We also identified three broad strategies participants use to set their priors: centrality matching, interval matching, and visual mass allocation. We discovered that participants' understanding of the notion of 'weakly informative priors"-a commonly-recommended normative approach to prior setting-manifests very differently across participants. Our results have implications both for how to develop prior setting recommendations and how to design tools to elicit priors in Bayesian analysis.
Progression Maps: Conceptualizing Narrative Structure for Interaction Design Support
Authors: Elín Carstensdóttir, Nathan Partlan, Steven C. Sutherland, Tyler Duke, Erika Ferris, Robin M. Richter, Maria Jose Valladares, Magy Seif El-Nasr
Published at CHI, 2020

Interactive narratives are frequently designed for learning and training applications, such as social training. In these contexts, designers may be inexperienced in storytelling and interaction design, and it may be difficult to quickly build an effective experience, even for experienced designers. Designers often approach this problem through iterative design. To augment and reduce iteration, we argue for the utility of employing models to reason about, evaluate, and improve designs. While there has been much previous work on interactive narrative models, none of them capture aspects of the interaction design necessary for testing and evaluation. In this paper we propose a new computational model called Progression Maps, which abstracts interaction design elements of the narrative's structure and visualizes its interaction properties. We report on the model, its implementation, and two studies evaluating its use. Our results demonstrate Progression Maps' effectiveness in communicating the underlying design through an easily understandable visualization.
Projection Boxes: On-the-fly Reconfigurable Visualization for Live Programming
Authors: Sorin Lerner
Published at CHI, 2020

Live programming is a regime in which the programming environment provides continual feedback, most often in the form of runtime values. In this paper, we present Projection Boxes, a novel visualization technique for displaying runtime values of programs. The key idea behind projection boxes is to start with a full semantics of the program, and then use projections to pick a subset of the semantics to display. By varying the projection used, projection boxes can encode both previously known visualization techniques, and also new ones. As such, projection boxes provide an expressive and configurable framework for displaying runtime information. Through a user study we demonstrate that (1) users find projection boxes and their configurability useful (2) users are not distracted by the always-on visualization (3) a key driving force behind the need for a configurable visualization for live programming lies with the wide variation in programmer preferences.
Pushing the (Visual) Narrative: The Effects of Prior Knowledge Elicitation in Provocative Topics
Authors: Jeremy Heyer, Nirmal Kumar Raveendranath, Khairi Reda
Published at CHI, 2020

Narrative visualization is a popular style of data-driven storytelling. Authors use this medium to engage viewers with complex and sometimes controversial issues. A challenge for authors is to not only deliver new information, but to also overcome people's biases and misconceptions. We study how people adjust their attitudes toward (or away from) a message experienced through a narrative visualization. In a mixed-methods analysis, we investigate whether eliciting participants' prior beliefs, and visualizing those beliefs alongside actual data, can increase narrative persuasiveness. We find that incorporating priors does not significantly affect attitudinal change. However, participants who externalized their beliefs expressed greater surprise at the data. Their comments also indicated a greater likelihood of acquiring new information, despite the minimal change in attitude. Our results also extend prior findings, showing that visualizations are more persuasive than equivalent textual data representations for exposing contentious issues. We discuss the implications and outline future research directions.
QMaps: Engaging Students in Voluntary Question Generation and Linking
Authors: Iman YeckehZaare, Tirdad Barghi, Paul Resnick
Published at CHI, 2020

Generating multiple-choice questions is known to improve students' critical thinking and deep learning. Visualizing relationships between concepts enhances meaningful learning, students' ability to relate new concepts to previously learned concepts. We designed and deployed a collaborative learning process through which students generate multiple-choice questions and represent the prerequisite knowledge structure between questions as visual links in a shared map, using a variation of Concept Maps that we call "QMap." We conducted a four-month study with 19 undergraduate students. Students sustained voluntary contributions, creating 992 good questions, and drawing 1,255 meaningful links between the questions. Through analyzing self-reports, observations, and usage data, we report on the technical and social design features that led students to sustain their motivation.
RunAhead: Exploring Head Scanning based Navigation for Runners
Authors: Danilo Gallo, Shreepriya Shreepriya, Jutta Willamowski
Published at CHI, 2020

Navigation systems for runners commonly provide turn-by-turn directions via voice and/or map-based visualizations. While voice directions require permanent attention, map-based guidance requires regular consultation. Both disrupt the running activity. To address this, we designed RunAhead, a navigation system using head scanning to query for navigation feedback, and we explored its suitability for runners in an outdoor experiment. In our design, we provide the runner with simple and intuitive navigation feedback on the path s/he is looking at through three different feedback modes: haptic, music and audio cues. In our experiment, we compare the resulting three versions of RunAhead with a baseline voice-based navigation system. We find that demand and error are equivalent across all four conditions. However, the head scanning based haptic and music conditions are preferred over the baseline and these preferences are impacted by runners' habits. With this study we contribute insights for designing navigation support for runners.
See, Feel, Move: Player Behaviour Analysis through Combined Visualization of Gaze, Emotions, and Movement
Authors: Daniel Kepplinger, Günter Wallner, Simone Kriglstein, Michael Lankes
Published at CHI, 2020

Playtesting of games often relies on a mixed-methods approach to obtain more holistic insights about and, in turn, improve the player experience. However, triangulating the different data sources and visualizing them in an integrated manner such that they contextualize each other still proves challenging. Despite its potential value for gauging player behaviour, this area of research continues to be underexplored. In this paper, we propose a visualization approach that combines commonly tracked movement data with - from a visualization perspective rarely considered - gaze behaviour and emotional responses. We evaluated our approach through a qualitative expert study with five professional game developers. Our results show that both the individual visualization of gaze, emotions, and movement but especially their combination are valuable to understand and form hypotheses about player behaviour. At the same time, our results stress that careful attention needs to be paid to ensure that the visualization remains legible and does not obfuscate information.
Surfacing Visualization Mirages
Authors: Andrew M. McNutt, Gordon L. Kindlmann, Michael Correll
Published at CHI, 2020

Dirty data and deceptive design practices can undermine, invert, or invalidate the purported messages of charts and graphs. These failures can arise silently: a conclusion derived from a particular visualization may look plausible unless the analyst looks closer and discovers an issue with the backing data, visual specification, or their own assumptions. We term such silent but significant failures . We describe a conceptual model of mirages and show how they can be generated at every stage of the visual analytics process. We adapt a methodology from software testing, , as a way of automatically surfacing potential mirages at the visual encoding stage of analysis through modifications to the underlying data and chart specification. We show that metamorphic testing can reliably identify mirages across a variety of chart types with relatively little prior knowledge of the data or the domain.
Tactile Presentation of Network Data: Text, Matrix or Diagram?
Authors: Yalong Yang 0001, Kim Marriott, Matthew Butler 0002, Cagatay Goncu, Leona Holloway
Published at CHI, 2020

Visualisations are commonly used to understand social, biological and other kinds of networks. Currently we do not know how to effectively present network data to people who are blind or have low-vision (BLV). We ran a controlled study with 8 BLV participants comparing four tactile representations: organic node-link diagram, grid node-link diagram, adjacency matrix and braille list. We found that the node-link representations were preferred and more effective for path following and cluster identification while the matrix and list were better for adjacency tasks. This is broadly in line with findings for the corresponding visual representations.
Techniques for Flexible Responsive Visualization Design
Authors: Jane Hoffswell, Wilmot Li, Zhicheng Liu 0001
Published at CHI, 2020

Responsive visualizations adapt to effectively present information based on the device context. Such adaptations are essential for news content that is increasingly consumed on mobile devices. However, existing tools provide little support for responsive visualization design. We analyze a corpus of 231 responsive news visualizations and discuss formative interviews with five journalists about responsive visualization design. These interviews motivate four central design guidelines: enable simultaneous cross-device edits, facilitate device-specific customization, show cross-device previews, and support propagation of edits. Based on these guidelines, we present a prototype system that allows users to preview and edit multiple visualization versions simultaneously. We demonstrate the utility of the system features by recreating four real-world responsive visualizations from our corpus.
Toward Automated Feedback on Teacher Discourse to Enhance Teacher Learning
Authors: Emily Jensen, Meghan Dale, Patrick J. Donnelly, Cathlyn Stone, Sean Kelly, Amanda Godley, Sidney K. D'Mello
Published at CHI, 2020

Like anyone, teachers need feedback to improve. Due to the high cost of human classroom observation, teachers receive infrequent feedback which is often more focused on evaluating performance than on improving practice. To address this critical barrier to teacher learning, we aim to provide teachers with detailed and actionable automated feedback. Towards this end, we developed an approach that enables teachers to easily record high-quality audio from their classes. Using this approach, teachers recorded 142 classroom sessions, of which 127 (89%) were usable. Next, we used speech recognition and machine learning to develop teacher-generalizable computer-scored estimates of key dimensions of teacher discourse. We found that automated models were moderately accurate when compared to human coders and that speech recognition errors did not influence performance. We conclude that authentic teacher discourse can be recorded and analyzed for automatic feedback. Our next step is to incorporate the automatic models into an interactive visualization tool that will provide teachers with objective feedback on the quality of their discourse.
Towards an Understanding of Augmented Reality Extensions for Existing 3D Data Analysis Tools
Authors: Xiyao Wang, Lonni Besançon, David Rousseau, Mickaël Sereno, Mehdi Ammi, Tobias Isenberg 0001
Published at CHI, 2020

We present an observational study with domain experts to understand how augmented reality (AR) extensions to traditional PC-based data analysis tools can help particle physicists to explore and understand 3D data. Our goal is to allow researchers to integrate stereoscopic AR-based visual representations and interaction techniques into their tools, and thus ultimately to increase the adoption of modern immersive analytics techniques in existing data analysis workflows. We use Microsoft's HoloLens as a lightweight and easily maintainable AR headset and replicate existing visualization and interaction capabilities on both the PC and the AR view. We treat the AR headset as a second yet stereoscopic screen, allowing researchers to study their data in a connected multi-view manner. Our results indicate that our collaborating physicists appreciate a hybrid data exploration setup with an interactive AR extension to improve their understanding of particle collision events.
Truncating the Y-Axis: Threat or Menace?
Authors: Michael Correll, Enrico Bertini, Steven Franconeri
Published at CHI, 2020

Bar charts with y-axes that don't begin at zero can visually exaggerate effect sizes. However, advice for whether or not to truncate the y-axis can be equivocal for other visualization types. In this paper we present examples of visualizations where this y-axis truncation can be beneficial as well as harmful, depending on the communicative and analytic intent. We also present the results of a series of crowd-sourced experiments in which we examine how y-axis truncation impacts subjective effect size across visualization types, and we explore alternative designs that more directly alert viewers to this truncation. We find that the subjective impact of axis truncation is persistent across visualizations designs, even for designs with explicit visual cues that indicate truncation has taken place. We suggest that designers consider the scale of the meaningful effect sizes and variation they intend to communicate, regardless of the visual encoding.
Understanding and Visualizing Data Iteration in Machine Learning
Authors: Fred Hohman, Kanit Wongsuphasawat, Mary Beth Kery, Kayur Patel
Published at CHI, 2020

Successful machine learning (ML) applications require iterations on both modeling and the underlying data. While prior visualization tools for ML primarily focus on modeling, our interviews with 23 ML practitioners reveal that they improve model performance frequently by iterating on their data (e.g., collecting new data, adding labels) rather than their models. We also identify common types of data iterations and associated analysis tasks and challenges. To help attribute data iterations to model performance, we design a collection of interactive visualizations and integrate them into a prototype, Chameleon, that lets users compare data features, training/testing splits, and performance across data versions. We present two case studies where developers apply \system to their own evolving datasets on production ML projects. Our interface helps them verify data collection efforts, find failure cases stretching across data versions, capture data processing changes that impacted performance, and identify opportunities for future data iterations.
Unwind: Interactive Fish Straightening
Authors: Francis Williams, Alexander Bock 0002, Harish Doraiswamy, Cassandra M. Donatelli, Kayla Hall, Adam Summers, Daniele Panozzo, Cláudio T. Silva
Published at CHI, 2020

The ScanAllFish project is a large-scale effort to scan all the world's 33,100 known species of fishes. It has already generated thousands of volumetric CT scans of fish species which are available on open access platforms such as the Open Science Framework. To achieve a scanning rate required for a project of this magnitude, many specimens are grouped together into a single tube and scanned all at once. The resulting data contain many fish which are often bent and twisted to fit into the scanner. Our system, Unwind, is a novel interactive visualization and processing tool which extracts, unbends, and untwists volumetric images of fish with minimal user interaction. Our approach enables scientists to interactively unwarp these volumes to remove the undesired torque and bending using a piecewise-linear skeleton extracted by averaging isosurfaces of a harmonic function connecting the head and tail of each fish. The result is a volumetric dataset of a individual, straight fish in a canonical pose defined by the marine biologist expert user. We have developed Unwind in collaboration with a team of marine biologists: Our system has been deployed in their labs, and is presently being used for dataset construction, biomechanical analysis, and the generation of figures for scientific publication.
Watch+Strap: Extending Smartwatches with Interactive StrapDisplays
Authors: Konstantin Klamka, Tom Horak, Raimund Dachselt
Published at CHI, 2020

While smartwatches are widely adopted these days, their input and output space remains fairly limited by their screen size. We present StrapDisplays-interactive watchbands with embedded display and touch technologies-that enhance commodity watches and extend their input and output capabilities. After introducing the physical design space of these StrapDisplays, we explore how to combine a smartwatch and straps in a synergistic Watch+Strap system. Specifically, we propose multiple interface concepts that consider promising content distributions, interaction techniques, usage types, and display roles. For example, the straps can enrich watch apps, display visualizations, provide glanceable feedback, or help avoiding occlusion issues. Further, we provide a modular research platform incorporating three StrapDisplay prototypes and a flexible web-based software architecture, demonstrating the feasibility of our approach. Early brainstorming sessions with 15 participants informed our design process, while later interviews with six experts supported our concepts and provided valuable feedback for future developments.
What's Wrong with Computational Notebooks? Pain Points, Needs, and Design Opportunities
Authors: Souti Chattopadhyay, Ishita Prasad, Austin Z. Henley, Anita Sarma, Titus Barik
Published at CHI, 2020

Computational notebooks - such as Azure, Databricks, and Jupyter - are a popular, interactive paradigm for data scientists to author code, analyze data, and interleave visualizations, all within a single document. Nevertheless, as data scientists incorporate more of their activities into notebooks, they encounter unexpected difficulties, or pain points, that impact their productivity and disrupt their workflow. Through a systematic, mixed-methods study using semi-structured interviews (n=20) and survey (n=156) with data scientists, we catalog nine pain points when working with notebooks. Our findings suggest that data scientists face numerous pain points throughout the entire workflow - from setting up notebooks to deploying to production - across many notebook environments. Our data scientists report essential notebook requirements, such as supporting data exploration and visualization. The results of our study inform and inspire the design of computational notebooks.
Would you do it?: Enacting Moral Dilemmas in Virtual Reality for Understanding Ethical Decision-Making
Authors: Evangelos Niforatos, Adam Palma, Roman Gluszny, Athanasios Vourvopoulos, Fotis Liarokapis
Published at CHI, 2020

A moral dilemma is a decision-making paradox without unambiguously acceptable or preferable options. This paper investigates if and how the virtual enactment of two renowned moral dilemmas---the Trolley and the Mad Bomber---influence decision-making when compared with mentally visualizing such situations. We conducted two user studies with two gender-balanced samples of 60 participants in total that compared between paper-based and virtual-reality (VR) conditions, while simulating 5 distinct scenarios for the Trolley dilemma, and 4 storyline scenarios for the Mad Bomber's dilemma. Our findings suggest that the VR enactment of moral dilemmas further fosters utilitarian decision-making, while it amplifies biases such as sparing juveniles and seeking retribution. Ultimately, we theorize that the VR enactment of renowned moral dilemmas can yield ecologically-valid data for training future Artificial Intelligence (AI) systems on ethical decision-making, and we elicit early design principles for the training of such systems.