How to Manipulate a Graph without a Checkbox or Slider?
By: Yanzhi Shen
Tags: Data Visualization, Interaction Design, Interactive, Graphic Design

Tired of using external widgets to visualize data?

BI tools, such as dashboards, are designed to integrate user interaction with data visualizations. In this way, the users code their own designed graphs with a small change on the dashboard menu, such as dragging through a time range slider to select a timestamp or a single click on a checkbox. It has become generally a trend to design visualizations with a user interaction that facilitates a greater user experience.

Nowadays, there arouses a growing interest for researchers and designers to develop visualization tools, which enables users to operate graphing with direct manipulation on graphs, without the need for external widgets on an operational panel.

Direct manipulation on graphs is amazing

The visualization community introduced the direct manipulation of the graphical encoding concept for drawing the attention of need to design more natural and fluid interactions for data visualization tools. Direct manipulation provides immediate feedback when users making interactions on these visual objects. With direct manipulation interfaces, users can focus on the visual features while playing with it, without distracting themselves by shifting attention. Besides, the interfaces could be simplified without external widgets.

Instrumental interaction theory also supports the direct manipulation by illustrating the advantage of minimizing the spatial indirection, which is the distance between the interaction source and target objects, and the temporal indirection, which is the delay between operation invoking and results observing.

Further, visualization researchers define interactions that users directly manipulate on visual marks, as embedded interactions. We can understand those visual marks as common visual elements we identify in a graph and embedded interactions as the behaviours users conduct to realize their graph manipulating goals. For example, bars are marks for one bar chart, and we could make embedded interactions on this bar chart through changing the position, height and color of bars to customize our own graph.

Fig.1. embedded interactions designed for bar charts

More specifically, if we want to sort a bar chart by values, we could directly drag the tallest bar to the extreme left or right; if we want to merge two bars, we could drag and drop one bar on top of another bar.

To be noticed, possible design for embedded interactions varies through different visualizations. The encoding for a bar chart could be position, height and color, while a 2D scatterplot could be position, size and color and a histogram could be width only. However, researchers make “position, size, color, height and width” the five main graphical encoding types for generalized visualizations as a base for further study.

Fig. 2. 15 basic operations for direct manipulation of graphical encodings in a 2D scatterplot, bar chart and histogram

Sounds pretty cool, isn’t it? In this article, we will further discuss more examples of such interesting direct-manipulate interactions, the users’ intended mental strategies behind those behaviors and possible setback and limitation of such embedded designs.

Match users operations and their mental strategies

Fig. 3. Strategy matrix for 15 operations intented of studys participants

Varying degrees of strategy agreement

As we could see from the strategy and operation matrix, if we go through the matrix table by row, in some rows we could find almost black shaded cells, such as row 12. For the Q12 intended operation, among all the study participants, 10 out of 12 strategies are focused on the strategy 23th, recoloring by values to assign a data attribute to the color of all bars.

Fig. 4. Row 12 for the strategy matrix (Q12)

The higher value in a cell, the darker the background of the cell. Throughout the matrix, we are able to find the darker cells. If we divide the darkest cell value of each row by the sum of the cell values in that row, we could identify the varying degrees of strategy agreement. We could also see that one strategy represents more than half of the distribution of all used strategies for two thirds (10/15) of the operations.

Given this fact, designers are researching to build a set of matching operations and consensual strategies. This could help designers incorporate commonly used or intended strategies for operating with direct manipulation of graphical encodings.

Conflicting strategies

In some strategy matrix rows, we could find more shaded color cells, such as row 2, the darkest cell values 5 out of the total 16 operations. With the low operation centralization rate within the Q2 strategy, the fig.5. shows that the study participant made various operations to switch a scatterplot to a bar chart.

Fig. 5. Row 2 for the strategy matrix (Q2)

The many to many relationships between strategies and operations in the matrix raise technical challenges in designing the direct manipulation regarding the graphical encoding. Further researches, such as collecting data on human’s preferences to develop contextual strategies, should be studied to improve the design of direct user interactions.

Conclusion: Organizing 48 strategies

Fig. 6. 48 low-level strategies participants tried in the case study

Four high-level categorized of strategies

Researchers identified four high-level encoding strategies from the 48 listed strategies, exemplification, declaration, instrumentation, and selection, to invoke and operation through various studies.

Exemplification: Participants are likely to show a few visual manipulations as an example of input and output for a series of repeatable operations. Through a repetitive set of actions, participants would be able to show the system how a part of the visual output should be like, indicating a part of the expected visual output to the system.

Participants are likely to show a few visual manipulations as an example of input and output for a series of repeatable operations. Through a repetitive set of actions, participants would be able to show the system how a part of the visual output should be like, indicating a part of the expected visual output to the system. Declaration: Apart from the direct matching strategies applied to operations, participants also tend to manipulate a kind of graphical encoding, such as applying new color schemas, in order to inform the system users that a new change has been made. These approaches are summarized and named to be the declaration strategy.

Apart from the direct matching strategies applied to operations, participants also tend to manipulate a kind of graphical encoding, such as applying new color schemas, in order to inform the system users that a new change has been made. These approaches are summarized and named to be the declaration strategy. Instrumentation: Participants tend to use a visual mark as an instrument or tool to manipulate the visualizations. For example, people would like to first apply updates to a certain visual mark and then use this visual mark as the instrumentation to update other data points like dragging the instrumentation mark to the data points we want to change in the visualization.

Participants tend to use a visual mark as an instrument or tool to manipulate the visualizations. For example, people would like to first apply updates to a certain visual mark and then use this visual mark as the instrumentation to update other data points like dragging the instrumentation mark to the data points we want to change in the visualization. Selection: Participants prefer to have access to selecting several data points, and the updated operation should apply to all selected data points if one of the selected data points changes and updates.

References