What Is The Boundary Between Data Visualization and Other Types of Images?
By: Jason Forrest
Tags: Data Visualization, Dvhistory, Data, Design, History

(I’d like to say that no butterflies were harmed in the making of these posts… but… )

I have a question, James Lytle and Stephanie Tuerk… why do you feel the data needs to be quantitative? Can qualitative data not be visualised… or does it only constitute data visualisation if the underlying data is quantitative

Pierre Dragicevic:

Stephanie Tuerk, I’ve been having the exact same questions as you. Jacques Bertin’s monosemic/polysemic distinction can help understand conceptual differences between data visualizations and other types of information-carrying images: “A system is monosemic when the meaning of each sign is known prior to observation of the collection of signs” […] “a system is polysemic when the meaning of the individual sign follows and is deduced from consideration of the collection of signs” (Semiology of Graphics). Others (e.g., Leland Wilkinson, Yuri Engelhardt) talk about visualizations as having a grammar. Robert Kosara also has a very interesting discussion on visualizations vs. other types of images:

In his blog post, there’s an interesting comment from Hadley Wickham that visualizations should be “invertible”. This is easier to understand in the context of automatically-generated computer visualizations, which have been often described as multi-stage processes that turn raw data into images (the “visualization pipeline”). The idea is that a reader should be able to start from the image and get back to the data somehow. None of these analyses goes very deep into what makes a good vs. bad visualization, but I think it’s interesting to try to reflect on what an artifact is (a philosophical question) irrespective of whether it is effective at achieving its designers’ intent (an empirical question).

lord

Insightful, Pierre Dragicevic… there’s something else too (I don’t know how this is expressed in theory) where there is emergent understanding from the collected signs (a gestalt) that is qualitatively different from the various data.

Pierre Dragicevic:

Right. There is a perceptual side to that (e.g., ensemble perception) and also a cognitive side: researchers talk about going from data to knowledge, insights, sometimes wisdom. Those are key aspects of visualization but it’s much more difficult for me to think in that space.

Hilma af Klint, Group IX/SUW, The Swan, №9, 1915. Oil on canvas, 149.5 x 149 cm. Photo: Albin Dahlström, The Moderna Museet, Stockholm.

Jason Forrest:

Pierre Dragicevic — this “invertible” idea is VERY interesting and one that I think is especially important when the subject matter is more intangible. I started thinking about this a lot when I wrote about Hilma af Klint as she was using a Theosophical semiology, which itself was part scientific and part poetic, so her visualizations were designed to be studied and meditated on as (invertible?) spiritual guidance.

lord, the mix of qualitative and quantitative I think is one of the defining issues at the moment, and I think it’s something that separates much of the more analytical work from being more persuasive. IMO, it’s that editorial/qualitative nudge that people seem to connect to.

Stephanie Tuerk:

Jason Forrest, I would argue that the “invertible” requirement regulates not only the how information is symbolized but what *kind* of information can be encoded in a visualization. Meaning that “information” that is too complex for its comprehension/perception to be easily and unambiguously confirmed can’t be represented in a visualization, i.e. images that attempt to embody that kind of “information” (which I would call something like a “concept” or “idea”) are some other type of image aside from a visualization. That is to say, representation is a larger category than visualization.

Jason Forrest:

I’m not sure I agree, as an illustration of “heaven” is still a translation of an idea. But maybe I’m pushing too far at an extreme with that example. Would you consider an early diagram of an atom a visualization even if it turns out to be wildly inaccurate later on?

Stephanie Tuerk:

No, I would consider any diagram of an atom to be a diagram, not a visualization! Contemporary or historical, irrespective of the image’s relationship to an actual atom.

Also, to me the status of something as a visualization has nothing to do with the veracity of the information that is visualized (responding to your comment about “even if the image is inaccurate later on”) but more so on the fact that the information shown begins as something that the author has in a form that is completely distinct from any graphical representation…and then is translated into graphic form….and then can be retrieved by the viewer and reconstructed in the distinct from graphical form.

The evolution of atomic models in the 20th century: Thomson, Rutherford, Bohr, Heisenberg/Schrödinger

lord:

I agree with Stephanie Tuerk about the Bohr Atom… while data drove his theory, he wasn’t visually presenting any underlying data — he was representing a theory of atomic structure… it was a theory visualisation…

Jason Forrest:

maybe it’s graphic translation?

James Lytle:

lord, Good question. So, in general, I tend to view DV as the language of “How much?” or the grammar of scale, which grounds the heart of vis in how much of this or that (how much time, how much space, how much who/what). Naturally, in order to properly describe who or what you are talking about it is helpful to describe the profile of things (with numbers and characteristics) but then it is back to how those things relate to each other (ie how does this system work or interrelate, how are the parts of the petal structured in relation to each other?) So system diagrams are still very much vis to me in as much as it communicates how different pieces relate to one another.

Arran LR:

I liked Boris Müller’s piece on considering DV as a cultural image interesting:

I think the question I’m interested in is considering DV as communication and as a tool for insight.

Stephanie Tuerk:

I totally agree with the argument that data visualizations are cultural images, but I think that there is a lot of literature in the history of science for example that argues that the kinds of images that Müller is setting up as strawmen/”technical” images are also cultural images. I mean, it’s the same argument that you see a lot these days (or at least I see a lot) arguing that the data themselves are cultural products and not “objective.” Essentially…cultural/technical is a false binary.

The question of “why quantitative” is a good one. (Am I allowed to say, “oh lord, that is a good question!“?) I guess first I want to retract a bit and say that I think that ordinal information can be visualized as well, but I think that once you get to the idea of turning an idea into an image, you are in linguistic territory (and that of classic semiotics) — in that you are making an image the idea of cat the same way that the word “cat” is the signifier of the signified “cat.” I guess that is to say, I think that a characteristic of visualization is that it, in and of itself, doesn’t create meaning, although it communicates information. (People may interpret meaning from it, but that meaning is not in the visualization itself.)

I mean, there is no one definition and sure some people are going to want to say that all diagrams are visualizations. But then we ignore that there is a smaller class of things within diagrams that make meaning in a very specific way — in which graphical elements are used to represent discrete relationships whose success in faithfully transmitting information relies on the lack of interpretability of the original information— and fail to recognize that that is something distinct because we have expanded the term visualization to include the larger class of things.

Anyway, I went to the wrong source (Tufte). Pierre already mentioned this, but (as almost always), Bertin provides sound counsel on the matter.

lord:

I loved the way you described the importance of relationships James Lytle. It’s these relationships that allow a discerning person to pick the signal from the noise…. Like, what aspects of a butterfly are functional (wing dimensions) and what are too variable (have too much noise) to be analytically interpreted (eg number of spots).

So I have another question… If the diagram of an atom is not a data visualisation… What about something that represents an equation? There’s no actual data… But the equation encodes data. Does that count? Example; a graph that represents the equation of an aerofoil or an animated metronome that is defined by a sinusoidal equation.

Pierre Dragicevic:

I like the equation case, that’s a tough one. You can also imagine you have a simulated dataset and visualize it with a set of complex plots. Most people would probably say they are visualizations, and yet that’s just a more complex version of your equation case. You could argue the parameters of your models are the data. In the case of a sinusoidal equation, your data would then be a set of three quantities (period, phase and amplitude). An interesting implication is that a sinusoidal curve can be seen as a technique for visualizing three quantities. That’s one possible response to your question, there are probably many others.

About butterflies: many would probably agree that naturally-occurring phenomena like footsteps on a beach are not visualizations. Yuri Engelhardt discusses this in his thesis. To him, a visualization needs to be an artifact (i.e., something created by a human). So if you come across a bunch of dead butterflies during a walk in the woods, you could infer a lot of things using many of the same perceptual and cognitive skills as the ones you use when you read a visualization, but you couldn’t really call this a visualization. In your photos, however, the dead butterflies were purposefully arranged in a way that makes it presumably easier to extract information. That, I think, comes closer to a visualization.

Stephanie Tuerk:

…but does visualization not imply the human act of translation of information from a non-visual medium into a visual medium?

Jason Forrest:

See, that’s where I am still/too. In my world, the data aspect is important, but not all-encompassing.

Stephanie Tuerk:

I mean, I would never argue that a display of butterflies does not convey information. It very much does! So to me, it returns to a different unanswerable question, which is, what are we hoping to get out of defining the term visualization? For me, I’m trying to find a definition that distinguishes (data) visualization from other forms of representation — like, a visualization may often be a diagram, but I’m interested in what makes not all diagrams visualizations. Obviously one could also want to define it as broadly as possible for different reasons, and then I guess the question is, when does something stop being a visualization? I’m not sure if these are two sides of the same coin or not…

lord:

I agree, Stephanie Tuerk that there needs to be a human act of presenting the data in a form that facilitates understanding that could not be gained otherwise… But I’m on the fence about the form not being visual to start with.

If you take a bar chart and plot a single dot for each observation instead of an incremental rectangle, you are visualising a number. Each dot has been arranged informatively… The overall visualisation gives you information you can glance at and reveals more about the data than you’d get from just looking at a bunch of numbers.

How is this different from Nabokov’s scientific arrangement of butterflies? If I had been counting butterflies and placed them one on top of each other, like I did with these chocolates… Is that no longer a visualisation?

Stephanie Tuerk:

lord as I see it, if you are counting the butterflies, and then using the butterflies that you counted to make the visualization by stacking them one on top of the other, the information you are visualizing is a number, not the butterfly itself. (This using the things you count to make the histogram is a particularly cheeky kind of visualization — it’s probably an index in Peirce’s terms) versus in scientific displays, what is on display is the butterfly itself.

lord:

Not really… I picked them up and arranged them in order of type, one on top of each other. Nabokov’s arrangement reveals even more about the data because he analysed more details about the butterflies and then arranged them in a structured way that reveals more about them to the discerning eye.

Stephanie Tuerk:

By the same token, if one took all of the Impressionist paintings in MoMA’s collection, and put them in a room, and ordered them by artist and decade, and put them on the wall in these groupings so that you can see the visual differences between them, has one made a visualization? I guess for me visualization involves a signified and a separate signifier (the graphic element that represents the signified), which come together to make a sign.

Jason Forrest:

You mean like this:

Stephanie Tuerk:

When is something the object itself and when is it a representation of the object? If you put images of artworks in a timeline, to me they come way closer to actually being “data”. They aren’t the paintings themselves. If you took a room at MoMA and painted a timeline on the wall and hung paintings on it though…1. omg would the art world be pissed, and 2. I feel like that is really some kind of precipice of something! To me that is the equivalent of the histogram of candy bars. (I appreciate the provocation btw!)

Jason Forrest:

I honestly think that most museums are exactly arranged by timeline. Some actually have a line and year, so I don’t see that aspect as that unusual.

Stephanie Tuerk:

Yeah, I mean, after all, these “artworks” here TRULY are data in that what we see is literally a reconstruction of hex code somewhere. Oh god, now I just made the whole internet a data visualization.