# Illustrative Molecular Visualization with Continuous Abstraction

Matthew van Der Zwan, Wouter Lueks, Henk Bekker, Tobias Isenberg

<figure><img src="image_1.png" /></figure>

## To cite this version:

Matthew van Der Zwan, Wouter Lueks, Henk Bekker, Tobias Isenberg. Illustrative Molecular Vi-sualization with Continuous Abstraction. Computer Graphics Forum, 2011, 30 (3), pp.683-690. 10.1111/j.1467-8659.2011.01917.x. hal-00781508

HAL Id: hal-00781508

https://inria.hal.science/hal-00781508

Submitted on 27 Jan 2013

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scien-tific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Public Domain
# Illustrative Molecular Visualization with Continuous Abstraction

Matthew van der Zwan,<fnref n="1" /> Wouter Lueks,<fnref n="1" /> Henk Bekker,<fnref n="1" /> and Tobias Isenberg<fnref n="1" />,<fnref n="2" />

## Abstract

Molecular systems may be visualized with various degrees of structural abstraction, support of spatial perception, and 'illustrativeness.' In this work we propose and realize methods to create seamless transformations that allow us to affect and change each of these three parameters individually. The resulting transitions give viewers a dedicated control of abstraction in illustrative molecular visualization and, consequently, allow them to seamlessly explore the resulting abstraction space for obtaining a fundamental understanding of molecular systems. We show example visualizations created with our approach and report informal feedback on our technique from domain experts.

Categories and Subject Descriptors (according to ACM CCS): I.3.m [Computer Graphics]; Miscellaneous—Scientific visualization; molecular visualization; illustrative visualization; dedicated seamless abstraction.

## 1. Introduction

Molecular visualization is of tremendous importance for understanding processes that are relevant in fields such as material sciences, genetics, pharmacy, immunology, and biology and chemistry in general. Researchers in these domains are trying to cope with an ever increasing complexity of molecular data while trying to gain insight in the structure and function of large molecules such as proteins. In our work we explore the application of illustrative visualization techniques to this field of molecular visualization which is particularly suitable for illustrative visualization approaches because no 'photorealism' exists at the sizes of atoms [Goo03].

Research into the structural properties of small and large molecules is increasingly gaining importance. The function and interaction of molecules is often primarily analyzed based on an understanding of this structural information. Researchers have a number of tools at their disposal to visualize the collected molecular data [OGF*10], for example PyMol, VMD and Chimera. Visualizations are created by blending (parts of) the molecule shown in different structural representations. However, the connection between these representations is often difficult to explore by switching/blending alone. It is this problem that we address with our work.

Based on the structural data that is available in large molecular databases [BWF*00, BBB*02] and by taking inspiration from traditional hand-made illustration styles

<figure><img src="image_2.png" /><figcaption>**Figure 1: Protein with selective structural abstraction, medium 'illustrativeness,' and support of spatial perception.**</figcaption></figure>

[Hod03], we describe how to visualize continuous transitions between different stages of structural abstraction as well as aspects of spatial perception and 'illustrativeness' (e. g., Fig. 1). All of these parameters can be controlled independently and continuously in real-time to enable users to interactively explore the structure of complex molecules.

<footnote n="1">¹Johann Bernoulli Institute of Mathematics and Computer Science, University of Groningen, The Netherlands</footnote>

<footnote n="2">²DIGITEO in collaboration with VENISE-LIMSI-CNRS and AVIZ-INRIA, Saclay, France</footnote>
In particular, we extend a previous GPU-based approach for molecular visualization [TCM06] such that we can transition from solid geometry to a planar one and provide means to move to a smooth, spline-based representation. In addition, we support a seamless interpolation between 'photorealistic,' cel shading, and stylized black-and-white depiction. Finally, we integrate techniques such as object attenuation and halos for all these abstraction stages.

In summary, our main contribution is the *continuous abstraction space for illustrative molecular visualization*. This abstraction space facilitates an *interactive and seamless exploration* of structural information as well as depiction styles for molecular data in which each of the aspects can be controlled individually. We discuss the realization of the transitions and their implementation using GPU techniques and report on informal feedback from domain experts.

In the remainder of this paper we first review work related to our own in Section 2. Next, we discuss abstraction techniques for molecular visualization and identify aspects that we combine to form a continuous abstraction space in Section 3. In Section 4 we then describe how we realize our molecular visualization using this abstraction space. Afterward, we present results of our technique and report informal feedback on our technique in Section 5. Finally, Section 6 concludes the paper and discusses aspects of future work.

## 2. Related Work

O'Donoghue et al. [OGF*10] give a quite comprehensive overview of the different techniques that are being practically applied to the domain of molecular visualization (for an overview of visualization techniques also see [LLC*11, Section 4.3]), as well as of the visualization tools available for researchers. They highlight non-photorealistic (or illustrative) visualization techniques as very effective methods to depict the overall shape and form of molecules, in particular for presentation to others and for publication. While O'Donoghue et al. mention and show an image of using “flat colors and outlines” as an example for such non-photorealistic depiction, others have introduced more advanced methods of illustrative visualization in this field which we describe next.

In an early example, Goodsell and Olson [GO92] describe a number of simple techniques to visualize the molecular surface using the parallel hatching and silhouettes techniques that were the state of the art in the early 1990s and also discuss first approaches to some structural abstraction using cylinders and cut-aways. A similar tool for grayscale shaded illustrations of detailed and abstracted proteins was described by Kraulis [Kra91]. In more recent approaches, Lampe et al. [LVRH07] use a two-level approach on the GPU to illustrate slow dynamics of molecules, while Weber [Web09] explores the use of texture-based approaches using shader programming to generate pen-and-ink renderings of molecules in real-time. Weber focuses, in particular,

on producing cartoon-style illustrations for publication and on permitting users to apply different types of abstraction to different parts of the molecule. While we also rely on shader programming for fast rendering, we concentrate, in particular, on enabling the seamless transition between different rendering and abstraction styles, in addition to being able to apply different styles to different parts of the model.

One particularly relevant illustrative visualization of large molecules was presented by Tarini et al. [TCM06]. They use *ambient occlusion* as an approximation of global illumination as well as additional techniques such as halos to improve the perception of the spatial structure of large balls-and-sticks and space-fill models. We also employ Tarini et al.'s imposter-based rendering with ambient occlusion and halos but focus on how to transition along the axes of structural abstraction and visual styles in an integrated fashion.

An approach that is different from using space-fill, balls-and-sticks, and related techniques that show the *inner structure* of a molecule is to visualize its *outer surface* [CG07, CPG09, KBE09, CWG*10] which is important to understand the interactions between different molecules. For example, Cipriano et al. [CG07] not only examine the illustrative depiction of the molecular surface but also explore abstraction as well as the placement of decals to represent features that have been removed through the simplification. While we do not employ surface-based visualization techniques it would be possible and interesting to combine them with the visualizations of internal structure that we explore.

Inspiration for our illustrative visualizations also comes from traditional illustration techniques [Hod03]. For example, Goodsell [Goo03] summarizes the state of the art of molecular illustration. He emphasizes the existence of traditional schemes including the space-filling diagram and the balls-and-sticks model as well as of structural abstractions such as the ribbon diagram [Ric85], all of which we also support with our technique. Interesting for our work are, in particular, Goodsell's black-and-white examples. Here he uses traditional black-and-white shading techniques such as hatching and stippling to portray the atom types for print reproduction which are otherwise often rendered in specific color schemes. We use a similar approach but show how to seamlessly transition between the two extremes.

## 3. A Continuous Abstraction Space

In their survey of molecular visualization [OGF*10], O'Donoghue et al. note the importance of being able to get an overview of a molecule's structure and point out that both non-photorealistic/illustrative visualization techniques as well as structural abstraction such as the ribbon diagram very well support this goal. They also remark, however, that being able to "see where sequence features are located in the three-dimensional structure can be of substantial practical value." This means that being able to mentally integrate
<figure><img src="image_3.png" /><figcaption>**Figure 2:** *Structural abstraction stages: (a) space fill, (b) balls-and-sticks, (c) licorice, (d) backbone, and (e) ribbon.*</figcaption></figure>

both detailed views with abstracted and potentially stylistic depictions is essential, which to date is usually only possible by switching or α-blending between different visualizations.

Abstraction in the context of molecular visualization typically refers to *structural abstraction*. Here, various forms of depicting the structure of a molecule are commonly used (e. g., see [Goo05]): the space fill diagram (Fig. 2(a)) which depicts each atom using its van der Waals radius, the balls-and-sticks model (Fig. 2(b)) which uses smaller radii and adds bonds as cylinders, and the ribbons model [Ric85] (Fig. 2(e)) which further abstracts parts of the molecule to ribbon helices and sheets so that secondary, tertiary, and quaternary structures become apparent. A stage between the balls-and-sticks and ribbon models is the licorice visualization (Fig. 2(c)) that only shows the bonds, colored according to the typical colors associated to atom types. The removal of less important parts of this structure leads to abstractions that emphasize the molecule's backbone [NCS88] (Fig. 2(d)). These structural abstraction stages form a natural progression (as in Fig. 2) which we can place along an *axis of structural abstraction*, and for which we later define transitions in order to support the mental integration.

Besides the mentioned stages there exists a variety of additional structural abstractions. In addition to the previously mentioned surface visualizations [CG07, CPG09, KBE09], the inner structure of molecules can be abstracted with cartoon views where β-sheets are depicted as arrows [DB04] or simplified 2D schematics (see, e. g., [OGF*10, Fig. 4(g, h)]). Other abstraction types are coarse-grained abstractions (e. g., [MRY*07]) that combine several atoms into larger pseudoequivalents to facilitate the simulation of very large systems. We currently do not support these abstractions because most represent a fundamental paradigm shift, while our work focuses on the aspect of dedicated control of abstraction. However, one can envision potential extensions such as transitions from the space fill diagram to the surface visualization or from the ribbon visualization to coarse-grained models, potentially leading to more than one possible abstraction path.

Aside from structural abstraction, stylistic rendering is also frequently applied in molecular illustration and visualization (e. g., [Ric85, Goo03, Goo05, Web09, OGF*10]). This type of depiction can also be considered to be a type of abstraction. By reducing the detailed shading, for example, il-

lustrative depictions of molecules can abstract from otherwise overwhelming detail and instead highlight the overall shape [Goo05, OGF*10]. Illustrative depiction can also support the use of structural abstraction by emphasizing through stylization the fact that abstraction has been applied [CG07]. Finally, we consider the use of visualization techniques that more or less support the perception of the spatial shape (ambient occlusion and halos [TCM06] or halos and line attenuation [EBRI09]) to be a third axis of abstraction that can be applied in molecular visualization. These techniques selectively reduce detail and emphasis in certain places such as the inside of a molecule or visually distant parts and, therefore, introduce localized abstraction [LKZD08].

Researchers work with visualizations in diverse combination of the mentioned abstractions, both structurally and visually [OGF*10]. Higher levels of abstraction, e. g., are well suited to provide an overview, but at the same time these lack detail [CG07] which may be required for other tasks. Moreover, it is not only desirable to be able to combine views of different levels of scale or abstraction [Goo05], but in fact to be able to *seamlessly transition between abstraction levels*. This seamless transition is what we are addressing in this work. For this purpose we define an *abstraction space* whose main axis is that of *structural abstraction*; this axis is augmented by changes to the visual style of the visualization: 'illustrativeness' and *support of perception of spatiality*. We need each of these independent axes to be continuous (i. e., not to contain visible 'jumps') to support the desired seamless navigation and exploration. This continuity implies that we need to define an order between the previously discrete stages within each axis and meaningful transitions between adjacent ones. This order is given for structural abstraction as depicted in Fig. 2, and understandable stages in-between these discrete stages can easily be envisioned.

More formally, the abstraction space is a space $\mathcal{F}$ of functions. Every function $f(t_s, t_p, t_i) \in \mathcal{F}$ with $t_s, t_p, t_i \in [0, 1]$ consists of a function triple $(f_s(t_s), f_p(t_p), f_i(t_i))$, where $f_s$ determines the degree of structural abstraction, $f_p$ the support of spatial perception, and $f_i$ the 'illustrativeness.' Each of these functions $f_k$ has several discrete levels $t_{k,n}$, evenly spaced in $[0, 1]$, associated to it that mark known styles. Each $f_k$ needs to specify how to seamlessly transition between $f_k(t_{k,n})$ and $f_k(t_{k,n+1})$. A mapping $A$ assigns to each amino-
acid *a* in the protein a tuple (*t*<sub>s</sub>(*a*), *t*<sub>i</sub>(*a*)), thus determining its style. The parameter *t*<sub>p</sub> can only be controlled globally.

**4. Interactive Visualization with Seamless Abstraction**

To achieve our goal of seamless structural and visual abstrac-
tion we first describe how to achieve the continuous transi-
tion of *f*<sub>s</sub> between the structural abstraction stages *t*<sub>s,n</sub> before
outlining the realization of the visual abstractions *f*<sub>p</sub> and *f*<sub>i</sub>.

**4.1. From Space Filling to Ribbon Visualization**

The seamless structural transition from space-fill ($t_{s,0}$) to-
wards balls-and-sticks ($t_{s,1}$) is straightforward, both concep-
tually and in its implementation. Spheres represent atoms
with their van der Waals radii in the space fill model, while
smaller spheres (whose sizes do not have a physical mean-
ing) with additional cylinders to represent the bonds are used
for balls-and-sticks. Thus, we can gradually abstract from
the space fill to the balls-and-sticks representation by reduc-
ing the spheres' sizes, hence revealing the bond-representing
cylinders that were previously hidden by the larger spheres.
A further shrinking of the spheres until they vanish results
in the licorice representation ($t_{s,2}$). We confirmed for the re-
sulting animation with our collaborating domain experts that
they create nothing but the expected visualizations.

The next discrete stage in the common structural abstrac-
tion sequence shows only the protein skeleton (t<sub>s,3</sub>). The core
of an amino acid consists of a sequence of a nitrogen atom
followed by two carbon atoms. The protein skeleton is the
sequence of these core atoms. Therefore, to transition from
the licorice representation to the protein skeleton we need
to remove those bonds that are not part of the core atoms
in a structured and continuous fashion. For this purpose we
define the *rank* of an atom as the minimum distance to the
protein skeleton along the bonds. All atoms in the protein's
skeleton are assigned rank 0, those immediately adjacent
rank 1, etc., up to a maximum rank for the molecule. Since
a bond connects two atoms, each endpoint has a naturally
defined rank as well. Hence, we can also compute a rank for
every point on the cylinder by linear interpolation. This rank
allows us to specify the continuous transition from the entire
molecule to only the backbone using a rank threshold *t*: only
the part of every cylinder with *rank* ≤ *t* is shown. Again, we
sought feedback from our collaborating chemists about this
transition that continuously removes the bonds, with bonds
distant from the skeleton being removed first. Also in this
case the chemists confirmed that this animation does not in-
troduce unwanted artifacts of structural or other nature.

The final discrete structural abstraction level supported by
our approach is the ribbon diagram (t<sub>s,4</sub>) in which helices
represent abstract spiral structures (α-helices) in the protein
backbone. Thus, in order to achieve a continuous abstraction
from the backbone we need to smoothly transition between
a cylinder representation with sharp bends and smooth lines

and ribbons. This transition comprises two aspects: to shift
from the solid geometry of the cylinders in the backbone to
the planar geometry of the lines and ribbons and to linearly
interpolate between polylines along the cylinders in the back-
bone and the smoothed ribbon representation. An additional
challenge that relates to both aspects is that the planar ge-
ometry of the lines and of the helices in the ribbon represen-
tation differs: the line normals are view-aligned, while the
helix normals point toward the primary axis of the helix.

To realize both aspects of the transition we obtain infor-
mation about the molecule’s geometry and structure from
the PDB [BWF*00, BBB*02]. We calculate the position of
the smoothed ribbons by fitting cubic splines through the
atoms of the protein skeleton. We achieve a smooth and in-
teractive visualization by extending Tarini et al.’s [TCM06]
 الشارك-based impostor technique. These impostors are view-
dependent planes upon which the (shaded) shape of the de-
picted primitive is rendered (spheres for atoms and cylinders
for bonds)—corresponding to how a viewer would see the
real object. Depth-displacement is applied corresponding to
the primitive’s geometry to obtain a correct 3D visualization
where cylinders are depicted as halfed pieces of a tube with
open ends. Because these open ends are not acceptable in the
abstraction stages without large spheres we augment Tarini
et al.’s technique to include endcaps.

In an early stage during the transition from the backbone
to ribbon visualization we change the backbone into smooth
curves. We achieve this change by rendering the backbone
using very small generalized cylinder patches which initially
have a circular cross-section. During the transition from
solid to planar geometry we change this cross-section into
an ellipse whose major axis (representing the width of the
bond) remains constant in length and which is oriented par-
allel to the viewing plane. Thus, we realize the transition of
the impostors to view-aligned planes by shrinking the minor
axis to zero. Because this is implemented using impostors
we effectively only have to adjust the depth displacement
to give rise to the planar patches. Later during the abstrac-
tion we also transition the line segments that represent spiral
skeleton structures into helices. In addition to their spatial re-
location and in contrast to non-helix lines we also smoothly
interpolate the normals of the helix planar patches towards
normals perpendicular to the helix direction.

We implemented this last part of the structural abstraction
through a linear interpolation from the bond geometry to cu-
bic splines fitted though the atoms' locations, and our col-
laborating computational chemists noted that this behavior
matches their expectations of what such a transition would
look like. Moreover, we do not use the impostor technique
for helices [BD04], thus achieve a more correct ribbon repre-
sentation due to using correct locations. In addition, we still
maintain interactive rendering because the structural abstrac-
tion leads to an overall simplification of the geometry with
respect to the space fill or balls-and-sticks levels.
<figure><img src="image_4.png" /><figcaption>**Figure 3:** Stages of support of spatial perception: (b)–(c) ambient occlusion and object attenuation and (d)–(e) added halos.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Figure 4:** *The two extremes of support of spatial perception applied to a fully stylized visualization with full structural abstraction. Notice the halos and line/ribbon attenuation (b).*</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Figure 5:** *Abstraction along 'illustrativeness': (a) 'photorealistic,' (b) cel shading, and (c) black-and-white.*</figcaption></figure>

**4.2. Transitions of Visual Style**

In addition to the seamless control of structural abstraction we also provide two means of changing the visual style. The reasons for providing such means lie not only in their use in traditional molecular illustration [Ric85, Goo03, Goo05] and recent visualization approaches [TCM06, Web09, OGF*10] but is also motivated by the fact that visual abstraction supports effective communication of knowledge [RBGV08].

To enhance the spatial perception of molecules at various structural abstraction stages we employ ambient occlusion [TCM06], haloing [ARS79, BG07], object attenuation, and change of perspective projection. While some of these techniques can be used in a similar manner for all structural abstraction stages, ambient occlusion can only be applied to space fill and balls-and-sticks because both have an inherent dense geometry, but cannot be used in the less dense, higher levels of structural abstraction. For the computation of ambient occlusion (AO) we follow Tarini et al.'s [TCM06] approach. To deal with changing scenes, we precompute the

AO for spheres at $t_{s,0}$ and interpolate toward full illumination at $t_{s,1}$, while for cylinders we precompute halfway between $t_{s,0}$ and $t_{s,1}$ and interpolate first from no illumination then toward full illumination. In addition to AO we employ depth-dependent halos [TCM06, EBRI09], either black or white when applied to 3D objects (relating to shadow or fog, resp.) or just in white for line-based representations. Finally, with dolly zoom (Vertigo effect) and object attenuation (change of object size depending on distance to the viewer) we provide two alternatives for enhancing depth perception from perspective, the latter often being employed in illustrative visualization (e. g., [EBRI09]).

To arrange all these effects along a continuous one-dimensional axis we first apply AO simultaneously with dolly zoom/object attenuation and only afterward add halos (Fig. 3). The simultaneous use of AO and zoom/attenuation ensures that the control changes spatial perception at all levels of structural abstraction, even for levels where no AO is possible (e. g., Fig. 4). Moreover, the late halo application prevents the halos from obscuring attenuated objects.

A second means of changing the visual style of molecular visualizations is inspired by the varying styles and conventions in molecular illustration. One such established convention uses photorealistic shading (originating from physical plastic models) and assigns specific colors to the chemical elements and chains in the ribbon model. At the other end of the spectrum are purely black-and-white illustrations [Goo03], and in-between the two extremes stylistic shading techniques are used [Goo05]. Color conventions are also extended to structural abstractions by interpolating the atoms' colors along the connecting bonds or lines, while different colors depict the front- and back-sides of helices.

We provide a similar control along an 'illustrativeness' axis. As we move away from traditional (plastic) shading (Fig. 5(a)) we first transition to cel shading [Dec96, Leb96, LMHB00] and change the colors to their pastel equivalents (using palettes from ColorBrewer [HB03, Bre09]; Fig. 5(b)) to create a cartoon look. The darker rims that result from the cel shading, at this stage, begin to create what Goodsell calls "illustrative outlines in molecular structures" [Goo05]. We create the black-and-white end of the spectrum (Fig. 5(c)) using slice-based hatching [Lei94, DHR*99, SEI10] to mark the atom types. As in the hand-drawn examples [Goo03] we hatch the helix back sides (following Weber [Web09]; see Fig. 4) and add silhouettes [IFH*03] to spheres and ribbons.
<figure><img src="image_7.png" /><figcaption>**Figure 6:** A selection of results: in (a) a focus is placed on the secondary structure of a chain while showing the back part of the molecule as context, (b) shows the same molecule but now with three structural abstraction levels: balls-and-sticks, ribbon, and licorice; the visualizations in (c) and (d) demonstrate how illustrativeness can be used to guide attention.</figcaption></figure>

# 5. Results and Feedback

Below we discuss a selection of example visualizations, men-
tion rendering performance, and report informal feedback.

## 5.1. Visual Results and Discussion

Individual transitions along each of the axes of the abstrac-
tion space have been used to illustrate the concepts earlier
in the paper. For example, Fig. 2 shows the effect of chang-
ing the structural abstraction while Fig. 3 and 4 demonstrate
the support of depth perception and Fig. 5 depicts the effects
of ‘illustrativeness.’ The use of different degrees of illustra-
tiveness facilitates the selective emphasis or de-emphasis of
selected parts of the structure. The desired amount of empha-
sis can be selected by continuously transitioning from tradi-
tional colored shading via a cartoon shading to a black-and-
white depiction. Fig. 6(a) demonstrates this focus+context
effect in which the larger part of the molecule was de-
emphasized using an intermediate stage between the black-
and-white and grayscale levels, while using ambient occlu-
sion and halos support the spatial perception of the context
shape. The chosen intermediate level of illustrativeness to-
gether with the use of halos for the context also has the effect
that individual atoms merge visually so that the molecule’s
overall shape is more visible [Goo05]. The focus is, instead,
created using more structural abstraction (ribbon view) and

less illustrativeness to add color. An example with less em-
phasis on the abstracted chains in focus is shown in Fig. 1.

Using the same molecule and the same view as in Fig. 6(a), Fig. 6(b) shows the application of three different degrees of structural abstraction, combined with different levels of illustrativeness. The background uses licorice with b/w, while in the foreground one sub-chain is shown using the balls-and-sticks model which is contrasted with two sub-chains in ribbon style to both its sides. In this specific case this allows viewers to compare the representations with each other.

The visual emphasis resulting from the use of different de-
grees of illustrativeness is illustrated in Fig. 6(c). This effect
can easily be used to guide a viewer's attention, in Fig. 6(c)
to the part shown in color, also allowing people to compare
different parts of a molecule. The intermediate stage of illus-
trativeness between black-and-white and grayscale, in this
case, ensures that the atom type normally communicated us-
ing the well-established color scheme is not lost altogether
(as it would be in a pure grayscale visualization) while at the
same time avoiding a high-contrast black-and-white only im-
pression (which would result from using the most extreme
illustrativeness). An example with less visual emphasis to a
specific part is shown in Fig. 6(d) where the only changes
with respect to Fig. 6(c) are in the amount of illustrativeness
assigned to the two parts.
<table><caption>**Table 1:** Performance measures: bonds<sub>p</sub> & bonds<sub>s</sub> are primary and secondary bonds, FR<sub>1</sub> & FR<sub>2</sub> are average frame rates (fps) in non- and fully abstracted modes, respectively.</caption><thead><tr><th>Fig.</th><th>atoms</th><th>bonds<sub>p</sub></th><th>bonds<sub>s</sub></th><th>FR<sub>1</sub></th><th>FR<sub>2</sub></th></tr></thead><tbody><tr><td>2–5</td><td>796</td><td>152</td><td>647</td><td>328</td><td>1,151</td></tr><tr><td>1, 6(a, b)</td><td>12,605</td><td>4,645</td><td>8,300</td><td>93</td><td>341</td></tr><tr><td>n/a</td><td>58,674</td><td>24,024</td><td>35,063</td><td>23</td><td>79</td></tr></tbody></table>

## 5.2. Performance

To illustrate the performance we give rendering frame-rates on a 2.4 GHz Intel Core2Quad with 3GB memory and a NVIDIA GeForce 8800GTS 512MB card run under 64bit Linux in Table 1, for a range of molecule sizes. The frame rates are given for the molecules rendered at approximately 920×720 pixels for the two extremes, not abstracted at all (slowest performance overall) and full abstraction on all axes (requiring the rendering of less primitives).

## 5.3. Informal Feedback

We collected informal feedback from three computational chemists to whom we first demonstrated our implementation, pointing out the different abstraction axes and the stages they allow the visualization to go through. We also pointed out the fact that the transitions are continuous and that structural abstraction can be applied by protein sub-chain.

The experts pointed out that, while the individual structural abstraction levels can be generated with their current tools such as VMD [HDS96], the seamless transition between the levels that we provide cannot be produced. Instead, tools such as VMD only permit the α-blending between different visualizations. The chemists see the advantages in our approach, therefore, in that it is possible to transition between adjacent levels without having to specifically define these levels first, and in the continuity of the transition without jumps—the latter specifically for teaching. They speculated that they may be able to observe aspects in intermediate stages that would not be observable in the discrete stages or blended versions. One expert commented that the structure was more apparent in the toon-shaded version than in the traditional shading, and that he likes object attenuation and dislikes perspective stretching in VMD. Comparing to PyMol, he also commented that it is very powerful but also requires much more training to achieve nice results.

The chemists also suggested numerous possibilities for extension. They were interested in the interactive selection of sub-strands and the individual control of their abstraction parameters. Also, it was important that very large molecular systems can be treated, including those ≥ 10⁶ atoms such as large protein complexes. In addition, they would like to visualize molecular systems that also include solvents and other molecules, time-dependent data with varying time resolutions, and the possibility to show scripted animations.

# 6. Conclusions and Future Work

There are many illustrative visualizations that employ abstraction to some degree [RBGV08], and Rautek et al. are right to argue that abstraction is fundamental to illustrative visualization. However, many existing approaches either use some form of stylization (called “low-level visual abstractions” by Rautek et al.) which inherently abstract or apply means such as focus+context or distortion (called “high-level visual abstractions” by Rautek et al.). In contrast, we explore the *interactive and dedicated application of abstraction* in illustrative visualization along three different axes that all permit *simultaneous and seamless control*. This *seamless abstraction space* permits viewers to make mental connections between detailed and abstracted levels and, thus, to better understand these relationships and to interactively adjust the visualization to their current needs or specific data. Moreover, high-level visual abstractions are also possible with our approach (e. g., focus+context in Fig. 6(a)), but with a seamless control of the degree of the effect.

With our technique and its application to molecular visualization we have started to explore this dedicated control of abstraction, but there are more possibilities for realizing this control. There are a number of design decisions that we made that could have been made differently, and whose specific settings could be adjusted. Also, we concentrated on the visualization of biomolecules due to the availability of the data, while most of our techniques can also be applied to other types of molecules more generally. In that case we would need to limit ourselves to structural abstraction only from the space fill to a licorice visualization (skipping the backbone and ribbon levels) and may need to add other structural abstraction types depending on the domain.

There are a number of limitations in the current realization that need to be addressed in the future. In addition to increasing the supported size of molecules beyond about 10⁵ atoms it would also be helpful to add abstraction for tertiary and quaternary structures or coarse-grained models [MRY*07] as well as surface views and to include larger molecular systems consisting of several molecules and their interactions.

## References

[ARS79] APPEL A., ROHLF F. J., STEIN A. J.: The Haloed Line Effect for Hidden Line Elimination. *ACM SIGGRAPH Computer Graphics* 13, 3 (Aug. 1979), 151–157. doi> 10.1145/800249.807437

[BBB*02] BERMAN H. M., BATTISTUZ T., BHAT T. N., BLUHM W. F., BOURNE P. E., BURKHARDT K., FENG Z., GILLILAND G. L., IYPE L., JAIN S., FAGAN P., MARVIN J., PADILLA D., RAVICHANDRAN V., SCHNEIDER B., THANKI N., WEISSIG H., WESTBROOK J. D., ZARDECKI C.: The Protein Data Bank. *Acta Crystallographica Section D* 58 Part 6, 1 (June 2002), 899–907. doi> 10.1107/S0907444902003451

[BD04] BAJAJ C., DJEU P.: TexMol: Interactive Visual Exploration of Large Flexible Multi-Component Molecular Complexes. In *Proc. VIS* (2004), IEEE Computer Society Press, Los Alamos, pp. 243–250. doi> 10.1109/VISUAL.2004.103
[BG07] BRUCKNER S., GRÖLLER E.: Enhancing Depth-Perception with Flexible Volumetric Halos. *IEEE Transactions on Visualization and Computer Graphics* **13**, 6 (2007), 1344–1351. doi> 10.1109/TVCG.2007.70555

[Bre09] BREWER C. A.: ColorBrewer 2.0. Web site: http://www.ColorBrewer.org/, 2009. Accessed December 2010.

[BWF*00] BERMAN H. M., WESTBROOK J., FENG Z., GILLILAND G., BHAT T. N., WEISSIG H., SHINDYALOV I. N., BOURNE P. E.: The Protein Data Bank. *Nucleic Acids Research* **28**, 1 (Jan. 2000), 235–242. doi> 10.1093/nar/28.1.235

[CG07] CIPRIANO G., GLEICHER M.: Molecular Surface Abstraction. *IEEE Transactions on Visualization and Computer Graphics* **13**, 6 (Nov./Dec. 2007), 1608–1615. doi> 10.1109/TVCG.2007.70578

[CPG09] CIPRIANO G., PHILLIPS JR. G. N., GLEICHER M.: Multi-Scale Surface Descriptors. *IEEE Transactions on Visualization and Computer Graphics* **15**, 6 (Nov./Dec. 2009), 1201–1208. doi> 10.1109/TVCG.2009.168

[CWG*10] CIPRIANO G., WESENBERG G., GRIM T., PHILLIPS JR. G. N., GLEICHER M.: GRAPE: GRaphical Abstracted Protein Explorer. *Nucleic Acids Research* **38**, Web Server issue (July 2010), W595–W601. doi> 10.1093/nar/gkq398

[DB04] DELANO W. L., BROMBERG S.: *PyMOL User's Guide*. DeLano Scientific LLC, 2004, ch. Cartoon Representations.

[Dec96] DECAUDIN P.: *Cartoon Looking Rendering of 3D Scenes*. Research Report 2919, INRIA, June 1996.

[DHR*99] DEUSSEN O., HAMEL J., RAAB A., SCHLECHTWEG S., STROTHOTTE T.: An Illustration Technique Using Intersections and Skeletons. In *Proc. Graphics Interface* (1999), Morgan Kaufmann, San Francisco, pp. 175–182. doi> 10.1145/351631.351683

[EBRI09] EVERTS M. H., BEKKER H., ROERDINK J. B. T. M., ISENBERG T.: Depth-Dependent Halos: Illustrative Rendering of Dense Line Data. *IEEE Transactions on Visualization and Computer Graphics* **15**, 6 (Nov./Dec. 2009), 1299–1306. doi> 10.1109/TVCG.2009.138

[GO92] GOODSELL D. S., OLSON A. J.: Molecular Illustration in Black and White. *Journal of Molecular Graphics* **10**, 4 (Dec. 1992), 235–240. doi> 10.1016/0263-7855(92)80075-0

[Goo03] GOODSELL D. S.: Illustrating Molecules. In *The Guild Handbook of Scientific Illustration*, Hodges E. R. S., (Ed.), 2nd ed. John Wiley & Sons, Hoboken, NJ, 2003, ch. 15, pp. 267–270.

[Goo05] GOODSELL D. S.: Visual Methods from Atoms to Cells. *Structure* **13**, 3 (Mar. 2005), 347–354. doi> 10.1016/j.str.2005.01.012

[HB03] HARROWER M. A., BREWER C. A.: ColorBrewer.org: An Online Tool for Selecting Color Schemes for Maps. *The Cartographic Journal* **40**, 1 (June 2003), 27–37. doi> 10.1179/000870403235002042

[HDS96] HUMPHREY W., DALKE A., SCHULTEN K.: VMD – Visual Molecular Dynamics. *Journal of Molecular Graphics* **14**, 1 (Feb. 1996), 33–38. doi> 10.1016/0263-7855(96)00018-5

[Hod03] HODGES E. R. S. (Ed.): *The Guild Handbook of Scientific Illustration*, 2nd ed. John Wiley & Sons, Hoboken, NJ, 2003.

[IFH*03] ISENBERG T., FREUDENBERG B., HALPER N., SCHLECHTWEG S., STROTHOTTE T.: A Developer’s Guide to Silhouette Algorithms for Polygonal Models. *IEEE Computer Graphics and Applications* **23**, 4 (July/Aug. 2003), 28–37. doi> 10.1109/MCG.2003.1210862

[KBE09] KRONE M., BIDMON K., ERTL T.: Interactive Visualization of Molecular Surface Dynamics. *IEEE Transactions*

on Visualization and Computer Graphics **15**, 6 (Nov./Dec. 2009), 1391–1398. doi> 10.1109/TVCG.2009.157

[Kra91] KRAULIS P. J.: MOLSCRIPT: A Program to Produce both Detailed and Schematic Plots of Protein Structures. *Journal of Applied Crystallography* **24**, 5 (Oct. 1991), 946–950. doi> 10.1107/S0021889891004399

[Leb96] LEBAREDIAN R.: Traditional Cel Animation Look with 3D Renderers. In *ACM SIGGRAPH 96 Visual Proceedings* (1996), ACM, New York, p. 154. doi> 10.1145/253607.253898

[Lei94] LEISTER W.: Computer Generated Copper Plates. *Computer Graphics Forum* **13**, 1 (Mar. 1994), 69–77. doi> 10.1111/1467-8659.1310069

[LKZD08] LUFT T., KOBS F., ZINSER W., DEUSSEN O.: Watercolor Illustrations of CAD Data. In *Proc. CAe* (2008), Eurographics Association, Goslar, Germany, pp. 57–63. doi> 10.2312/COMPAESTH/COMPAESTH08/057-063

[LLC*11] LIPŞA D. R., LARAMEE R. S., COX S. J., ROBERTS J. C., WALKER R.: Visualization for the Physical Sciences. In *Eurographics 2011 State of the Art Reports*. Eurographics Association, Goslar, Germany, 2011, pp. 49–73.

[LMHB00] LAKE A., MARSHALL C., HARRIS M., BLACKSTEIN M.: Stylized Rendering Techniques for Scalable Real-Time 3DAnimation. In *Proc. NPAR* (2000), ACM, New York, pp. 13–20. doi> 10.1145/340916.340918

[LVRH07] LAMPE O. D., VIOLA I., REUTER N., HAUSER H.: Two-Level Approach to Efficient Visualization of Protein Dynamics. *IEEE Transactions on Visualization and Computer Graphics* **13**, 6 (Nov./Dec. 2007), 1616–1623. doi> 10.1109/TVCG.2007.70607

[MRY*07] MARRINK S. J., RISSELADA H. J., YEFIMOV S., TIELEMAN D. P., DE VRIES A. H.: The MARTINI Force Field: Coarse Grained Model for Biomolecular Simulations. *The Journal of Physical Chemistry B* **111**, 27 (2007), 7812–7824. doi> 10.1021/jp071097f

[NCS88] NAMBA K., CASPAR D. L. D., STUBBS G.: Enhancement and Simplification of Macromolecular Images. *Biophysical Journal* **53**, 4 (Apr. 1988), 469–475. doi> 10.1016/S0006-3495 (88)83125-4

[OGF*10] O'DONOGHUE S. I., GOODSELL D. S., FRANGAKIS A. S., JOSSINET F., LASKOWSKI R. A., NILGES M., SAIBIL H. R., SCHAFFERHANS A., WADE R. C., WESTHOF E., OLSON A. J.: Visualization of Macromolecular Structures. *Nature Methods* **7**, 3s (Mar. 2010), S42–S55. doi> 10.1038/nmeth.1427

[RBGV08] RAUTEK P., BRUCKNER S., GRÖLLER E., VIOLA I.: Illustrative Visualization: New Technology or Useless Tautology? *ACM SIGGRAPH Computer Graphics* **42**, 3 (Aug. 2008), 4:1–4:8. doi> 10.1145/1408626.1408633

[Ric85] RICHARDSON J. S.: Schematic Drawings of Protein Structures. In *Diffraction Methods for Biological Macromolecules Part B*, vol. 115 of Methods in Enzymology. Academic Press, 1985, pp. 359–380. doi> 10.1016/0076-6879(85)15026-3

[SEI10] SVETACHOV P., EVERTS M. H., ISENBERG T.: DTI in Context: Illustrating Brain Fiber Tracts In Situ. *Computer Graphics Forum* **29**, 3 (June 2010), 1024–1032. doi> 10.1111/j.1467-8659.2009.01692.x

[TCM06] TARINI M., CIGNONI P., MONTANI C.: Ambient Occlusion and Edge Cueing to Enhance Real Time Molecular Visualization. *IEEE Transactions on Visualization and Computer Graphics* **12**, 5 (Sept./Oct. 2006), 1237–884. See too QuteMol (http://qutemol.sourceforge.net/). doi> 10.1109/TVCG.2006.115

[Web09] WEBER J. R.: ProteinShader: Illustrative Rendering of Macromolecules. *BMC Structural Biology* **9**, 19 (May 2009), 1–19. doi> 10.1186/1472-6807-9-19