# Text Priming - Effects of Text Visualizations on Readers Prior to Reading

Tilman Dingler, Dagmar Kern, Katrin Angerbauer, Albrecht Schmidt

► To cite this version:

Tilman Dingler, Dagmar Kern, Katrin Angerbauer, Albrecht Schmidt. Text Priming - Effects of Text Visualizations on Readers Prior to Reading. 16th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2017, Bombay, India. pp.345-365, 10.1007/978-3-319-67687-6_23. hal-01717225

HAL Id: hal-01717225

https://inria.hal.science/hal-01717225

Submitted on 26 Feb 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Text Priming - Effects of Text Visualizations on Readers Prior to Reading

Tilman Dingler¹, Dagmar Kern², Katrin Angerbauer¹, and Albrecht Schmidt¹

¹ VIS, University of Stuttgart, Stuttgart, Germany
`firstname.lastname@vis.uni-stuttgart.de`

² GESIS Leibniz Institute for the Social Sciences, Cologne, Germany
`dagmar.kern@gesis.org`

**Abstract.** Living in our information society poses the challenge of having to deal with a plethora of information. While most content is represented through text, keyword extraction and visualization techniques allow the processing and adjustment of text presentation to the readers' individual requirements and preferences. In this paper, we investigate four types of text visualizations and their feasibility to give readers an overview before they actually engage with a text: word clouds, highlighting, mind maps, and image collages. In a user study with 50 participants, we assessed the effects of such visualizations on reading comprehension, reading time, and subjective impressions. Results show that (1) mind maps best support readers in getting the gist of a text, (2) they also give better subjective impressions on text content and structure, and (3) highlighting keywords in a text before reading helps to reduce reading time. We discuss a set of guidelines to inform the design of automated systems for creating text visualizations for reader support.

**Keywords:** priming, reading interfaces, comprehension, text visualization

## 1 Introduction

Reading is an activity needed in most facets of our daily life. By reading, people gather information from text, extract meaning, build knowledge and foster other cognitive abilities, such as reasoning [38]. The information age changes our reading behavior. With an abundance of information made available through the Internet and accessible on electronic devices, reading becomes a ubiquitous activity and the overall time spent reading increases [29]. However, the activity of reading on computers and other electronic devices tends to be shallower than on conventional media [12] and the skill to quickly scan a document in order to get the gist of its content becomes important.

Visualizations can be used to make readers quickly filter out the essence of a text and help with the decision on whether a text is worth reading in the first place [28]. Understanding the gist of a text supports overall text comprehension and allows readers to process a text's content and meaning in more detail. By
facilitating text reading in this way, people could manage their daily readings
more effectively by reducing overall reading time or increasing reading volume.

In this work, we investigate the use of text visualizations to support reading activities on electronic devices with a special focus on effective gist extraction, comprehension, and perceived usefulness. Here, we make use of the *Priming Effect* – a concept taken from psychology that describes an implicit memory effect: by exposing users to a current stimulus, a response to another stimulus is triggered in the future [4]. We base our use of the term on the more conceptual definition of how the exposure to one stimulus influences the perception of a subsequent one. While this is barely a recognition task in the range of milliseconds, we use it to investigate an effect in higher-level cognitive tasks by applying the effect to reading. Therefore, we launched an investigation into the utility and effects of four different text visualizations: word cloud, highlighting, mind map, and image collage. After having conducted an initial focus group, we collected both quantitative and qualitative results in a comprehensive user study with 50 participants. Results show the feasibility of using mind maps and text highlights as primers. Based on our findings we derive requirements for building automated text visualizations for reading on electronic devices. The contribution of our work is as follows:

1. We show that text visualizations can be used to get the gist of a text prior to reading.

2. We compare four visualization types with regard to their utility, finding that mind maps are most useful in aiding understanding.

3. We provide insights into how highlighting can help to reduce reading time by giving readers a better text understanding and to go through text quicker.

4. We present an architecture for an automated system to support readers on electronic reading devices through visualizations.

## 2 Background and related work

Our research combines aspects of different research fields. In this section, we provide same insights into reading behavior and how it has changed as well as reading comprehension. Furthermore, we introduce the priming effect and provide information about text summaries and visualizations.

### 2.1 Reading Behavior

Cunningham and Stanovich [38] speak of the *Matthew's Effect* when describing a 'rich get richer, poor get poorer' - phenomenon for reading. The ability to comprehend and read large volumes creates a reciprocal relationship: readers with a broader knowledge base carry out a smaller amount of effort while gaining a more positive experience than readers who struggle to comprehend text [10]. Positive experiences can be a great motivation which puts avid readers into a positive feedback loop, which eventually affects vocabulary development, abilities of expression, and reasoning. Liu [29] investigated the effects the information age has
had on people's reading behavior: with more information constantly available, the time spent on reading increased, specifically the amount of work related reading. On the other hand, reading patterns get more shallow and fragmented as people read in less depth. This seems to be a coping mechanism through which people get through the high amount of daily readings and only read a text in detail if it is assumed to be highly relevant. The ability to scan and browse text for pointers to valuable information becomes increasingly important.

## 2.2 Reading Comprehension and Recall

Reading comprehension is essential to extract meaning from text. Comprehending a text is a combination of recalling what has been read and the understanding of the underlying concepts [31]. Therefore, the text needs to be interacted with on a mental level and connected to the reader's existing knowledge of the world [1]. Being aware of the context instead of just the meaning of single words is also crucial for understanding. Federmeier and Kutas [17] showed that words are often predictable in sentence contexts and that sentence contexts can facilitate comprehension. Bransford and Johnson [8] investigated the role of contextual knowledge where comprehension often results from the context. In their experiments, they showed that participants who had to comprehend a text passage without being provided with the appropriate context did worse in comprehension tests than those who had been provided with the context. Another result of their experiments is that prior knowledge alone is not always helpful for the comprehension process but that it has to be activated first through some stimulus. Text comprehension and memory are deeply intertwined. Kintsch [24] explicitly contrasted remembering and learning, where remembering entails the process of simply reproducing the information of a text. This requires a shallower form of comprehension, whereas if a reader has effectively learned from a text, she can apply that knowledge to new situations. Reading comprehension is therefore a multi-faceted measure, which often is assessed by the reader's ability to recall what is stated in the text (literal comprehension), interpret the text's meaning by connecting rather implicitly stated information (inferential comprehension), or to relate the text's content to prior experiences and knowledge (evaluative comprehension) [7]. Learning has occurred, if a reader has been able to build a well-structured representation of the text and connect this representation to the background knowledge she possesses [30].

## 2.3 The Priming Effect

When a stimulus subconsciously influences the processing of the following stimulus, psychologists speak of a *Priming Effect*. Influencing stimuli, the so-called *primes*, can have an effect on memory processing tasks [4]. Research in psychology has confirmed this effect for reading activities. Typical implicit memory testing methods include word primes, such as, for example for the prime 'apple' [33]: for word stem completion, such as AP - -, for word fragment completion: _ P _ L -, for anagram solving: 'plape', and word identification tasks.
Rajaram and Roediger [33] apply these tasks with different primes in different modalities (auditory or visual priming), forms (word or picture) and typefaces to evaluate the magnitude of priming effects. Their findings show that for modalities showing visual primes for visual tasks and auditory primes for auditory tasks is recommendable. Hence, we were interested in whether this effect can be used when using text summary visualizations as primers before reading a text to foster text comprehension and recall.

## 2.4 Text Summaries

Related work about text summaries belongs to the field of text mining. Text mining, in general, describes the process of extracting high-quality information from documents (see [22] for an overview). In this work, we focus on extracting keywords from text and make use of the term-frequency-inverse document frequency (tf-idf) method [34]. This method determines the relevance for each term by combining the term frequency (tf) of a term (a term with a high frequency in a document is probably more important than a term with a low frequency) with the inverse document frequency (a term that has a high frequency over all documents is less descriptive for a particular document). For example, the frequency of the term “and” is high in a single document as well as in a set of documents. Therefore, the tf-idf score is quite low. Whereas, the term “priming” has a high frequency in one document but not in the set of specific documents the tf-idf score is high and indicates an importance of this term. To reduce the impact of larger tf-idf values, a log-scale is applied:

$$tf - idf_{t,d} = tf_{t,d} \times \log\left(\frac{N}{df_t}\right)$$

Here, *t* stands for the term under investigation, *d* for the document in a corpus *t* belongs to. *N* depicts the total number of documents in the corpus which is divided by the number of documents containing the term *df*.

## 2.5 Text Visualization

Banas and Brown [5] define visualization as "the conversion of information to a symbolic representation of a particular idea, concept, or data object". Hence, representations can reveal patterns, relationships, and concepts helping observers to new understandings and deeper comprehension of text. Text visualizations also convey information quickly and efficiently and generally aim to make the comprehension of aspects easier. Kalmane [23] stresses the potential of online visualization tools to assist reading comprehension, especially for language learners. Adding context-relevant images, for example, helps understanding and learning overall [11]. In traditional textbooks, visualizations were often crafted for illustrative purposes, which were rather laborious to create. Now it is pragmatical to generate visualizations automatically and support readers in-situ. A lot of
research has been carried out to automatically generate text visualizations. The complexity of these text visualizations ranges from visualizing single documents (see, e.g., [21, 40]) over visualizing the context of a book (see, e.g., [14, 32]) to visualizing documents and their relationships in digital libraries (see, e.g., [42]). Comprehensive overviews of different text visualization techniques are provided by Alencar et al. [2] or Kucher and Kerren [25]. In our work, we focus on single document visualizations based on their relation to a corpus of related documents.

# 3 Visualizations for text priming

We decided to investigate four different types of text visualizations with different characteristics regarding their effects on reading activities: word cloud (representing all extracted words), highlighting (representing keywords in their context), mind map (representing selected keywords and their connections to each other), and image collage (representing a couple of keywords in a more abstract form). We chose these common visualization types as they are rather well-established representations of texts. With regard to our user study, we thereby attempted to minimize possible novelty effects. In the following section, we describe these four visualization types, discuss why we think that they are worth considering for invigorating priming effects and how we designed them for our user study. In all cases, we build upon the 50 most relevant keywords extracted from a text through tf-idf.

## 3.1 Word Cloud

A set of visually presented words arranged in a cloud is commonly known as a "tag cloud". If the content of a text is summarized, it is also often called a "word cloud", which is the term we will use throughout this paper. Viegas and Watenberg [41] provide a comprehensive overview of the history and different types of tag or word clouds. Such word clouds have been widely used for teaching and learning (see, e.g., [6]), which is why we are interested in their feasibility to prime a reader for a text. We apply a simple design for visualizing the 50 most descriptive keywords of a text by varying font sizes according to keyword's importance and a color scheme that does not distract the reader but fosters readability. In our design, size and color reflect the importance of a word. The sequential word order of the keywords aims to mirror the text's structure. Figure 1A shows a word cloud, which we used in our study.

## 3.2 Highlighting

Printed text often contains in-text highlights in the form of keywords marked by being given a background color. Different studies showed the positive effects highlighting has on text retention (see, e.g., [18, 19]), whereby actively highlighting, performed by the reader herself during reading is considered more effective than pre-existing highlights [19]. However, there are also studies that conclude
<figure><img src="image_2.png" /><figcaption>Fig. 1. Visualizations used in our user study summarizing a text about learning.</figcaption></figure>

that reading a text that has been highlighted by expert readers improves re-
call of the highlighted text passage [13, 15]. This also implies that the quality of
the highlighting plays an important role, as inappropriate highlighting can have
negative effects on reading comprehension [20, 37]. The highlighting approach in
contrast to using word clouds retains the context in which the word occurs and
may provide more information at a glance. According to the guidelines on text
highlighting techniques in [39], we chose to color the background of the 50 most
relevant keywords in yellow. Figure 1B shows an extract of a highlighted text,
which we used in our study.

3.3 Mind Map

Mind maps [9] are often used for organizing information by grouping impor-
tant aspects into clusters and thereby have the ability to structure the informa-
tion [43]. Structured information can be learned better and is processed more
efficiently [4, 26]. We assume that due to the visualized connection, the gist of a
text might be derivable and prime the reader for the text. The mind maps used
in our studies are manually created, and we concentrate only on a few keywords
and their relation to each other, to make the mind maps clear and usable. Dur-
ing initial trials we found about 15 keywords to be appropriate to represent our
study texts whose length varied between 500 and 600 words. Some of them are
grouped together to describe a single concept, e.g., the keywords "successful",
"learning" and "happiness" are part of the concept: "successful learning leads
to happiness". Figure 1C shows a summary of one of the texts used throughout
our study.

3.4 Image Collage

Following the ancient proverbs "every picture tells a story" or "a picture says
more than thousand words", an image collage seems to be a suitable way to sum-
marize text visually. Results of studies performed throughout the 1990s support
this assumption as they have shown that depictive images aid the learning pro-
cess [11]. Zhu et al. [44] presented a Text-to-Picture system capable of creating
an Image Collage of short texts. The images are synthesized from some of the
text's most relevant keywords. Another method was introduced by Aramini et
al. [3]: their system generates unsupervised and automatically visualization of
shortened texts using images that can be found on the Web. We pursue a similar
approach and search for suitable images corresponding to important keywords
on creative common image portals like *Flickr* or *Google Creative Commons Im-
age Search*. Not all of the 50 keywords can be well represented by an image.
For creating the image collages for our user study, we analyzed the keywords
regarding their metaphorical expression. We conclude that for each text eight
pictures representing the main keywords can be chosen (see Figure 1D).

4 Pre-Study

To collect initial qualitative feedback on our general approach we conducted a
pre-study in the form of focus groups.

## 4.1 Methodology and Procedure

We held two focus groups with four participants (3 female) each lasting about
90 minutes. All participants were university students with different backgrounds
and between 19 and 24 years old (M = 22.25, SD = 1.67). The native tongue
of everyone was German. We chose four texts with different text styles (fiction,
news and science) and created four visualizations whereby the word cloud vi-
sualized a fictional text, highlighting was used for an abstract of a scientific
paper, the mind map summarized a news article and the image collage a sci-
entific article. Discussions were recorded in a written protocol, after which the
participants' comments were sorted into categories. Both visualization and text
were in German and printed on paper. The main part of the focus group com-
prised discussions on the four visualizations regarding their usage and utility
before, during and after reading the text. The four visualizations were discussed
one after the other, and in the end, general design aspects of such visualizations
were debated.

*Results and Discussion* From the discussions and preliminary design explorations we derived ideas for using visualizations as pre-, during-, and after-reading support

*Pre-Reading* Before diving into the actual text, participants found the visualiza-
tions helpful in general but mind map and image collage in particular. On the
other hand, however, they were sometimes confused as some content in the visu-
alizations did not make immediate sense to them. In some cases, the keywords
were even found misleading. Furthermore, it was mentioned that visualizations
could also help deciding whether a text is relevant to a certain topic.

*During-Reading* Having visualizations accessible during reading, participants found it easier to keep an overview of the structure of the text. One participant stated that the mind map visualization helped to structure the text while another said the highlighting helped to pay attention to important parts of the text.
*Post-Reading* All participants found the visualizations to be very helpful in the post-reading phase, especially as memory aids when they needed to retain the texts' contents. One participant had doubts about the usefulness of pictures as a memory aid after a long period and supposed textual cues would be more helpful. Another participant found post-reading support more important than pre-reading support.

In general, participants preferred getting a quick overview of a text. If a lot of
time was required to look at a visualization to make sense of it, it was regarded as
less beneficial. Participants said they had to spend the most time trying to figure
out text contents from the word cloud, whereas the content of the image collage
visualization was perceived quickest. During the study, no clear preference for
one type of visualization could be observed. Especially the use of images as cues
was controversial: some participants stated they found images most helpful, while
others considered them least informative and preferred textual cues. Because no
clear preferences for one of the visualization could be determined within the
focus groups, we designed a user study for investigating all four visualizations.
We set out to research the effect of such visualizations on the reader before,
during and after reading. We start with the before-reading phase and present
this user study as well as its results in the following sections.

5 User Study

With the goal of investigating the feasibility and appropriateness of our four text visualizations to prime readers for going through a text on electronic devices, we conducted a user study. As the Priming Effect is complex and cannot be regarded as a unitary effect [35] we focus on different aspects to approximate it:

1. The first impression the reader gets about the text after seeing the visual-
ization but before reading it.

2. The effects on reading comprehension and time spent reading.

3. The subjective utility of visualizations and how they help readers gain an overview of a text and its content.

5.1 Apparatus

As reading material, we chose five standardized texts from the Institute for Ger-
man as a Foreign Language (TestDaF). Since we expected participants to be
native German speakers, we selected texts in German dealing with issues of gen-
eral interest (e.g., “Earthquakes” or “The effect of nuts on health”). Text lengths
ranged between 500 and 600 words. Each text came with ten multiple choice
questions, each with three possible answers for assessing reading comprehension.
For each text we extracted the 50 most relevant keywords: each text was prepro-
cessed with *TreeTagger* [36]. From its output, we only considered nouns, verbs,
and adjectives as well as removed frequent stopwords. For tf-idf calculations, we
<table><caption>**Table 1.** Structure of the survey. In the baseline condition (no visualization) only the steps 'Text' and 'Comprehension questions' were utilized in section B.</caption><tr><td rowspan="3">A</td><td>Introduction</td></tr><tr><td>Demographic questions</td></tr><tr><td>Questions on reading behavior</td></tr><tr><td rowspan="4">B<br/>repeated for each visualization and the<br/>baseline condition</td><td>Visualization of the text (visible for 1 minute)</td></tr><tr><td>Question "What do you think the text is about?"</td></tr><tr><td>Text</td></tr><tr><td>Evaluation of the visualization according to the following characteristics on a 5-point Likert-scale from "I disagree" (1) to "I strongly agree" (5):<br/>1. The visualization helped me to understand the text. (<b>Comprehension</b>)<br/>2. The visualization helped to get the gist of the text. (<b>Gist</b>)<br/>3. The visualization reflected the content of the text comprehensively. (<b>Fit</b>)<br/>4. The visualization provided an overview of the text structure. (<b>Structure</b>)</td></tr><tr><td></td><td>Comprehension questions</td></tr><tr><td>C</td><td>Final ratings: Which is the most helpful visualization? Which visualization do you like most regarding design? And why?</td></tr></table>

clicking on "next". A question about the content of the text followed before the actual text was presented to participants. There was no time-limit for reading the text. Then, questions about the visualization were asked, before ten comprehension questions had to be answered. This step B (see Table 1) was repeated for each visualization and the baseline condition. In the baseline condition, participants had only to read the text and to answer the comprehension questions. The study took about 45 minutes for each participant.

## 5.4 Participants

We recruited 50 participants (15 female) with their age ranging from 18 to 60 ($M = 27.7, SD = 6.19$). 32 were university students from different fields of study, while 10 were researchers and the remaining seven worked as software developers or secretaries. 45 participants stated German to be their mother tongue, and the others spoke German fluently. 21 participants were considered to be frequent readers (more than seven hours reading time per week), 12 to be casual readers (between four and six hours reading time per week) and 17 to be scarce readers (less than three hours reading time per week).

## 6 Results

In the following, we present our results from the subjective assessments, pre-impression analysis, comprehension tests, reading time, and free commenting.
## 6.1 Quantitative Measures

*User Ratings* For the following analysis, we applied Friedman tests followed by Wilcoxon signed-rank tests with a Bonferroni correction applied, which resulted in a significance level of *p*<0.008. Figure 2 contains an overview of all subjective ratings with regard to each visualization and the four characteristics: Comprehension, Gist, Fit and Structure as shown in Table 1.

Looking at the subjective user feedback regarding the utility of the visualizations, there was a statistically significant difference in perceived **comprehension** with regard to visualization type, $\chi^2(3) = 33.725, p < 0.001$. Median perceived comprehension ratings for word cloud, mind map, highlighting, and image collage were 2 (1 to 3), 4 (2.75 to 4.25), 3.5 (2 to 4), and 2 (1 to 3), respectively. There was a statistically significant difference in perceived comprehension in favor of mind map over word cloud ($Z = -4.36, p < 0.001$) and over image collage ($Z = -4.022, p < 0.001$). Also, highlighting was rated significantly higher than word cloud ($Z = -2.636, p = 0.008$) and image collage ($Z = -3.344, p < 0.001$).

As for perceived text content overview—i.e., getting the **gist** of a text— there was a statistically significant difference in how much of the the different visualization types conveyed, $\chi^2(3) = 38.173, p < 0.001$. Median perceived content overview ratings for word cloud, mind map, highlighting, and image collage were 4 (2 to 4), 5 (4 to 5), 4 (3 to 4), and 2 (1 to 4), respectively. Post-hoc tests showed a statistically significant difference in how well the visualizations provided a content overview with mind map being rated best over word cloud ($Z = -4.336, p < 0.001$), image collage ($Z = -4.534, p < 0.001$), and highlighting ($Z = -2.806, p = 0.005$). Highlighting was better received than image collage ($Z = -3.998, p < 0.001$). No statistically significant difference was found between highlighting and word cloud ($Z = -1.496, p = 0.135$) and between word cloud and image collage ($Z = -1.678, p = 0.093$).

There was a statistically significant difference in how comprehensive the corresponding visualization **fit** the text content, $\chi^2(3) = 36.491, p < 0.001$. Median perceived ratings for word cloud, mind map, highlighting, and image collage were 2 (1 to 3.25), 4 (2 to 4), 3 (2 to 4), and 2 (1 to 2), respectively. There were statistically significant differences with mind map being rated better than word cloud ($Z = -3.822, p < 0.001$) and image collage ($Z = -4.627, p < 0.001$). Also, highlighting was better perceived to comprehensively reflect the content than image collage ($Z = -3.719, p < 0.001$). No statistically significant difference was found between highlighting and word cloud ($Z = -1.984, p = 0.47$), image collage and word cloud ($Z = -1.921, p = 0.055$) or between highlighting and mind map ($Z = -2.169, p = 0.03$).

Finally, results yielded a statistically significant difference in how well visualization provided an overview of the text **structure**, $\chi^2(3) = 48.850, p < 0.001$. Median perceived ratings for word cloud, mind map, highlighting, and image collage were 2 (1 to 4), 4 (3 to 5), 4 (2 to 4), and 2 (1 to 2), respectively. Post-hoc tests showed a statistically significant difference in structural overview ratings with mind map being rated better than word cloud ($Z = -4.599, p < 0.001$) and image collage ($Z = -5.224, p < 0.001$). Highlighting was better perceived
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Subjective user ratings (mean and standard errors) for each visualization type with regard to their utility for supporting text comprehension, understanding the gist of a text, its fit to reflect the text's content and its adequacy to reflect the text's structure.</figcaption></figure>

than word cloud ($Z = -2.940, p = 0.003$) and image collage ($Z = -4.670, p < 0.001$). No statistically significant difference was found between word cloud and image collage ($Z = -1.965, p = 0.49$) or between highlighting and mind map ($Z = -2.136, p = 0.033$).

*Pre-Impressions* After being shown each visualization and before actually being presented the full text, participants were asked about their pre-impression, i.e. what they thought the text's content was about. We had three independent researchers assess each comment according to how well they fit the actual text content on a school grading scale from 1 (being best) to 6 (being mislead). Therefore, we instructed the researchers to assign grade 4 when the rough topic was identified, 5 when the topic was incorrectly identified but was at least similar, and 6 when the participant's assumption did not match the content of the text at all. Grades from 1 to 3 depended on the level of detail. We then calculated a mean inter-annotator agreement score for each pre-impression statement and performed a Friedman test on the resulting final grade. There was a statistically significant difference in calculated grades based on the quality of participants' pre-impressions, $\chi^2(3) = 47.730, p < 0.001$. Median perceived ratings for word cloud, mind map, highlighting, and image collage were 4 (3 to 4.25), 3 (2 to 4), 4 (3 to 4), and 4 (4 to 5), respectively. There were statistically significant differences with mind map leading to most accurate pre-assessments compared to tag cloud ($Z = -3.820, p < 0.001$), highlighting ($Z = -3.987, p < 0.001$),
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Participants’ ratings of different visualization types regarding appeal and helpfulness.</figcaption></figure>

and image collage (Z = -5.393, *p* < 0.001). Also, tag cloud performed better than image collage (Z = -3.053, *p* = 0.002), so did highlighting (Z = -4.046, *p* < 0.001). Highlighting and tag cloud seemed to perform similarly resulting in a non-significant difference (Z = -0.767, *p* = 0.443).

*Comprehension Scores and Time Spent Reading* For comprehension scores, a one-way repeated measures ANOVA with a Greenhouse-Geisser correction yielded no statistical significance between conditions (F(3.314, 162.402) = 1.086, *p* = 0.36). We looked at how much time it took participants to read the prospective texts followed by studying the visualizations for 60 seconds. Most participants took full advantage of this time interval. Hence, there was no statistically difference in visualization viewing time (F(1.158, 56.731) = 0.296, *p* = 0.622). With regard to time spent reading the full text, there was a significant difference based on which visualizations participants studied beforehand (F(3.335, 163.435) = 3.087, *p* = 0.024). Post hoc tests using the Bonferroni correction revealed that readers spent less time reading after a highlighting visualization compared to viewing an image collage (Z = -3.384, *p* = 0.001). On average, readers who viewed a highlighted version of the text were 24.9 seconds (i.e., 12.1%) faster than when viewing an image collage (compared to word cloud: 10.4%, mind map: 5.5%, and without visualization: 9.2%) No other pairwise comparisons yielded any statistically significant results, however there is a tendency of highlighting to help reduce reading time as compared to word clouds (Z = -2.736, *p* = 0.006) and reading without any prior visualizations (Z = -2.667, *p* = 0.008).
## 6.2 Qualitative Measures

Participants were asked which visualization they found most appealing and most helpful (see Figure 3). They were encouraged to provide explanations of their assessment, which we analyzed by dividing them into single statements and assigning them to respective visualizations. Altogether, we collected 32 statements regarding appeal and 52 statements regarding helpfulness. In the following, answers given by more than one participants are listed and the number of occurrences is shown in brackets after each provided statement: the image collage was described as a simple and clear design, but at the same time with a lack of expressivity (N=3). It is clearly arranged (N=3), arouses interest to read the text (N=2), is good to remember things (N=2) and helps to formulate a mental image of the content (N=2). On the other hand, participants pointed out that “Pictures are open to interpretation, therefore, they might mislead the reader.” (N=2). Highlighting provides the structure of the text (N=5) and shows the words in their context (N=4). However, two subjects were overstrained by the volume of text. Comments for the mind map were predominantly positive. It was stated as clearly and logically structured (N=9). It summarized the essential information of the text (N=11), shows connection between the information (N=10), allows recognizing the structure of the text (N=9) and contributes to the understanding (N=4). Participants appreciated the word cloud for providing the most important words (N=4) in a pretty way (N=3). But the word cloud was also considered as too overloaded (N=2). In general, there was a perceived benefit of looking at text visualizations before reading a text. This is supported by the fact that nobody answered the question “Which visualization do you find most helpful?” with none.

## 7 Discussion

The initial focus group gave us insights into the applicability of visualizations before, during, and after the actual reading process. Due to the feedback, we set out to assess the feasibility of using text visualizations as primers to improve text reading. Therefore, we conducted a user study examining the utility of four different visualizations types.

We could not confirm an objective effect difference on comprehension between those visualizations. When designing studies around reading it is crucial to not only pay attention to the nature of the texts used, but also to the difficulties of measuring text comprehension. We opted for a solution in which the texts came with pre-defined comprehension tests as they were commonly used in current language proficiency assessments. Hence, they often required not only literal translation, but also transfer skills. When reviewing the test results, we noticed that many participants had had difficulties with these type of questions resulting in a rather homogeneous distribution of mean comprehension scores, which is why we also analyzed the subjectively perceived effects that visualizations had on readers. Here, we found effects on subjective perception of using visualizations before reading with regard to their utility to convey the text's
content and structure. The mind map was rated most helpful in providing an overview and getting the gist of a text. Its nature allows communicating content and structural information more comprehensively since the mind map's visual design often reflects semantic and structural relationships in the text.

Also, highlighting lead to better subjective text comprehension. By studying highlighted keywords of a text before reading it was shown in our study to give readers an impression of the text's gist. We also found that it reduced the actual reading time by up to 12.1%. This could be due to the readers already being exposed to the entire text beforehand, where participants were able to read ahead. However, some of the participants claimed that due to the time limit of 60 seconds they had problems to capture all of the highlighted keywords, which might be an argument against the premature detailed reading.

By objectively assessing the quality of participants' presumed text summaries prior to reading, we find that visualizations can give readers a good first impression of the content. Mind maps—due to being rich in structure and content information—helped participants' most in grasping the text content, while highlighting and word cloud performed better than image collage. These findings show the usefulness of visualizations, such as mind maps, to allow readers to skim and possibly disregard a text quickly. This skill is especially useful in situations or jobs, where people need to process a lot of text.

We were further able to relate effects of inappropriate highlighting to participants' feedback stating that visualizations could be misleading. One participant pointed out that highlighting a verb without its negation might be interpreted completely different. The benefit of highlighting is strongly dependent on the quality of the underlying keyword extraction. As Silvers and Kreiner [37] showed that inappropriate highlighting could have negative effects on reading comprehension, merely using tf-idf scores might be too simplistic. More sophisticated topic models, such as Latent Dirichlet Allocation (LDA), need to be exerted in order to extract meaningful keywords and therefore convey text context more accurately to support comprehension. Image collages also bear the risks to prime the user in different directions. This is caused by the rather ambiguous nature of images since they often trigger different associations, which is challenging to design for deliberately.

During the pre-study, we identified three strands of application scenarios, namely using visualizations as primes before reading, as an aid to keep an overview during, and as a summary support for after reading. A combination of those could have profound effects on readers' ability to quickly get the gist of a text in order to decide on its relevancy. Then during reading contextual graphics are provided to aid readers' sense of text position and sequence of argumentation. And afterwards, having a summary highlighting the text essence to be committed to long-term memory. Specific visualizations during each of these stages can differ due to their utility, which is why we investigated the first stage with all four visualization types mentioned. Further studies will need to be conducted to get insights into their specific design, most effective content generation, and application cases. For now, we have been focusing on the pre-reading phase and
the use of such visualizations to prime readers. Based on our quantitative and qualitative findings we derive a set of implications in the following and introduce our vision for an automated system that takes text cues from a user's reading device and provides effective visualizations tailored to the reader to support the reading experience.

# 8 A System to Support Readers through Visualizations

In our study of visualizations, we found that prior viewing impacted the readers' ability to make out the gist of a text and in the case of highlighting reduced the actual reading time. Mind maps, on the other hand, were best fit to convey the structure and the essential content of a text.

In the following, we apply our findings into a vision of an automated system capable of providing such visualizations on the fly as users deal with their daily readings. Thereby, not only the text at hand needs to be considered, but also the reading goal. Visualizations, for example, can help readers disregard or engage with a text in more detail based on their pre-assessment. Further, as discussed by our focus groups, visualizations can serve different purposes based on whether they are being used before, during, or after reading. A system that takes a text as an input and creates visualizations on the fly can tailor to those requirements. Figure 4 shows an overview of the key components of such a system, which takes text from the reader's device as cues to generate visualizations while considering the user's reading goal and characteristics (such as background knowledge and reading habits). From our results, we derived the following important aspects that we propose to be considered while creating such a system.

**Extraction** An automated system for reader support through visualizations needs to comprise a representative corpus to ensure a reliable keyword extraction. The extraction itself and its quality are vital for the utility of the system, as we have learned from the participants' feedback who were not always able to connect the topics presented in some visualizations, especially in image collages, which mislead some readers. A reliable keyword extraction then informs the system which images to look for in order to enhance visualizations or build the corresponding image collages.

**Visual Clues** Providing visual clues about the semantic link between the keywords is crucial since we have learned that keywords on their own are not meaningful enough. Such links can be of structural nature, as the mind maps showed to be more effective when conveying text overview. Such links—also to externally related sources— can help readers understand content since providing context increases text comprehension [8].

**Image Selection** Images are appealing and can be useful to motivate readers to read a text, but bare little insights about the gist of a text and can even be misleading. For an automated system, there are two important aspects:
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** System pipeline for supporting in-situ text visualizations: a central instance collects available texts, processes them, links them to additional content, such as relevant images, and creates visualizations taking into account readers' preferences and the devices' attributes on which these visualizations are being rendered.</figcaption></figure>

1) filtering out meaningful keywords that describe a text in its essence, and 2) a comprehensive image database that contains a well-indexed pool of images to avoid ambiguities. In the study presented, we focused on the applicability of image collages as primes, hence we opted for manual image selection, but sophisticated tagging and visual processing algorithms might automate this process in the future.

*Address Information Overload* For pre-assessments of text, highlighting of most relevant text aspects is vital to convey the gist of a text effectively. Avoiding clutter is crucial since otherwise the visualization overloads the reader and loses its merit alongside the text. In our study, it became clear that mind maps conveyed a stronger structural overview as compared to word clouds, which were more likely to be perceived as cluttered. The hierarchical structure of mind maps helps to highlight important concepts where details can be pushed further down the branches.

*Text Structure* Conveying a text's structure helps readers orient themselves in the text and allows them to follow the string of argumentation. The mind maps used during our study conveyed that structure, which leads to increased overview of structure and content. The simple structure is given by text flow, but using semantics and language processing, a chain of argumentation could be extracted from text and displayed accordingly.

*User Characteristics* People are inherently different in reading aptitude, their interests, and background knowledge. A system that learns about user prefer-
ences regarding preferred topics, language, and reading device, could adapt to
these needs. Such preferences could be collected by explicit user settings or im-
plicitly derived by tracking and analyzing reading content and reading behavior
over time. Also, real-time assessments of user states regarding visual fatigue
and engagement can help adjusting visualizations to match users' cognitive ca-
pacities better [27], e.g., creating more complex visualizations when readers are
engaged vs. simplifying content when they are fatigued. Based on such inter-
nal user states, a dynamic reading interface could further be instructed by the
system to guide the reader quicker through the essential parts of a text [16].

Taking these aspects into account, we see great potential for visualizations
as text primes. Future work includes the integration of an automated system as
depicted in Figure 4, so that readers can get in-situ cognitive support as they
deal with their daily readings on electronic devices. By receiving visualizations
that break down the gist of a text into small, digestible chunks. Such visualiza-
tions can guide readers through the string of argumentation and support text
comprehension.

9 Conclusions

In this paper, we focused on the use of visualizations as primers to support read-
ing. In a comprehensive user study with 50 participate we investigated in what
way four types of text visualizations were useful to convey text content and struc-
ture before reading. Comparing word cloud, highlighting, mind map and image
collage we found that mind maps were rated best to support comprehension and
structure of a text. Highlighting, on the other hand, showed that readers who
study keywords in their context later require less time spent reading.

We conclude with a proposed design of a system that receives a text and
provides context-specific visualizations. Context includes text cues produced by
keyword extraction, the combination of different sources (e.g., image databases),
and taking into account specific goals and characteristics of the reader. This could
allow visualizations not only to serve as text summaries but further convey text
structure and provide enhancements through images. Our study showed that
readers were able to pre-assess the content of a text based on visualizations
—foremost through mind maps—, which allowed them to absorb the gist of a
text quickly. With the growing demands of our information society to keep up
with daily readings, visualizations can help readers making quick and informed
decisions about the utility and content of a text prior to reading.

10 Acknowledgments

We thank our study participants and acknowledge the financial support of the
Future and Emerging Technologies (FET) programme within the 7th Frame-
work Programme for Research of the European Commission, under FET grant
number: 612933 (RECALL).
References

1. Ahmadi, A.: Comprehension of a Non-Text: The Effect of the Title and Ambiguity Tolerance. Journal of Pan-Pacific Association of Applied Linguistics 15(1), 163–176 (Nov 2010)

2. Alencar, A.B., de Oliveira, M.C.F., Paulovich, F.V.: Seeing beyond reading: A survey on visual text analytics. Wiley Int. Rev. Data Min. and Knowl. Disc. 2(6), 476–492 (Nov 2012), http://dx.doi.org/10.1002/widm.1071

3. Aramini, S.A., Ardizzone, E., Mazzola, G.: Automatic illustration of short texts via web images. In: Proceedings of the 6th International Conference on Information Visualization Theory and Applications - Volume 1: IVAPP, (VISIGRAPP 2015). pp. 139–148. INSTICC, ScitePress (2015)

4. Baddeley, A., Eysenck, A., Anderson, M.: Memory. New York: Psychology Press (2009)

5. Banas, J.R., Brown, C.A.: Web 2.0 visualization tools to stimulate generative learning. pp. 77–90. IGI Global (2012)

6. Baralt, M., Pennestri, S., Selvandin, M.: Action Research: Using Wordles to Teach Foreign Language Writing. Language Learning & Technology 15(2), 12–22 (JUN 2011)

7. Basaraba, D., Yovanoğlu, P., Alonzo, J., Tindal, G.: Examining the structure of reading comprehension: do literal, inferential, and evaluative comprehension truly exist? Reading and Writing 26(3), 349–379 (2013)

8. Bransford, J.D., Johnson, M.K.: Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of verbal learning and verbal behavior 11(6), 717–726 (1972)

9. Buzan, T.: Make the most of your mind. Simon and Schuster (1984)

10. Caillies, S., Denhière, G., Kintsch, W.: The effect of prior knowledge on understanding from text: Evidence from primed recognition. European Journal of Cognitive Psychology 14(2), 267–286 (2002)

11. Carney, R., Levin, J.: Pictorial illustrations still improve students’ learning from text. Educational Psychology Review 14(1), 5–26 (MAR 2002)

12. Carr, N.: The shallows: What the Internet is doing to our brains. WW Norton & Company (2011)

13. Cashen, V.M., Leicht, K.L.: Role of the isolation effect in a formal educational setting. Journal of Educational Psychology 61(6p1), 484 (1970)

14. Collins, C., Carpendale, S., Penn, G.: Docuburst: Visualizing document content using language structure. In: Proceedings of the 11th Eurographics / IEEE - VGTC Conference on Visualization. pp. 1039–1046. EuroVis’09, The Eurographs Association; John Wiley & Sons, Ltd., Chichester, UK (2009), http://dx.doi.org/10.1111/j.1467-8659.2009.01439.x

15. Crouse, J.H., Idstein, P.: Effects of encoding cues on prose learning. Journal of Educational Psychology 63(4), 309 (1972)

16. Dingler, T., Shirazi, A.S., Kunze, K., Schmidt, A.: Assessment of stimuli for supporting speed reading on electronic devices. In: Proceedings of the 6th Augmented Human International Conference. pp. 117–124. AH ’15, ACM, New York, NY, USA (2015), http://doi.acm.org/10.1145/2735711.2735796

17. Federmeier, K.D., Kutas, M.: A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing. Journal of Memory and Language 41(4), 469–495 (Nov 1999), http://www.sciencedirect.com/science/article/pii/S0749596X99926608
18. Foster, J.J.: The use of visual cues in text pp. 189–203 (1979)

19. Fowler, R.L., Barker, A.S.: Effectiveness of highlighting for retention of text material. Journal of Applied Psychology 59(3), 358 (1974)

20. Gier, V.S., Herring, D., Hudnell, J., Montoya, J., Kreiner, D.S.: Active reading procedures for moderating the effects of poor highlighting. Reading Psychology 31(1), 69–81 (2010), http://dx.doi.org/10.1080/02702710903397082

21. Hearst, M.A.: Tilebars: Visualization of term distribution information in full text information access. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 59–66. CHI ’95, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA (1995), http://dx.doi.org/10.1145/223904.223912

22. Hotho, A., Nrnberger, A., Paas, G.: A brief survey of text mining. LDV Forum - GLDV Journal for Computational Linguistics and Language Technology (2005)

23. Kalmane, R.: Improving Reading Comprehension with Online Text Visualization Tools. Lulu Press, UK

24. Kintsch, W.: Text comprehension, memory, and learning. American Psychologist 49(4), 294–303. (1994)

25. Kucher, K., Kerren, A.: Text visualization techniques: Taxonomy, visual survey, and community insights. In: 2015 IEEE Pacific Visualization Symposium (PacificVis). pp. 117–121 (2015)

26. Kudelic, R., Konecki, M., Malekovic, M.: Mind map generator software model with text mining algorithm. In: Information Technology Interfaces (ITI), Proceedings of the ITI 2011 33rd International Conference on. pp. 487–494 (June 2011)

27. Kunze, K., Sanchez, S., Dingler, T., Augereau, O., Kise, K., Inami, M., Tsutomu, T.: The augmented narrative: Toward estimating reader engagement. In: Proceedings of the 6th Augmented Human International Conference. pp. 163–164. AH ’15, ACM, New York, NY, USA (2015), http://doi.acm.org/10.1145/2735711.2735814

28. Li, Z., Shi, S., Zhang, L.: Improving relevance judgment of web search results with image excerpts. In: Proceedings of the 17th International Conference on World Wide Web. pp. 21–30. WWW ’08, ACM, New York, NY, USA (2008), http://doi.acm.org/10.1145/1367497.1367501

29. Liu, Z.: Reading behavior in the digital environment: Changes in reading behavior over the past ten years. Journal of documentation 61(6), 700–712 (2005)

30. McNamara, D.S., Kintsch, W.: Learning from texts: Effects of prior knowledge and text coherence 22, 247–288 (1996)

31. Morineau, T., Blanche, C., Tobin, L., Guéguen, N.: The emergence of the contextual role of the e-book in cognitive processes through an ecological and functional analysis. International Journal of Human-Computer Studies 62(3), 329–348 (Mar 2005), http://www.sciencedirect.com/science/article/pii/S1071581904001089

32. Paley, W.B.: Textarc: Showing word frequency and distribution in text. In: Poster presented at IEEE Symposium on Information Visualization. vol. 2002 (2002)

33. Rajaram, S., Roediger, H.L.: Direct comparison of four implicit memory tests. Journal of Experimental Psychology: Learning, Memory, and Cognition 19(4), 765 (1993)

34. Ramos, J.: Using tf-idf to determine word relevance in document queries. In: Proceedings of the first instructional conference on machine learning (2003)

35. Ratcliff, R., McKoon, G.: A retrieval theory of priming in memory. Psychological review 95(3), 385 (1988)

36. Schmid, H.: Treetagger— a language independent part-of-speech tagger. Institut für Maschinelle Sprachverarbeitung, Universität Stuttgart 43, 28 (1995)
37. Silvers, V.L., Kreiner, D.S.: The effects of pre-existing inappropriate highlighting on reading comprehension. Literacy Research and Instruction 36(3), 217–223 (1997)

38. Stanovich, K.E., Cunningham, A.E.: What reading does for the mind. Journal of Direct Instructions 1(2), 137–149 (2001)

39. Strobelt, H., Oelke, D., Kwon, B.C., Schreck, T., Pfister, H.: Guidelines for effective usage of text highlighting techniques. IEEE Transactions on Visualization and Computer Graphics 22(1), 489–498 (Jan 2016)

40. Strobelt, H., Oelke, D., Rohrdantz, C., Stoffel, A., Keim, D., Deussen, O., et al.: Document cards: A top trumps visualization for documents. Visualization and Computer Graphics, IEEE Transactions on 15(6), 1145–1152 (2009)

41. Viégas, F.B., Wattenberg, M.: Timelines: Tag clouds and the case for vernacular visualization. interactions 15(4), 49–52 (Jul 2008), http://doi.acm.org/10.1145/1374489.1374501

42. Wise, J.A., Thomas, J.J., Pennock, K., Lantrip, D., Pottier, M., Schur, A., Crow, V.: Visualizing the non-visual: Spatial analysis and interaction with information from text documents. In: Proceedings of the 1995 IEEE Symposium on Information Visualization. pp. 51–. INFOVIS ’95, IEEE Computer Society, Washington, DC, USA (1995), http://dl.acm.org/citation.cfm?id=857186.857579

43. Zhang, F.: The application of visualization technology on knowledge management. In: Proceedings of the 2008 International Conference on Intelligent Computation Technology and Automation - Volume 02. pp. 767–771. ICICTA ’08, IEEE Computer Society, Washington, DC, USA (2008), http://dx.doi.org/10.1109/ICICTA.2008.479

44. Zhu, X., Goldberg, A.B., Eldawy, M., Dyer, C.R., Strock, B.: A text-to-picture synthesis system for augmenting communication. In: Proceedings of the 22Nd National Conference on Artificial Intelligence - Volume 2. pp. 1590–1595. AAAI’07, AAAI Press (2007), http://dl.acm.org/citation.cfm?id=1619797.1619900