SPATIAL COGNITION AND COMPUTATION, vol(issue), start–end
Copyright © YYYY, Lawrence Erlbaum Associates, Inc.

The Arrow Semantic Interpreter
Yohei Kurata1 and Max J. Egenhofer2
1
SFB/TR-8 Spatial Cognition, Universität Bremen
2
National Center for Geographic Information and Analysis
and Department of Spatial Information Science and Engineering,
University of Maine
People often sketch diagrams to facilitate their communication. If
computers understood the meaning of such diagrams, people could operate
information systems more intuitively by sketching. It remains achallenging problem, however, to make computers interpret arrow symbols,
which are a frequent and versatile ingredient of such diagrams. This paper
develops an algorithm for deducing possible semantic roles of arrowsymbols. This, algorithm, called the Arrow Semantic Interpreter (ASI),
emphasizes the elements around arrow symbols and their spatial
arrangement, which apparently have a strong influence on the semantics.
The semantic roles of arrow symbols are classified into orientation,
behavioral description, annotation, and association, and for each class therequirements on the arrangement of surrounding elements are identified.
By combining these requirements with the rules for adding optional
elements, we can deduce potential semantic roles of an arrow symbol fromthe pattern of surrounding elements. The assessment shows the ASI
deduces only 1.31 potential semantic roles per arrow symbol, but still they
include the correct answer for 79% of sample arrow symbols. This result
indicates that the pattern of surrounding elements is highly useful forderiving the interpretation of arrow symbols.
Keywords: Arrow Symbol, Diagram, Interpretation, Sketching Interface,
Individual Structure.
Correspondence concerning this article should be addressed to Yohei Kurata, SFB/TR-8
Spatial Cognition, Universität Bremen, Postfach 330440, 28334 Bremen, Germany;
email: ykurata@informatik.uni-bremen.de.

2

Kurata and Egenhofer

1 Introduction
People often sketch diagrams. Diagrams are often more intuitive than verbal
expressions, because we can directly express the spatial and structuralcharacteristics of the information in diagrams. If computers understood suchdiagrams, people could operate information systems more intuitively, forinstance, by sketching diagrams to explain our ideas and knowledge. Indeed, anumber of sketch-based computer systems have been developed, and theirusefulness has been reported repeatedly (Oviatt 1996; Egenhofer 1997; Landay
and Myers 2001; Davis 2002; Ferguson and Forbus 2002). These pioneeringsystems have demonstrated that computational diagram understanding is ahighly promising technology that will enrich human-computer interactions.
Arrow symbols are a frequent ingredient of such diagrams. One reason for thearrow symbols’ popularity is their versatility—even though their shapes are
extremely simple, they capture a large variety of semantics, such as directions,movements, orders, transitions, and relations. Arrow symbols themselves,
however, do not have any specific meaning—they provide the information about
the other elements to which the arrow symbols refer. This function of arrowsymbols is called their semantic role (Kurata and Egenhofer 2005b). Arrowsymbols may have a large variety of semantic roles, such as labeling, specifying
a direction, and illustrating a movement. Thus, in order to correctly understand adiagram with arrow symbols, its readers have to deduce the semantic roles ofeach arrow symbol. This is usually not difficult for adults, who have sufficientexperience in diagrammatic communication. For sketch-based systems, however,this remains a challenging problem and, accordingly, the current sketch-based
systems impose some restrictions on the use of arrow symbols (Section 3). This
paper, therefore, develops an algorithm for deducing the semantic roles of arrowsymbols. This algorithm is called the Arrow Semantic Interpreter (ASI). With acapability of deriving interpretations of arrow symbols, sketch-based
information systems will allow people to use arrow symbols as naturally as they
do in human communication.
The semantic role of an arrow symbol depends on what elements the arrowsymbol refers to, and also how the arrow symbol refers to those elements (e.g.,
attaching oneself to one element or linking two elements) (Kurata andEgenhofer 2005a). Therefore, the ASI emphasizes the influence of theseelements and their spatial arrangement around the arrow symbol. The
combination of arrow symbols with the elements to which the arrow symbols
refer is considered a syntactic unit, called an arrow diagram, and the elements towhich the arrow symbols refer are called the components of the arrow diagram(Kurata and Egenhofer 2005a). A component may be represented by an icon, atext label, or a specific position in a map or a background drawing. In order tosystematically study the influence of components and their spatial arrangement,

A Theory for Interpreting Arrow Symbols

3

this paper develops a model of components’ spatial arrangement and theirpatterns (Section 5).
In addition to the components and their arrangements, visual appearance ofarrow symbols, reader’s background knowledge, and context may also influence
the semantic roles of arrow symbols (Kurata and Egenhofer 2006a; Tversky et al.
2007). To simplify the discussion, however, this paper ignores these factors,
considering that these factors contribute to the judgment of the validity ofinterpretations, but not directly to the derivation of the interpretations.
The remainder of this paper is structured as follows: Section 2 reviews thestudies on arrow symbols. Section 3 reviews sketch-based systems, highlighting
how they deal with arrow symbols. Section 4 distinguishes four classes of arrowsymbols’ semantic roles. Section 5 develops a model of spatial arrangement ofcomponents around arrow symbols, which is called the individual structure.
Section 6 identifies the requirements on the individual structures when arrowsymbols are used for each class of semantic roles. Based on these requirements
and the rules for adding optional components (Section 7), we can determine
potential semantic roles of arrow symbols from the pattern of components’arrangement. This algorithm is explained in Section 8. Section 9 evaluates thedeveloped algorithm using sample arrow symbols. Finally, Section 10 concludes
with a discussion of future problems.

2 Studies on Arrow Symbols
Tversky (2001) defined an arrow symbol as a special kind of line, with one end
marked, inducing an asymmetry. This definition highlights two essential featuresof arrow symbols: linearity and asymmetry. With these two features, an arrowsymbol establishes an affordance (Gibson 1979) to prompt the diagram readers
to move their attention from the tail side to the head side of the arrow symbol.
Accordingly, if the arrow symbol connects two elements, these elements arenaturally ordered. Also, if the diagram is mapped onto a physical or conceptual
space, people naturally imagine a movement in this space. Thanks to thesecharacteristics, arrow symbols have a large variety of semantic roles.
Semantic diversity of arrow symbols has attracted researchers’ interest. Van
der Waarde and Westendorp (2000) identified seven usages of arrow symbols in
pictorial user instructions: moving direction, physical change or transformation,
indication of a dimension, labeling, focusing the attention, indication of asequence, and a part of designed symbols. Similarly, Blaser (2000) identified
four usages of arrow symbols in sketch maps: pointing north, indicating a pathdirection, indicating the direction of a view, and describing where a road leads
to. Horn (1998) collected various usages of arrow symbols, which are
schematized them in a tree graph. Kurata and Egenhofer (2005b) also collectedvarious semantic roles of arrow symbols, which are classified into four groups:
direction, action, annotation, and conjunction. In addition, Kurata andEgenhofer (2006b) analyzed the semantic roles assigned to a group of arrow

4

Kurata and Egenhofer

symbols, such as the illustration of meeting, separation, diversion, andconfluence. Westendorp (2006) discusses how the usages of arrow symbols has
diversified in history, pointing out that the widespread use of pictorial
instructions in modern days largely influence the diversification.
Another important characteristic of arrow symbols, which contributes to their
popularity, is that the use of arrow symbols enables the illustration of dynamic
information in a static diagram. Bertin (1983) claimed that arrow symbols are
the most efficient and often the only formula for illustrating a complex
movement. Monmonier (1990) insisted that arrow symbols are useful for
illustrating spatial diffusion of ideas, migrations of tribes and refugees, advancesof armies, and so forth. Tversky et al. (2000) confirmed by an experiment thatthe presence of arrow symbols encourage people to interpret a diagram from its
causal and functional aspect.
The polysemy of arrow symbols triggers the problem of disambiguation.
Tversky et al. (2007) insisted that carefully crafted context can disambiguatemeanings of depictive symbols, just as they can disambiguate meanings ofwords. Kurata and Egenhofer (2005b) demonstrated that possible semantic rolesof arrow symbols in simple diagrams can be derived from the pattern ofcomponents’ arrangement, although they left the development of aninterpretation algorithm for general arrow symbols for future work.

3 Arrow Symbols in Current Sketch-Based Systems
Over the last ten years, a variety of sketch-based systems have been developed,
aiming at more natural and effective human-computer interaction. For instance,
Spatial-Query-by-Sketch (Egenhofer 1997; Blaser and Egenhofer 2000) enables
its users to query spatial data by drawing a sketch map. ASSIST (Alvarado andDavis 2001a; 2001b; Davis 2002) interprets a mechanical sketch and predicts
how the illustrated mechanism would behave. Similarly, SketchIT (Stahovich
1997; Kurtoglu and Stahovich 2002) interprets mechanical sketches andrecreates new designs that realize the same functions. Landay and Myers (2001)
developed a computer-aided GUI-design support systems, which interprets a
hand-drawn screen layout and generates a prototype program.
While these systems were designed for specific tasks, GeoRep (Ferguson andForbus 2000) was developed as a common platform for sketch-based system.
GeoRep was applied to sKEA (Ferguson and Forbus 2002; Forbus and Usher
2002), which enabled its user to teach the computer his or her knowledge bysketching a diagram.
Some systems combined a sketching interface with a speech interface.
Egenhofer (1996) developed the framework of Sketch-and-Talk in GIS, in which
the user explain a place of interest by sketch and speech in order to query spatial
data. QuickSet (Cohen et al. 1997; Johnston 1998; Cohen et al. 2000; Oviatt andCohen 2000) is a multi-modal system for map-based tasks, which is operated byspeech and pen input. Similarly, nuSketch COA creator (Forbus et al. 2001;

A Theory for Interpreting Arrow Symbols

5

Ferguson and Forbus 2002) is a multi-modal system for a map-based task
(military action planning) based on GeoRep. Finally, ASSISTANCE (Davis
2002) is a multi-modal system that facilitates mechanical designs with sketch
and verbal input.
Our question is how these sketch-based systems deal with arrow symbols insketches. Some systems accept only one or a few semantic roles of allow
symbols. For instance, in the Landay and Myers’ (2001) computer-aided GUIdesign system, arrow symbols are primarily used for specifying button-window
relations—which window emerges or gets focus when a button is clicked. In
SketchIT, arrow symbols specify the movable direction of mechanicalcomponents. In ASSIST, the user may use one arrow symbol for specifying thegravity direction. In nuSketch COA creator, arrow symbols with different
semantic roles are distinguished by their different shapes. Such restriction ofarrow symbols to few semantic roles works effectively for specific tasks, since
the ambiguity of arrow symbols is excluded. As a drawback, the users of thesesystems are forced to get used to such restriction of arrow symbols.
QuickSet accepts arrow symbols with a variety of semantic roles, such as
specifying a direction, illustrating a route, and indicating relations. Furthermore,
in sKEA, arrow symbols may represent arbitrary binary relations. These systems,
however, have room for improvement, because the users have to specify thesemantic role of every arrow symbol by speech (in QuickSet), text input, or
menu selection (in sKEA). Such specification processes, which are rarely seen inhuman communication, reduce the smoothness of sketch-based interactions.
Overall, most sketch-based systems do not allow the natural use of arrowsymbols, due to the lack of a human-like ability to deduce the semantic roles ofarrow symbols. One exception is ASSISTANCE, which automaticallydistinguishes the arrow symbols representing causality and those representing
external force. Unfortunately, such distinction depends on the domain-specificrules, which cannot be applied directly to other sketch-based tasks.

4 Classification of Semantic Roles
Arrow symbols have a large variety of semantic roles. In addition, the semanticroles may be assigned to a group of arrow symbols, as well as individual arrowsymbols (Kurata and Egenhofer 2006b). To simplify the discussion, this paper
only considers the semantic roles assigned to individual arrow symbols andclassifies them into four groups (Kurata and Egenhofer 2005b). Figure 1
summarizes the process of the classification.
First, the semantic roles assigned to individual arrow symbols are
dichotomized into those that require only one component and those that requireat least two components. In the former group, the arrow symbol is attached tothe single component. Naturally, the arrow symbol specifies a directional
property of this component. This semantic role is called orientation.

6

Kurata and Egenhofer

The latter group, which requires two or more components, is further
dichotomized into behavioral description and others, depending on whether thearrow symbol reflects the transition of a certain entity or not. This transition
includes not only movement (spatial transition), but also continuous existence
possibly accompanied by a change (temporal transition). Therefore, illustrationsof spatial movement, interaction, duration, and change are all categorized into
behavioral description.
Arrow symbols with the remaining semantic roles simply connect thecomponents, without implying a transition. Among those arrow symbols, thosefor labeling are exceptional, because the connected components (usually a label
and an icon) refer to the same single subject, while the others associate (two)different subjects. Thus, their semantic roles are distinguished into annotation
and association.
Semantic Roles of Arrow Symbols
requires only one component

requires at least two components

Orientation
does not assume a transition

assumes a transition

Behavioral Description
features one subject

Annotation

associates multiple subjects

Association

Figure 1. Classification of semantics roles of arrow symbols.

5 Individual Structures of Arrow Symbols
To systematically analyze how the patterns of components’ arrangements
influence the semantic roles of arrow symbols, this section develops a model ofcomponents’ arrangements around each arrow symbol. This model is called theindividual structure associated with the arrow symbol.
When an arrow symbol explicitly refers to a component, this component islocated in front of, behind, or along the arrow symbols. This paper, therefore,
considers that an arrow symbol is a deictic reference frame (Retz-Schmidt 1988),
which identifies three different conceptual areas where the components can belocated (Figure 2). These three areas are called the component slots of an arrowsymbol, or the tail slot, body slot, and head slot, respectively (Kurata andEgenhofer 2005a). Every component, to which an arrow symbol explicitly refer,is assigned uniquely to one of the three component slots, thereby making thedistinction of tail components, body components, and head components.

A Theory for Interpreting Arrow Symbols
Tail slot

Body slot

7

Head slot

Figure 2. Three component slots associated with an arrow symbol.
The individual structure associated with an arrow symbol a (or simply called
a’s individual structure) is defined as a list of the components in a’s threecomponent slots. It is denoted by a 3-tuple, (Ctail(a), Cbody(a), Chead(a)), where
Ctail(a), Cbody(a), and Chead(a) are the non-ordered sets of tail, body, and headcomponents, respectively. For instance, the individual structure associated withthe arrow symbol in Figure 3 is ({“Mr. K”, traveler}, {“July 4”}, {firework
show, “Boston”}).
Mr. K
July 4
BostonFigure 3. An arrow diagram, which illustrates that a traveler, whose name is Mr.
K, goes to a firework show in Boston on July 4th.
The components are dichotomized into primary components (PCs) andmodifier components (MCs). The primary component represents an independent
entity or concept, while the modifier component modifies something else. For
instance, in Figure 3, the traveler icon and firework icon are primary
components that represent a traveler and a firework show, while the labels “Mr.
K,” “July 4,” and “Boston” are modifier components, which modify the traveler
icon, the movement represented by the arrow symbol, and the firework icon,
respectively. On the analogy of natural languages, an arrow symbol plays therole of a verb, each primary components corresponds to a subject or an object,and each modifier component correspond to an adjective or an adverb. The
distinction of primary and modifier components is purely conceptual. There are,however, the following conventions and rules, which are useful for the
distinction of primary and modifier components in a visual domain:⋅ Icons are usually primary components;⋅ Text labels attached to icons are usually modifier components⋅ If an arrow symbol refers to only one component through three slots, thiscomponent is a primary component, which is the subject in orientation(Section 6); and8⋅Kurata and Egenhofer
If the head slot has only one component, this component is a primary
component, because the modifier component cannot be used in the head slot
(Section 7).

The primary components are further categorized into the following four
subclasses:⋅ A location (PCL) is a position in space (e.g., “Boston” in Figure 4a). A
modifier component may also represent a position in space (e.g., “Boston” inFigure 3), but it is not included in the location.
⋅ A moment (PCM) is a position in time (e.g., “9:00pm” and “10:00pm” inFigure 4b). A modifier component may also represent a position in time (e.g.,“July 4” in Figure 3), but it is not included in the location.
⋅ An object (PCO) is an entity or its unit, which exists in a physical orconceptual space, and takes an action or gets manipulated (e.g., a vehicle in
Figure 4a). Objects are continuants, which endure through some extended
interval of time (Worboys and Hornsby 2004).
⋅ An event (PCE) occurs in time, at an instant or over an interval (e.g., afirework show in Figure 4b). It is characterized by a set of changes that theevent triggers. Events are occurents, which happen and are then gone
(Worboys and Hornsby 2004).

Boston(a)

9:00pm

10:00pm(b)

Figure 4. Two arrow diagrams, which contains (a) a location (“Boston”) and (b)two moments (“9:00pm” and “10:00pm”).
With the distinction of primary and modifier components (PC and MC), andfurther distinction of four subclasses of the primary components (PCL, PCM,
PCO, and PCE), the pattern of the individual structure is defined as follows:
pattern_of_individual_structure ::= “(” tail_components“,” body_components “,”head_components “)”
tail_components ::= [components]
body_components ::= [components]
head_components ::= [components]
components::= component [components]
component::= PCL|PCM|PCO|PCE|MCFor instance, the individual structures of arrow symbols in Figures 3, 4a, and 4b
have the patterns of (MC PCO, MC, PCE MC), (PCO, –, PCL) and (PCM, PCE,
PCM), respectively.

9

A Theory for Interpreting Arrow Symbols

6 Formats of Individual Structures
When using an arrow symbol, we have to arrange the components following
certain rules. These rules determine the format of the individual structureassociated with the arrow symbol. This section identifies the sets of suchformats for the four classes of semantic roles.

Orientation
An arrow symbol for orientation specifies a certain directional property of acomponent, called the subject. The arrow symbol points to, originates from, or
passes through/by the subject, typically implying that the directional property isrelated to an outgoing action, a passing action, or an incoming action,
respectively (Figure 5). Accordingly, orientation corresponds to the threeformats in Figure 6.

(a)

(b)

(c)

Figure 5. Diagrams with an arrow symbol for orientation, specifying (a) avehicle’s moving direction, (b) a wind direction at a geographic location, and (c)the direction of an external force by which a board cracks.
s
s

s

(a)

(b)

(c)

Figure 6. Formats of individual structures for orientation (s: subject).
Obviously, the subject must be a primary component, which represents an
independent entity or concept. Among the four subclasses of primary
components (i.e., location, moment, object, and event), a moment cannot be thesubject, since the moment is a zero-dimensional concept and has no directional
property. On the other hand, the arrow diagrams in Figure 5, whose subjects area vehicle, a geographic location, and a board-cracking event, indicate that the
subject may be an object, a location, and an event, respectively. Consequently,
when an arrow symbol is used for orientation, its individual structure satisfies
one of the basic patterns in Table 1. In these basic patterns, each blank indicatesthat the corresponding component slot may be empty or filled by optional
components. PCL|O|E represents PCL, PCO, or PCE. For instance, the pattern (PCL
MC, MC, –) satisfies the basic pattern (PCL|O|E, , ).

10

Kurata and Egenhofer
Table 1. Basic patterns of individual structures for orientation (s: subject).
Format Basic patterns(s, , ) (PCL|O|E, , )( , s, ) ( , PCL|O|E, )( , , s) ( , , PCL|O|E)

Behavioral description
An arrow symbol for behavioral description illustrates the transition of acomponent, called the subject. In addition to the subject, the arrow symbol may
refer to the components, which specify spatial or temporal positions. The tail,body, and head components, if each represents a spatial position, specify theorigin, the intermediate point, and the destination of the transition, respectively(Figure 4a). Similarly, the tail, body, and head components, if each represents a
temporal position, specify the start time, the intermediate time, and end time of
the transition, respectively (Figure 4b). In addition, the arrow symbol may also
refer to the components, with which the subject interacts during the transition(Figures 7a-b). These components are called the participants. If the participants
are located in the tail, body, and head slots, the interaction takes place before,
during, and after the transition, respectively.

(a)

(b)

(c)

Figure 7. Diagrams with an arrow symbol for behavioral description, illustrating(a) installation of a wheel, (b) removal of a wheel, (c) change of a vehicle. In (ac) the subject is the vehicle, and in (a-b) the participant is the wheel.
When an arrow symbol is used for behavioral description, its individual
structure must satisfy the following constraints:⋅ The subject s is located in any component slot (Figures 7a-b), except when
the diagram highlights the change of the subject (Figure 7c).
⋅ In addition to the subject s, the arrow symbol refers to at least one transitionrelated spatial/temporal position po or one participant pa; otherwise the arrowsymbol refers to the subject alone and, accordingly, the semantic role of thisarrow symbol becomes orientation instead of behavioral description.
⋅ The participant pa cannot be located at the same place with the subject s;otherwise the diagram no longer implies the interaction between s and pa. In
addition, we assume that multiple participants cannot be located at the sameplace (if apparently multiple participants are located at the same place, they

11

A Theory for Interpreting Arrow Symbols

are considered a single participant as a group). Accordingly, pa cannot belocated in the tail or head slot that already contains s or another pa.
⋅ The transition-related position po cannot be located at the same location withthe subject s, the participant pa, or another transition-related position;otherwise, po is regarded an adjective component (Section 7). Accordingly, pcannot be located in the tail or head slot that already contains s, pa, or
another po.
⋅ The body slot may contain the subject s, one or more participant pa, and one
or more transition-related positions pa at the same time, because it has length.
These constraints determine the formats of individual structures for behavioral
description (Table 2).
Table 2. Formats of individual structure for behavioral description (s: subject,
pa: participant, po: transition-related position). [X]n represents one or more X.
Formats
in
Head
Slot
in
Tail
Slot
SingleSubject[Pa|Po]n

s

Pa|Po

Pa|Po

s

Pa|Po

s

e|p

s

Pa|Po

Pa|Po

Pa|Po

Pa|Po

s

s

s

s

[Pa|Po]n

s
Pa|Po

s

[Pa|Po]n

Pa|Pos

Two Subjectss[Pa|Po]n

Pa|Po

in
Body
Slot

[Pa|Po]ns[Pa|Po]n
Pa|Po[Pa|Po]n[Pa|Po]n

s

s

s

s

The subject s, each participant pa, and transition-related position po must be a
primary component, as they are independent existence. Among four subclasses
of primary components, either an object or an event can be the subject s, since
objects and events may change their spatial or temporal positions, but locations
and moments cannot. Each participant pa must be either an object, an event, or alocation can be pa, because they can be a target of action, but a moment cannot.
Each transition-related position po is obviously either a location or a moment.

12

Kurata and Egenhofer

Consequently, when an arrow symbol is used for behavioral description, its
individual structure must satisfy one of the basic patterns in Table 3.

in Tail Slot in Head Slot

Two
Subjects

in Body Slot

Single Subject

Table 3. Basic patterns of individual structures for behavioral description (s:subject, pa: participant, po: transition-related position).
Format(s, , pa|po)(s, [pa|po]n, )(s, [pa|po]n, pa|po)(pa|po, , s)
( , [pa|po]n, s)(pa|po, [pa|po]n, s)(pa|po, s, )( , s, pa|po)(pa|po, s, pa|po)(pa|po, s[pa|po]n, )( , s[pa|po]n, pa|po)(pa|po, s[pa|po]n, pa|po)( , s[pa|po]n, )(s, , s)(s, [pa|po]n, s)

Basic patterns(PCO|E, , PCL|M|O|E)(PCO|E, [PCL|M|O|E]n, )(PCO|E, [PCL|M|O|E]n, PCL|M|O|E)(PCL|M|O|E, , PCO|E)( , [PCL|M|O|E]n, PCO|E)(PCL|M|O|E [PCL|M|O|E]n, PCO|E)(PCL|M|O|E, PCO|E, )( , PCO|E, PCL|M|O|E)(PCL|M|O|E, PCO|E, PCL|M|O|E)(PCL|M|O|E, PCO|E[PCL|M|O|E]n, )( , PCO|E[PCL|M|O|E]n, PCL|M|O|E)(PCL|M|O|E, PCO|E[PCL|M|O|E]n, PCL|M|O|E)( , PCO|E[PCL|M|O|E]n, )(PCO|E, , PCO|E)(PCO|E, [PCL|M|O|E]n, PCO|E)

Annotation
An arrow symbol for annotation attaches a text component (label) to anothercomponent (subject), thereby specifying the subject’s name, category, status,
spatial position, temporal position, and so on (Figure 8). Conventionally,annotation corresponds to the unique format in Figure 9, where an arrow symbol
originates from the label and points to the subject. This format implies that thelabel is assigned to the subject.
Mr. K

airporttravelergoing to Boston

9:00pmFigure 8. An arrow diagram with five arrow symbols, all used for annotation.

A Theory for Interpreting Arrow Symbols

l

13

s

Figure 9. The unique format of individual structures for annotation (l: label, s:
subject).
The label, which modifies the subject, is naturally a modifier component,
while the subject is a primary component of any subcategory. Consequently,
when an arrow symbol is used for annotation, its individual structure mustsatisfy the basic pattern in Table 4.
Table 4. The basic pattern of individual structures for annotation (l: label, s:
subject).
Format(l, , s)Basic pattern(MC, , PCL|M|O|E)

Association
An arrow symbol for association associates two components (subjects),
indicating the presence of an asymmetric relation between them. Conventionally,
these two subjects are placed in the tail and head slots (Figure 10), such thatthese two subjects look equally emphasized while their order is highlighted.

s1

s2

Figure 10. The unique format of individual structures format for association (s1,s2: associated subjects)
The asymmetric relation, which holds between the two subjects, is called theeffective relation. The caption, legend, or adverbial component (Section 7) may
specify the effective relation; otherwise, the diagram reader has to infer theeffective relation from the context or their own knowledge about typicalrelations between the subjects. The effective relation may provide an ordering
rationale, which determines the order of the associated subjects (Table 5).

14

Kurata and Egenhofer

Table 5. Examples of association by arrow symbols, where the effective
relations between the subjects may naturally determine the subjects’ order.
ExampleAssociatedSubjects

Effective
Relation

Ordering
Rationale

El NiñoÆFish Catch↓

El Niño,Fish Catch ↓Causality

logical orderPlanÆDoÆSee

Plan, Do, See

workprocess

temporal order

Niagara FallsÆLake Ontario

Niagara Falls,
Lake Ontario

water flow

spatial order(high to low)

LobsterÆMaine
MaineÆLobster

Lobster,
Maine

local
product–

Each subject must be a primary component, which represents an independent
entity or concept. Any subcategory of primary components can be the subject, aslong as people can identify an effective relation between the subjects.
Accordingly, when an arrow symbol is used for association, its individual
structure must satisfy the basic pattern in Table 6.
Table 6. The basic pattern of individual structures for association (s1, s2:
associated subjects).
Format(s1, , s2)

Basic patterns(PCL|M|O|E, , PCL|M|O|E)

7 Optional Components
Individual structures of arrow symbols may have optional components, which
enrich the diagram’s semantics. Two types of optional components are
distinguished: adjective components and adverbial components. An adjective
component describes a character of its nearby component, such as name, spatial
position, and temporal position (Figure 11). Meanwhile, an adverbial component
provides the following information about the semantic role of the arrow symbol:⋅ When an arrow symbol is used for orientation, its adverbial components
describe the type, name, and scale of the illustrated directional property(Table 7a1-a3).
⋅ When an arrow symbol is used for behavioral description, its adverbial
components describe the type and scale, and when and where the transition
takes place (Table 7b1-b3).

A Theory for Interpreting Arrow Symbols

15⋅

When an arrow symbol is used for association, its adverbial components
describe the effective relation that associates the components and when andwhere the relation holds (Table 7c1-c3).
When an arrow symbol is used for annotation, the arrow symbol does not referto adverbial components. On the analogy of natural languages, adjective
components and adverbial components correspond to adjective and adverbial
phrases, which modify a noun and a verb, respectively.
Naturally, both adjective and adverbial components belong to modifiercomponents. Adjective components are placed in the same slot with themodified components. Adverbial components are placed in the body slot,
normally around its center, implying that the adverbial component is assigned tothe entire arrow symbol.

Mr. K

Maine

Figure 11. An arrow diagram with two adjective components, “Mr. K” and“Maine,” which specify the name of the nearby components.
Table 7. Information provided by adverbial components.

Semantic Role
Behavioral
Association
description

Orientation

Provided Information
external force(a1) property type(a2) property nameinstall(b1) transition type

belong to

10 mph(b2) transition scale

2006
Germany

W-cup(c1) effective
relation

50 kgm/s2

f1

tourist ↑(c2) when and where
the relation holds(a3) property scalenoon(b3) when and where thetransition takes place

16

Kurata and Egenhofer

8 Deriving Interpretations from Individual Structures
Let us assume that we already know the individual structure of an arrow symbol
a. By combining the formats of individual structures (Section 6) and the rules ofadding optional components (Section 7), we can parse a’s individual structureand determine a’s all potential semantic roles among orientation, behavioral
description, annotation, and association.
Let a’s individual structure be sind (a ) = C tail ij , C body ij , C head ij , where C tail (a ) ,()

Cbody (a ) , and Chead (a ) represent the sets of tail, body, and head components,

respectively. Let σ1-σ4 are orientation, behavioral description, annotation, andassociation, respectively. The arrow symbol a nay have a semantic role σi only
if sind (a ) satisfies one of the σi’s formats and every extra component in sind (a ) is
considered as either an adverbial component or an adjective component. LetthT ij , B ij , H ij be the j basic pattern of the semantic role σi (Tables 1, 3, 4, and 6).
Assume we have a function ct (c ) that gives the type of a component c among
PCL, PCM, PCO, PCE, and MC. With this setting, a’s all potential semantic roles
are deduced by the following algorithm:

(

)

1:

result Å { }2:(T , B, H ) Å ({ct (c ) c ∈ Ctail (a )}, {ct (c ) c ∈ Cbody (a )}, {ct (c ) c ∈ Chead (a )})3:

For every semantic role class σ i

4:
5:

(
) thenFor σ i ’s every basic pattern T ij , B ij , H ij

(

) (

) (

If T ij ⊂ T ∧ B ij ⊂ B ∧ H ij ⊂ H)

7:(T , B , H ) Å (T \ T , B \ B , H \ H )(T , B , H ) Å (T , B , H )8:IF σ i ≠“Annotation” then remove all MC from B *9:10:

S = T * ∪ B* ∪ H *If ¬[{PC L ∈ S } ∨ {PCM ∈ S } ∨ {PCO ∈ S } ∨ {PC E ∈ S }] and _

6:

+

+

+

*

*

*

[{(

ij

+

) (

+

ij

ij

+

)} {(

)}
) (
∨ {(H ≠ φ ) ∧ (H = φ )}] _¬ T * ≠ φ ∧ T ij = φ ∨ B * ≠ φ ∧ B ij = φ*ijthen add σ i to result
11:
End If
12:
Next
13: Next

This algorithm is called the ASI (Arrow Semantic Interpreter). (T , B, H )shows the pattern of a’s individual structure sind (a ) . (T + , B + , H + ) shows theextra components in sind (a ) under the assumption that (T , B, H ) satisfies theA Theory for Interpreting Arrow Symbols

(

17

)

basic pattern T ij , B ij , H ij . If this assumption is correct, every component in(T + , B + , H + ) must be either an adverbial component or an adjective component,
and also every component in (T * , B * , H * ) must be an adjective component afterline 8. Since each adjective component must be a modifier component, T*, B*,
and H* cannot include a primary component PCL, PCM, PCO, or PCE (line 10). In
addition, if T*, B*, and H* contain an adjective component, there must be at least
one component in T ij , B ij , and H ij that the adjective component modifies,
respectively (line 10). If these two conditions are satisfied, the semantic role σiis recorded as a’s potential semantic role. After checking every basic pattern offour classes of semantic roles, the set variable result shows a’s all potential
semantic roles.

9 Evaluation
To evaluate the developed algorithm (i.e., the ASI), we developed a prototype
program, which deduces all potential semantic roles of an arrow symbol fromthe pattern of its individual structure (Figure 12). At this stage, the user has tospecify the pattern of the individual structure. The automation of this process isa subject for future work.

Figure 12. A screenshot of the ASI’s prototype.
Sample arrow symbols for the assessment were collected from an
introductory GIS textbook, “Geographical Information Systems and Computer
Cartography” (Jones 1997) because this material satisfies the following
conditions:⋅ the material contains a sufficient number of figures that contain arrowsymbols;⋅ the material is supposed to be read by people without special education ortraining in diagram reading;⋅ the figures are neither drawn by few designers nor adhere to in-houseaesthetic standards (this is why newspapers and magazines are avoided); and⋅ the semantic roles of sample arrow symbols are not biased (Figure 14a);18

Kurata and Egenhofer

We also examined two more textbooks in biology and astronomy, but thesemantic roles of arrow symbols in these two textbooks are considerably biased;
the biology textbook predominantly uses arrow symbols for illustrating chemical
reactions or organism’s movement (both belong to behavioral description),while the astronomy textbook frequently uses arrow symbols for illustrating aninterval, which cannot be categorized into the four classes of semantic roles thatthe current ASI supports.
The correct semantic roles of sample arrow symbols were assigned manuallybased on the figures plus context, sometimes drawn from the caption and the
body text. We confirmed that the assignment of the correct semantic roles to thearrow symbols correspond to the result of votes by human subjects.
From the GIS textbook, 304 arrow symbols in 64 figures were collected.
Some figures contain a large number of similar arrow symbols (Figure 13).
These similar arrow symbols, if counted individually, may distort the statistic
result. Thus, for every set of similar arrow symbols in each figure (i.e., arrowsymbols with the same semantic role and the same pattern of individual
structure) one representative is selected. Finally, 94 arrow symbols wereprepared for the assessment. This selection process did not bias the proportion of
the arrow symbols that belong to the four classes of semantic roles (Figure 14).

Figure 13. An arrow-containing diagram that contains a large number of similararrow symbols.
Other
5%

Orientation
25%

Association
22%

Other
10%

Orientation
27%

Association
12%

Annotation
22%
Annotation
21%

Behavioral
description
29%

Behavioral description
27%

(a)

(b)

Figure 14. Semantic roles of (a) 304 arrow symbols found in the textbook and(b) 94 arrow symbols selected from the 304 arrow symbols for the assessment.

A Theory for Interpreting Arrow Symbols

19

The evaluation started with counting the number of potential semantic roles
that the ASI deduced for each arrow symbol. Then, we counted the number ofsample arrow symbols whose interpretation yields:⋅ exact match, where the ASI deduced only one potential semantic role, whichmatches the correct semantic role;⋅ partial match, where the ASI deduced multiple potential semantic roles, oneof which matches the correct semantic role;⋅ oversight, where the potential semantic role(s) deduced by the ASI do notinclude the correct semantic role, even though the correct one is either
orientation, behavioral description, annotation, or association; and⋅ no-answer, where the potential semantic role (s) deduced by the ASI do notinclude the correct semantic role, simply because the correct one is notamong the four classes of semantic roles.
Finally, we calculated the detection rate (i.e., the proportion of sample arrowsymbols for which the ASI deduced potential semantic role(s) that includes thecorrect semantic role) and compared it with the detection rate under random
choices.

ResultFigure 15 shows the number of potential semantic roles that the ASI deduced forthe 94 arrow symbols. In most cases the ASI deduced one or two potential
semantic roles. This result indicates that ASI certainly reduces the ambiguity ofarrow symbols, since there are initially four choices. For 6% of the sample
arrow symbols, however, the ASI failed the deduction, due to the use of irregularformats or formats that belong to unsupported semantic roles. On average, 1.31
potential semantic roles were deduced per arrow symbol.
Nointerpretation
6%
Twointerpretatons37%

One
interpretaton
57%

Figure 15. The number of semantic roles that the ASI deduced for the 94 arrowsymbols.
Figure 16 shows the proportion of the four types of interpretation results. For
44% + 35% = 79% of the sample arrow symbols, the ASI successfully detected
the correct semantic role. Especially, 44% of the sample arrow symbols yielded
an exact match, which requires no further process for narrowing down potential
semantic roles. On the other hand, for 11% + 10% = 21% of the sample arrow

20

Kurata and Egenhofer

symbols, the ASI failed to detect the correct semantic role. Especially, 10% of
the sample arrow symbols were unable to be interpreted because their correct
semantic roles were unsupported.
No-answer
10%
Oversight
11%
Exact Match
44%

Partial Match
35%

Figure 16. The proportion of the four types of interpretation results for the 94
arrow symbols.
Assume a computer randomly selects zero, one, or two semantic roles fromthe four classes at the probability of 6%, 57%, and 37%, respectively (Figure 15).
Then, the expected detection rate (the probability that the randomly-selected
potential semantic role(s) include the correct one) is 30% with respect to the 94
sample arrow symbols. On the other hand, the ASI’s detection rate is 79%,
which is much higher than the expected detection rate under random selection.
This result indicates that the ASI’s interpretation is highly reliable.

Analysis of Misinterpretations
To identify the reasons for the misinterpretations and the solutions, we analyzed
the sample arrow symbols whose interpretation yields oversight, no-answer, or
partial match.
Oversight: 11 Cases
In our samples, oversight was triggered by the omission of components (Figure17a) or the use of unsupported formats (Figure 17b). The omission ofcomponents occurs when they are obvious from the caption, legend, body text,
or context. Accordingly, the use of these information sources is a key for thesolution.

(a)Figure 17. Examples of arrow symbols yielding oversight.

(b)A Theory for Interpreting Arrow Symbols

21

No-Answer: 9 cases
Through the analysis of nine no-answer cases, the following semantic roles ofarrow symbols are newly identified:⋅ to illustrate an interval (Figure 18a);⋅ to highlight a certain point in the space (Figure 18b); and⋅ to imply a gradation (Figure 18c)
The first semantic role is assigned to a pair of arrow symbols, instead ofindividual arrow symbols. Such group-oriented semantic roles of arrow symbols
are related to the formations of the arrow symbols (Kurata and Egenhofer
2006b). It is, therefore, a future problem to develop an interpretation algorithm
that considers the formation of arrow symbols in addition to the individual
structures. The second semantic role can be regarded as a special case ofannotation where the label is omitted, because of its obvious message (forinstance, look here). The third semantic role is probably not as popular as theprevious two.

(a)

(b)

(c)

Figure 18. Examples of arrow symbols yielding no-anwer. Arrow symbols areused for (a) illustrating an interval, (b) highlighting, and (c) implying a gradation.
Partial Match: 33 Cases
Table 8 summarizes the patterns of the individual structures of 33 arrowsymbols that has yielded partial match. This table shows that all partial matchcases has stemmed from the overlap of the patterns that correspond tobehavioral description and the patterns that correspond to association. Amongsuch patterns, (PCO[MC]*, [MC]*, PCO[MC]*) is frequently observed (24 out of
33 cases). This pattern is used for three scenarios:⋅ To illustrate the change of a subject;⋅ To illustrate an interaction between a subject and a participant; and⋅ To associate two different subjects.
The first two belong to behavioral description, while the last one belongs to
association. The following clues may be useful for the distinction of these three
scenarios:

22

Kurata and Egenhofer⋅

If the arrow symbol has non-simple shape, an arrow symbol is usually used
for behavioral description (in this case, the second scenario), because thearrow symbol’s shape often implies the course of spatial transition.
⋅ If two PCO s cannot be considered two different states of the same subject, anarrow symbol may not illustrate a change.
⋅ If two PCO are immovable, an arrow symbol may not illustrate an interaction.
Thus, the use of arrow’s visual information and background knowledge about
components’ characteristics becomes an issue. Similar ideas can be applied forthe distinction of behavioral description and association in other cases.
Table 8. Patterns of the individual structures of the 33 arrow symbols whichhave yielded partial match.
Pattern of
Individual Structure(PCO[MC]*, [MC]*, PCO[MC]*)(PCO[MC]*, [MC]*, PCE[MC]*)(PCO[MC]*, [MC]*, PCO[MC]*)(PCE[MC]*, [MC]*, PCL[MC]*)(PCO[MC]*, [MC]*, PCO[MC]*)(PCO[MC]*, [MC]*, PCL[MC]*)

Deduced Semantic Roles
Correct
Unnecessary

behavioral
description

association

association

behavioral
description

Num. ofSamples
17
3
3
2
7
1

10 Conclusions and Future Work
This paper developed an algorithm for deducing potential semantic roles ofarrow symbols. This algorithm makes use of the requirements on components’arrangement, which are specific to each class of semantic roles. The assessment
showed that the average number of potential semantic roles deduced by thisalgorithm is only 1.31, but still they include the correct answer for 79% of
sample arrow symbols. This result indicated that the pattern of components’arrangement around each arrow symbol is highly useful information for deriving
the interpretation of the arrow symbol. Through the analysis of misinterpreted
samples, we identified the following challenges for the improvement:⋅ Identification of omitted components using captions, legends, and so on;⋅ Deduction of semantic roles assigned to a group of arrow symbols;⋅ Judgment of valid semantic role making use of arrow’s visual information
and background knowledge; and⋅ Support of additional semantic roles and formats, if necessary.
Another potential challenge is to furnish details to the interpretations. The
current algorithm only distinguishes four classes of semantic roles, which might
be too coarse for some applications.

A Theory for Interpreting Arrow Symbols

23

Our future goal is to apply the developed algorithm for sketch-based systems
in a practical way, such that their users can use arrow symbols freely andnaturally in their sketch-based interaction. To achieve this goal, the following
techniques are necessary.
⋅ Symbol and text recognition techniques to detect arrow symbols andcomponents in raw sketches.
⋅ Automated linking between arrow symbols and components (i.e., judging
whether each arrow symbol refers to each component and, if so, in whichslot), probably based on their distance and size.
⋅ Automated distinction of component types. Some diagrammatic conventions
help to make the distinction between primary and modifier components(Section 5), but further distinction of their subclasses requires certain
background knowledge about the components’ characteristics.
The development of these techniques is left for future research.
Arrow symbols are frequently used in people’s everyday communication.
Especially, they are essential for the communication of spatial and spatiotemporal information. We, therefore, believe that studies on arrow symbols
contribute to the improvement of user interfaces in spatial information systems,
as well as the development of innovative application systems.

Acknowledgment
We would like to appreciate Kate Beard, Mike Worboys, Kathleen Hornsby,
Werner Kuhn, and Ronald Ferguson for their advice. This work was partially
supported by the National Geospatial-Intelligence Agency under grant numbers
NMA201-01-1-2003 and NMA401-02-1-2009.

References
Alvarado, C. and Davis, R. (2001a) Preserving the Freedom of Paper in a
Computer-Based Sketch Tool. in: Smith, M., Salvendy, G., Koubek, R. andHarris, D. (eds.) HCI International, New Orleans, LA, 687-691, Lawrence
Erlbaum Associates.
Alvarado, C. and Davis, R. (2001b) Resolving Ambiguities to Create a Natural
Sketch Based Interface. in: Nebel, B. (ed.) 17th International Joint Conferenceon Artificial Intelligence (IJCAI-01), Seattle, WA, 1365-1374, Morgan
Kaufmann.
Bertin, J. (1983) Semiology of Graphics: Diagrams, Networks, Maps. Madison,
WI, University of Wisconsin Press.
Blaser, A. (2000) A Study of People's Sketching Habits in GIS. Spatial
Cognition and Computation 2(4): 393-419.
Blaser, A. and Egenhofer, M. (2000) A Visual Tool for Querying Geographic
Databases. in: Gesù, V., Levialdi, S. and Tarantini, L. (eds.) Advanced VisualInterfaces, Salerno, Italy, 211-216.

24

Kurata and Egenhofer

Cohen, P., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Chen, L. and Clow,
J. (1997) Quickset: Multimodal Interaction for Distributed Applications. in: 5th
ACM international conference on Multimedia, Seattle, WA, 31-40, ACM Press.
Cohen, P., McGee, D. and Clow, J. (2000) The Efficiency of MultimodalInteraction for a Map-Based Task. in: 6th Applied Natural Language Processing
Conference (ANLP2000), Seattle, WA, 331-338, Morgan Kaufmann.
Davis, R. (2002) Sketch Understanding in Design: Overview of Work at the Mit
Ai Lab. in: Davis, R., Landay, J. and Stahovich, T. (eds.) AAAI Spring
Symposium on Sketch Understanding, Stanford, CA, 24-31, AAAI Press.
Egenhofer, M. (1996) Multi-Modal Spatial Querying. in: Kraak, M. J. andMolenaar, M. (eds.) 7th International Symposium on Spatial Data Handling,
Delft, Netherlands, 785-799, Talyor & Francis.
Egenhofer, M. (1997) Query Processing in Spatial-Query-by-Sketch. Journal of
Visual Languages and Computing 8(4): 403-424.
Ferguson, R. and Forbus, K. (2000) Georep: A Flexible Tool for Spatial
Representation of Line Drawings. in: 17th National Conference on Artificial
Intelligence, Austin, TX, 510-516, AAAI Press.
Ferguson, R. and Forbus, K. (2002) A Cognitive Approach to Sketch
Understanding. in: Davis, R., Landay, J. and Stahovich, T. (eds.) AAAI Spring
Symposium on Sketch Understanding, Stanford, CA, 67-72, AAAI Press.
Forbus, K., Ferguson, R. and Usher, J. (2001) Towards a Computational Modelof Sketching. in: 6th International Conference on Intelligent User Interfaces,
Santa Fe, NM, 77-83, ACM Press.
Forbus, K. and Usher, J. (2002) Sketching for Knowledge Capture: A Progress
Report. in: 7th International Conference on Intelligent User Interfaces, San
Francisco, CA, 71-77, ACM Press.
Gibson, J. (1979) The Ecological Approach to Visual Perception. Boston,
Houghton Mifflin.
Horn, R. (1998) Visual Language: Global Communication for the 21st Century.
Bainbridge Island, WA, MacroVu, Inc.
Johnston, M. (1998) Multimodal Language Processing. in: Mannell, R. andRobert-Ribes, J. (eds.) 5th International Conference on Spoken Language
Processing, Sydney, Australia, 2343-2346, Australian Speech Science andTechnology Association.
Jones, C. (1997) Geographical Information Systems and Computer
Cartography. Harlow, UK, Longman.
Kurata, Y. and Egenhofer, M. (2005a) Semantics of Simple Arrow Diagrams. in:Barkowsky, T., Freksa, C., Hegarty, M. and Lowe, R. (eds.) AAAI Spring
Symposium on Reasoning with Mental and External Diagram: Computational
Modeling and Spatial Assistance, Stanford, CA, 101-104, AAAI Press.
Kurata, Y. and Egenhofer, M. (2005b) Structure and Semantics of Arrow
Diagrams. in: Cohn, A. and Mark, D. (eds.) COSIT '05, Ellicottville, NY,
Lecture Notes in Computer Science 3693, 232-250, Springer.

A Theory for Interpreting Arrow Symbols

25

Kurata, Y. and Egenhofer, M. (2006a) Ontology-Based Interpretation of Arrow
Symbols for Visual Communication. in: Andrienko, G., Andrienko, N.,
Jankowski, P. and MacEachren, A. (eds.) GIScience 2006 Workshop on
Visualization, Analytics & Spatial Decision Support, Münster, Germany, CDROM.
Kurata, Y. and Egenhofer, M. (2006b) Topological Relations of Arrow Symbols
in Complex Diagrams. in: Barker-Plummer, D., Cox, R. and Swoboda, N. (eds.)
Diagrams'06, Stanford, CA, Lecture Notes in Artificial Intelligence 4045, 112126, Springer.
Kurtoglu, T. and Stahovich, T. (2002) Interpreting Schematic Sketches Using
Physical Reasoning. in: Davis, R., Landay, J. and Stahovich, T. (eds.) AAAI
Spring Symposium on Sketch Understanding, Stanford, CA, 78-85, AAAI Press.
Landay, J. and Myers, B. (2001) Sketching Interfaces: Toward More Human
Interface Design. IEEE Computer 34(3): 56-64.
Monmonier, M. (1990) Strategies for the Visualization of Geographic TimeSeries Data. Cartographica 27(1): 30-45.
Oviatt, S. (1996) Multimodal Interfaces for Dynamic Interactive Maps. in:
Bilger, R., Guest, S. and Tauber, M. (eds.) Conference on Human Factors in
Computing Systems (CHI '96), Vancouver, Canada, 95-102, ACM Press.
Oviatt, S. and Cohen, P. (2000) Multimodal Interfaces That Process What
Comes Naturally. Communications of the ACM 43(3): 45-53.
Retz-Schmidt, G. (1988) Various Views on Spatial Prepositions. AI Magazine
9(2): 95-105.
Stahovich, T. (1997) Interpreting the Engineer's Sketch: A Picture Is Worth a
Thousand Constraints. in: Anderson, M. (ed.) AAAI Fall Symposium on
Reasoning with diagrammatic Representations II, Stanford, CA, 31-38, AAAI
Press.
Tversky, B. (2001) Spatial Schemas in Depictions. in: Gattis, M. (ed.) Spatial
Schemas and Abstract Thought. MIT Press: Cambridge, MA: 79-111.
Tversky, B., Heiser, J., Lozano, S., Mackenzie, R. and Morrison, J. (2007)
Enriching Animations. in: Lowe, R. and Schnotz, W. (eds.) Learning withAnimation: Research and Implications for Design. Cambridge University Press:
Cambridge, UK: 263-285.
Tversky, B., Zacks, J., Lee, P. and Heiser, J. (2000) Lines, Blobs, Crosses andArrows: Diagrammatic Communication with Schematic Figures. in: Anderson,
M., Cheng, P. and Haarslev, V. (eds.) Theory and Application of Diagrams(Diagrams 2000), Edinburgh, UK, Lecture Notes in Artificial Intelligence 1889,
221-230, Springer.
van der Waarde, K. and Westendorp, P. (2000) The Functions of Arrows in User
Instructions. in: The IIID Expert Forum on Manual Design, Eskilstuna, Sweden,
WW1-7, International Institute for Information Design.
Westendorp, P. (2006) Design & Evolution of the Symbolic Arrow. in: de Rijk,
T. and Drukker, J. W. (eds.) Design & Evolution Conference, Delft,
Netherlands, W1-26, Design History Society.

26

Kurata and Egenhofer

Worboys, M. and Hornsby, K. (2004) From Objects to Events: Gem, theGeospatial Event Model. in: Egenhofer, M., Freksa, C. and Miller, H. (eds.)
GIScience 2004, Adelphi, MD, USA, Lecture Notes in Computer Sceince 3234,
327-344, Springer.