Ontology-based Interpretation of Arrow Symbols for Visual Communication
Yohei Kurata and Max J. Egenhofer
National Center for Geographic Information and Analysis and Department of Spatial Information Science and Engineering University of Maine{yohei, max}@spatial.maine.edu

Motivation

Necessity of Background Knowledge

Ontology for Detecting Mobility

Arrow symbols are ubiquitous in visual representations.

The following three figures have the same structural patterns—“object Ælocation”—which correspond to action or association.

An ontology*3 is useful for deriving computationally, for instance,
the mobility of a component.
* We used WorldNet (Fellbaum, 1998) as a sample ontology.

BerlinBerlinbut only the first figure may capture action (i.e., a trip to Berlin),because among a car is movable, while a broken car, Brandenburg Gate,
and Berlin are immovable.

However, since arrow symbols are polysemic, people actually
have to interpret the meaning of arrow symbols.

Feeding Protein
Fish Flour
Soybeans

Fish Catch• Property specification requires acomponent that may have aproperty related to orientation, suchas moving speedv1v1• To gain an insight into the appropriate use of arrow symbols
in visual representations• To contribute the development of pen-based user interfaces

Köln

2. The meanings of arrow symbols arecategorized into four types: action,property specification, annotation, andassociation*2
3. A method for deriving candidates forvalid interpretation from the structural
patterns is developed*2

Sep. 20

GI science
MünsterMr. K

MomentObjectSep. 20• Association requires two associable
components

animal…

mammalsdog…

plant…

: movable

An entity has mobility if it has:
- an active operation move (in its intransitive sense) or
one of its subclasses, or- a passive operation be moved or one of its subclasses

hit

move
transitive

intransitive

propel, impel

go, travel

throw

…

kickIn addition, the ontology is potentially useful for detecting:Kölsch beerThus, an interpretation is detected invalid if the interpretation impose acertain characteristics on a component even though this component
cannot have this characteristics.

In this way, making use of the knowledge on whether
components may have the above characteristics, we cannarrow down the candidates for valid interpretation• Direction-dependent mobilitye.g., elevator ::= a lifting device consisting of a platform or
cage that is raised and lowered mechanically*3• Field-dependent mobilitye.g., vessel ::= a craft designed for water transportation*3

What about the characteristics other than mobility?

EventGI science
Münster

Location

Mr. K

Note

Then, how can we obtain such knowledge?
—Ontology is a potential source.

ObjectEvent, Moment ,Note
Location

Action or Association

The next question is how tonarrow down these candidatesCase 3

Kölsch beer

Related Work on Arrow Interpretations
1. Structures associated to individual arrowsymbol are defined*1

organism, beinge.g., Ball ::= a round object that is hit orthrown or kicked in games*3• Annotation requires a component
that can be assigned the given label

To study the mechanism of the interpretation of arrow symbols
through modeling the interpretation process

Mobility is inherited fromsuperclass (=parent)cat

El Niño

Research Goal

Case 2

These examples indicate that background knowledge about
components’ mobility is necessary for judging whether an arrowsymbol really captures action.

Similarly, other types of semantics require components with the following
characteristics:
For instance, in this figure, arrow symbols
illustrate causality, decrease, and shift.
How do we distinguish them unconsciously?

Mobility is employed in the definition of an entity classe.g., Animal ::= living organism characterized by voluntary movement*3

in
herit

Arrow symbols are believed to facilitate the understanding of a
spatio-dynamic process or mechanism.

Case 1inher itBerlin3?
Action*1 Kurata and Egenhofer (2005) Semantics of Simple Arrow Diagrams. Mental and External Diagrams, AAI Spring Symposium.
*2 Kurata and Egenhofer (2006) Structure and Semantics of Arrow Diagrams. COSIT’05.

Ontology:• A formal model of people’s conceptualizations about the world• Typically consists of⁃ vocabularies,⁃ properties and operations associated with each vocabulary⁃ relations between the vocabularies•Typically hierarchicalFuture Work• To investigate further use of ontologies to derive theknowledge about the other characteristics of components,
which is useful for detecting invalid interpretations of arrowsymbols.
• To make a guideline for appropriate use of arrow symbols
from a viewpoint of the mechanism that people interpret arrowsymbols, which facilitates successful understanding of visual
representations.