SiteLens: Situated Visualization Techniquesfor Urban Site Visits
Sean White
Steven Feiner
Columbia University
New York, NY 10012{swhite, feiner}@cs.columbia.edu
ABSTRACT

Urban designers and urban planners often conduct site visitsprior to a design activity to search for patterns or better understand existing conditions. We introduce SiteLens, anexperimental system and set of techniques for supportingsite visits by visualizing relevant virtual data directly in thecontext of the physical site, which we call situated visualization. We address alternative visualization representations
and techniques for data collection, curation, discovery,
comparison, manipulation, and provenance. A real use scenario is presented and two iterations of evaluation with faculty and students from the Columbia University Graduate
School of Architecture, Planning and Preservation provide
directions and insight for further investigation.

(a)

(b)

Author Keywords

(c)

(d)

Augmented reality, situated visualization, mobile computing, urban planning, urban design, site visits, data curation.
ACM Classification Keywords

H.5.1 [Information Interfaces and Presentation]: Multimedia Information Systems—Artificial, augmented, andvirtual realities; H.5.2 [Information Interfaces and Presentation]: User Interfaces—GUI
INTRODUCTION

Urban planners, urban designers, and architects often visit asite prior to a design activity related to the site. These site
visits are used for different purposes by different professionals, but the general goals are to get a sense for the physical site, find patterns, and discover and record new insights
about the physical location and its characteristics. Site visits
are similar to ethnographic study in human-computer interaction research, but focus on the physical place, as well as
the people in that place.
For example, an urban planner might first create a series ofmaps about a site that represent its demographics and use.
She may then visit the physical site to view and photographit, and look for patterns such as congregations of people,
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise,or republish, to post on servers or to redistribute to lists, requires priorspecific permission and/or a fee.
CHI 2009, April 4–9, 2009, Boston, MA, USA.
Copyright 2009 ACM 978-1-60558-246-7/08/04…$5.00

Figure 1. (a) Map of CO sensor data and photographs to provide context for the sensor data. (b) First iteration of SiteLens
with data panels (gray boxes in upper-left corner) and detailpane (gray box in middle-right). (c) Dynamic map view.
(d) Comparing locally sensed data (red) and remote EPA
sensor reading associated with the site (green).

traffic flows, and vegetation. On returning to her office, shemight record patterns she found onto the maps. Existing
tools for this process include geographic information systems such as ArcGIS, and still and video cameras. A sample
map (Fig. 1a) shows geocoded carbon monoxide data (CO)and photos provide location context.
Several issues arise in the current process. First, there may
be aspects of the site that are not visually apparent while
visiting the site; for example, air quality and CO levels canbe important when considering development, health andenvironmental justice issues, but cannot be seen with the
naked eye. Second, the map data and the physical site are
separate, imposing additional cognitive load on the user toplace data in the scene or recall the scene when looking at amap offsite. Finally, still photos and video may not represent the dynamics of the physical site and environment
when trying to understand correlations or associations between the data and the site.
To address these issues, we introduce SiteLens (Fig. 1b–d),
a prototype hand-held visualization tool to support site visits, for interacting with aspects of a physical site that do nothave a natural or perceivable visual representation. Our
goal is to develop a tool that helps urban planners and urban

designers see new patterns and gain insight into a place byvisualizing its data.
Visualizations are typically shown on a stand-alone display,whether desktop, hand-held, or head-worn. In the figureground relationship, the physical environment that serves as
the ground in which the visualization is presented needhave no meaningful relationship to it. In contrast, we use
the term situated visualization to describe a visualizationthat is related to and displayed in its environment; for example, a display of CO data directly overlaid on the user’sview of the physical location in which it was sampled (Fig.
1b) or demographic census data displayed in the context of
the relevant city block (Fig. 2a). Situated visualizations gain
meaning through the combination of the visualization andthe relationship it has to its environment. In the context ofsite visits, the visualizations become a virtual part of thesite. Note that visualization in augmented reality (AR) is
not necessarily situated visualization by our definition.
However, there are several excellent examples that we discuss next.
RELATED WORK

We draw inspiration from several projects. Reitmayr et al.
[10] developed systems for managing and displaying large
scale models and annotations in urban environments. The
Vidente project [11] has been investigating visualization of
subsurface features such as pipelines and power cables forutility field workers. Their approach takes geographic data
models of these subsurface features and transcodes them forvisualization and filtering. In contrast, we focus on invisible
aspects of a site, beyond the built environment, that may nothave a natural visual or spatial representation, and on comparing multiple related datasets.
We note that sensed data has become an important topic in
the HCI community as new ways of collecting data such as
participatory sensing [3] and mobile sensors [8] evolve. Our
work complements these systems by exploring alternative
ways to visualize and interact with sensed data.
INTERACTION

The following use scenario provides a description of thetypes of interaction and tasks we support, and is followedby explanations of specific elements of our prototype.
John is an urban planner. He typically looks for patterns ina physical location when he visits a site and today is interested in environmental issues. He arrives at the corner of
133rd Street and Broadway, an area of interest for future
design activities, and takes out his SiteLens. The SiteLensshows him there are several different datasets in the location, so he filters for environmental data. He sees two setsof CO data. He opens one and sees that it is displayed in theworld (Fig. 1b), so he knows that it was collected at the site.
He opens the next set and notices that it is displayed fixed
to the screen, indicating that it was not collected nearby. He
tilts the SiteLens down to get a larger scale map view (Fig.
1c) to determine where the data was captured. It is quite far

away, so he tilts the SiteLens back up. He freezes the scene,queries both sets for provenance and notices that someone
from the community collected the first dataset and the US
Environmental Protection Agency (EPA) collected the second. He wants to compare them, so he drags the EPA data
to the local data to spatialize it (Fig. 1d). He makes sure thatboth datasets are visualized differently, and sees that there
is a large difference between the EPA data and local data.
He freezes the image again, captures it for later use, andwalks to the next street to investigate further.
Data Curation and Availability

Although we focus on visualization and interaction, data
curation is a necessary and integral component of situatedvisualizations. As part of this project, we have been collecting and curating a variety of datasets to better understandthe tools for collection, aggregation, and distribution. The
red dataset in Figs. 1–2 encodes CO levels we collectedwith a Lascar EL-USB-CO data logger and a Honeywell
GyroDRM, which combines GPS with a gyro-stabilized
dead reckoning module, for geocoding. Custom software
combines data logs and converts the output to KML [1], anXML-based language schema and file format for representing geographic data, maintained by the Open Geospatial
Consortium. KML is used in Google Earth and Google
Maps, making it easy to import datasets from these applications into SiteLens. CO data was also obtained from EPA
sites, and additional datasets have been curated from geocoded US census data and single-location environmental
sensing stations.
Loci of presentation

SiteLens has three primary loci of presentation: a screenfixed display in the upper left corner, a screen-fixed, worldoriented map display, and a world-fixed augmented reality
display. Our system considers the nature of the data itself
and defaults to displaying it in a locus that is appropriate to
the spatial nature of the data. For instance, our system defaults to display data as screen-fixed if it is beyond the current view or is not inherently spatial. In Fig. 2a, census data
is relevant to the site, but is recorded on a block or superblock scale. Therefore, we present the data screen-fixed in
the upper-left corner. In contrast, the locally recorded COdata is presented world-fixed because it is displayed in thelocations in which it was recorded. Later, we discuss breaking these boundaries when comparing data.
Visual Representations

When mapping a non-physical characteristic such as COlevel to properties of a visual mark such as the size or altitude of a sphere, we consider the representation both byitself and in the context of the physical scene. To explore
different representations, we use three different visualtypes: spheres, cylinders, and smoke. These three representations were developed in collaboration with our colleagues
in urban design and urban planning. We chose these genericrepresentations as a first cut at virtual representation ofphysical data because dots (spheres) are familiar carto-

(a)

(b)

graphic representations, the representations were meant tobe generic to other sensor data, and the abstractions lendthemselves to redundant encoding of values. In each of therepresentations, the visual mark is displayed in the location
where the data was sensed (Fig. 2a–c). For spheres, theparts per million (ppm) value is mapped to both continuous
altitude and bi-level color. Higher, red spheres have higher
values, while lower, grey spheres have lower values. For
cylinders, ppm is mapped to both length of the cylinder andcolor. Taller cylinders have higher values and color mapping is the same as spheres. For smoke, ppm is mapped todensity. Denser smoke represents higher ppm values.
Comparing and Querying Data

Data comparison facilities provide a way to validate existing datasets. If two datasets contain spatial data relevant toa given physical location, they can be compared directly.
However, sometimes data intended to represent a physicallocation is actually collected remotely. In this case, we provide a means to spatialize data to match a related dataset.
For example, in Fig. 1d, the red CO dataset was collected in
the locations in which it is represented. However, the green
CO dataset, which was collected several miles away, is theclosest EPA dataset. Instead of comparing the red data to
the single value representing the green data by default, we
let the user spatialize the green data. We do this in SiteLens
by touching the data panel for the EPA data and dragging itto the spatialized red data. We hypothesize that this makes
visual comparison simpler without losing the relevance ofthe physical context.

(c)Figure 1. (a) Combining ascreen-fixed readout (upper-left) with world-fixeddata representation using
spheres. Two alternative
representations of the same
data: (b) cylinders, and (c)smoke.

are active in this mode. In addition, a newly visible button
activates a scene grabber to save the image for later use.
IMPLEMENTATION

Our prototype runs on a 1.2lb Sony VAIO VGN-UX390N
Ultra Mobile PC with a built-in camera, GlobalSat BT-338
GPS, and InterSense InertiaCube3 (IC3) inertial orientation
tracker. SiteLens is built on top of Goblin XNA [2], which
supplements Microsoft’s XNA infrastructure with AR functionality, including 6DOF optical marker tracking using
ARTag [5]. Each visual representation is a subclass of adata node in our architecture, so we can easily create new
visual representations and data mappings. Spatial data isstored in an octree to provide quick access to location anddistance information from a given data node to the current
location. Current location (and camera orientation) is gathered through a combination of ARTag fiducials, GPS, andIC3. (In contrast to Vidente, we use optical markers to address urban areas with limited GPS satellite visibility.) Stability of distant objects is increased by combining IC3 orientation and ARTag or GPS location.
EVALUATION

As with any medium, additional information about the data
being visualized can help the viewer better understand potential issues such as bias or reliability. For example, avisualization of CO data may be perceived differently depending on whether it was created by a community member
or a known industrial polluter. To help address this, the user
can select any visualization node by touching it to bring up
an information pane (Fig. 1b) that provides metadata suchas provenance and creation date.

Plaisant [9] argues that evaluations of visualization techniques should incorporate real tasks and field studies. North[7] suggests insight as an indicator for validating visualization techniques. As a first step in evaluating our prototype,
we obtained feedback from urban designers and planners in
the Columbia University Graduate School of Architecture,
Planning and Preservation (GSAPP) through two iterative
field studies. The studies were conducted in the Manhattanville neighborhood of New York City. In the first study,
two colleagues from GSAPP explored the site using the
scenario described earlier in this paper. In the second study,
four participants from GSAPP used a revised prototype atthe same site and were given a brief post hoc questionnaire,
eliciting opinions about visual representations and systemuse. In both cases, researchers were present and observed
subjects as they used the system. Additional unstructured
discussions with subjects followed both studies.

Freezing to interact

OBSERVATIONS AND DISCUSSION

Selection of a particular data point can be difficult when theworld and associated visualizations are moving. To address
this, we provide a button that freezes the video image but
not the dynamics of the AR system, similar to Güven etal.’s use of freezing to author [6]. All regular interactions

Moments of insight were observed. For example, map dataalone could not explain why the locally recorded CO levels
were higher towards the end of one street; however, visualinspection of that street during the field study revealed that,near where the higher CO levels were recorded, cars were
idling as they prepared to enter the highway. This combina-

tion of virtual and physical observation provided insight
into potential causality.
One frustration with the system was that the data was considered stale. This brought up two issues. First, while there
was a closer spatial association between the site and sensed
data, the temporal association was unclear. Second, there
was a desire to have live or dynamic sensing coupled withexisting data to “further explore an area or fill in the gaps.”
Representation and Presentation

Reactions to the different representations were mixed.
Spheres were considered better than cylinders for localizing
the data. In terms of specific data values, participants were
initially confused about whether the CO ppm value wasmapped to sphere size or height. Surprisingly, we foundthat the psychological impact of the smoke was more important than the more accurate localization and value of theother representations. One participant said “I like thesmoke…It’s hard to see quantity of things, but… psychologically it helps to represent the idea better.” Another suggested that perhaps “you just need to know bigger orsmaller, but not the actual value.” In further discussion,
smoke with the option of visualizing spheres was suggested
because the initial representation of smoke provided astronger psychological effect, provoking stronger reactions.
In general, we see the need to provide the user with more
control over visual form (geometry, color, size) and data
mapping in the spirit of Chuah et al.’s SDM [4]. For example, while shadows were considered useful for enhancing
the sense of realness and provided distance cues, our design
choice for mapping CO concentration to height was notconsidered obvious. Participants wanted to try alternate
visual representations to explore changing data perception.
In terms of presentation, a difficulty with the screen-fixed
display was that, while the data was representative of thesite (including census data), participants felt that it was insufficiently dynamic. One stated that the data “doesn’tchange when I move around, so it feels less important.”
Interaction

In our first iteration of the system, subjects were distracted
by the instability of data. Our combination of data positionand sensor fusion in the second iteration significantly stabilized the visualization. While the actual placement of data
was slightly less accurate, the location of data was sufficient for associating with local features of the environment.
Freezing the camera image, when desired, while keeping
the overlaid graphics live, supported manipulating the interface and visualization without having to keep SiteLens
pointed at the scene being overlaid. As an extension of this,
we found that the on-screen user interface controls werebest positioned in the lower left and lower right of thescreen and along the edges where the user’s thumbs could
easily access them. However, direct manipulation of thevisualizations, such as touching them to show metadata,

was useful once the display was frozen. Selection of specific nodes in dense areas of data was still difficult becauseof overlapping nodes.
Our users felt that capturing combined images of the physical and virtual scene to create a single “real” image was
useful for documenting the site visit. Using the SiteLens
prototype was not felt to be significantly harder than using avideo or still camera and could be imagined as a common
tool. It was even suggested that SiteLens could be used for
an iterative process of data curation, where visualizationand sensing are combined with organizational tools to help
create new datasets that create a portrait of the site.
CONCLUSION

Our contributions include a new application space that
benefits from AR and visualization techniques, a prototype
system incorporating techniques for presenting and interacting with situated visualizations, novel visualizations of CO
sensor data, and discussion of early feedback from colleagues in urban design and urban planning. Based on initial usage, we plan to pursue two areas further. First, we areinterested in increasing the dynamics and symmetry ofsensing and visualization by extending the system to live
sensor data. Second, we plan to further explore alternativevisual representations for different data types.
ACKNOWLEDGMENTS

We thank Sarah Williams, Petia Morozov, Ohan Oda, Steve
Henderson, and Lauren Wilcox. This work was funded inpart by a grant from the Boston Society of Architects, NSF
Grant IIS-03-25867, and a gift from Microsoft.
REFERENCES
1. http://www.opengeospatial.org/standards/kml/
2. http://www1.cs.columbia.edu/graphics/projects/goblin/
3. Burke, J., Estrin, D., Hansen, M., Parker, A., Ramanathan, N.,
Reddy, S., and Srivastava, M.B., "Participatory sensing," ACM
Sensys World Sensor Web Workshop, 2006.
4. Chuah, M.C., Roth, S.F., Mattis, J., and Kolojejchick, J.,"SDM: selective dynamic manipulation of visualizations,"
UIST. 1995, pp. 61-70.
5. Fiala, M., "ARTag, a fiducial marker system using digitaltechniques," CVPR 2005, 590-596.
6. Guven, S., Feiner, S., and Oda, O., "Mobile augmented reality
interaction techniques for authoring situated media on-site,"
ISMAR 2006, 235-236.
7. North, C., "Toward measuring visualization insight," IEEE
Computer Graphics and Applications, vol. 26, pp. 6-9, 2006.
8. Paulos, E., Honicky, R.J., and Goodman, E., "Sensing Atmosphere," Workshop on Sensing on Everyday Mobile Phones in
Support of Participatory Research, 2007.
9. Plaisant, C., "The challenge of information visualization evaluation," in Advanced Visual Interfaces, 2004, pp. 109-116.
10. Reitmayr, G. and Schmalstieg, D., "Collaborative Augmented
Reality for Outdoor Navigation and Information Browsing,"
Symp. Loc. Based Services and TeleCartography, 2004, 31-41.
11. Schall, G., Mendez, E., Kruijff, E., Veas, E., Junghanns, S.,
Reitinger, B., and Schmalstieg, D., "Handheld Augmented Reality for Underground Infrastructure Visualization," To appear
in Journal of Personal and Ubiquitous Computing, 2008.