The Volume Slicing Display
Alvaro Cassinelli∗, Masatoshi Ishikawa
Ishikawa Komuro Laboratory, University of Tokyo

Abstract
The Volume Slicing Display is a device enabling the interactive exploration of volumetric data (e.g. medical images) using a piece ofPlexiglass or paper that functions both as a control interface and asa passive, untethered projection screen. With the VSD, radiologists
would be able to retrieve a certain amount of three-dimensionality
from a flat X-ray plate at any time, by just touching certain portionsof the screen, orientating and manipulating it freely above a calibrated projector. It is interesting to note that such interface could
solve another important issue, that of the confidentiality of the patient data, since without the machine the piece of paper will only
show an undecipherable 2d-barcode identifying the patient.

1

Introductiona plexiglass screen (P4P algorithm) [1]. Each corner was identified
through a different temporal code sequence. The new prototype
presented here uses instead retro-reflective ARToolkit markers to
track position and orientation of a piece of Plexiglass or translucent
paper. A small computer (mac Mini) calculates the corresponding
slice of a virtual volume hovering in mid-air above the projector,
and prepares the image to be displayed by correctly anamorphosing
the slice in real time.
Secondary markers are used either as discrete switches to toggle
modes or operation (such as locking and dragging the position of
the virtual volume on space, or taking a snapshot of the current slice
for printing purposes) or as continuous rotating knobs (to set thezooming factor, change the contrast, etc - Fig. 2). All the hardware(a small LED-based projector, a webcam, some infrared LED and aMac mini) is integrated in a small box (35x35x35cm).

This experimental visualization interface may one day enable ateam of experts (surgeons, geologists, designers, architects) to explore 3D virtual objects as if these co-exist in the physical space,
as well as to explore them interactively a piece of paper or plastic
that will function both as a control interface (through the additionof physical widgets) and as a passive, untethered projection screen.
An interesting possibility currently being explored is to make thesewidgets an integral part of the screen (for instance, bendable corners) or use an erasable marker to directly write commands on thescreen.

Figure 2: Zooming using a rotating physical widget

Figure 1: Horizontal and saggital cuts with locked volume
This is a second prototype conceived with portability and cost constraints in mind, and can be deployed over off-the-shelf hardware.
A first prototype used a custom-made vision chip for pose estimation by tracking four infrared LEDs situated at the corners of∗ e-mail: alvaro@k2.t.u-tokyo.ac.jp

This system represents a logical evolution of our previous researchon the ’Khronos Projector’ interface [2], which is a tangible interface with which one is capable to ’sculpting’ a volume of data(in particular stacked video frames). Although the screen was deformable, its borders were fixed and it was impossible to rotate thescreen with respect to the volume being sliced. The Volume Slicing
Display enables this, but the screen is rigid. Although we have beenworking towards a screen that is both orientable and deformableusing structured laser light to compute the deformation in real time[3], we feel that this complex mode of interaction is only necessary
in a limited number of applications.

2

Preliminary results and discussion

We demonstrated a procedure to rotate the volume of data without
relying on a secondary input device: a marker on a corner can bepartially covered to lock or unlock the volume on space. Then thescreen motion will either affect the position and orientation of thevirtual volume (unlock mode), or serve to control slicing place (lockmode). This way, one can smoothly and intuitively pass from a saggital/coronal/horizontal slice, while obtaining an observed slice thatis always horizontal at the end (Fig. 1). The unlock mode can also
serve to freeze a slice and pass to another person for scrutiny (checkvideo here [5]). We also demonstrated a way to alter these markers
in order to generate new commands or record information pertaining to the history of the manipulation (by folding some parts, oraltering specific markers by superimposing figures or filling blanks
with an erasable marker).
Camera/projector calibration is performed off-line. Intrinsic projector matrix is obtained manually or using synthetic imaging: a pointof light generated by the projector automatically follows the contours of a custom designed calibration pattern (Fig. 3). The contour
following algorithm is similar to the one developped in our ’Sticky
Light’ project [4]. Intrinsic and extrinsic parameters are estimatedusing Matlab Calibration Toolbox. Extrinsics are obtained by considering the projector as a secondary camera, and proceeding withthe stereo-calibration toolbox.

workable space as large as a whole room. Another goal is to makethe markers invisible to the naked eye (we have tried using infraredink, but the contrast is not sufficient). This will allow for a clear
and large projection surface. Related to this is the need for a simple auto-calibration procedure for multiple projectors and cameras.
This could be done by active marker tracking as in [11], using stable
markers or feature points on the projection screen. Since the present
prototype uses an infrared source collinear with the projector, an interesting possibility is to use a sanded plastic semi-transparent sheet
which will couple light into the screen. Markers can then be engraved on the surface, and will appear as more bright than the restof the screen. Interestingly, this configuration will permit FTIRbased tracking of fingers over the surface, as done in [9]. This will
enable zoom and rotation of images using bare finger and gestures,
as well as annotate and draw trajectories (such as surgical paths) inspace. Among the interesting directions that this project highlights,
is the development of an ”origami”-like user interface, in which
different shapes and folds of the flexible screen are interpreted by
the machine as specific displaying commands; also of interest is
the possibility of tracking the user head so as to represent volumes
through the screen (as has been done in the ”Deskrama” project[7] and in our previous research on the ’Parallax Augmented Desktop’ [10]). Finally, we would like to extend the applications of thistechnology in the domain of Media Arts (interective architectures,dance performances) and 3d spatial games.

References[1] Ito, T., Cassinelli, A., Komuro, T. and Ishikawa, M.: 3D Object
Representation Using a Tangible Screen, Proc. of the SICE2006
Conf. (2006).
Figure 3: Calibration contour for syntethic image formation[2] Cassinelli, A. and Ishikawa, M.: Khronos Projector. Emerging
Technologies, SIGGRAPH 2005, Los Angeles (2005).

Finally, it is interesting to note that the software can run on a desktop computer with a commercial webcam and without a projector;
although the images are not projected on space, the computer generated slice is superimposed to the live video on the computer monitor; this may still be helpful when one does not have access to theVSD machine, and can be sufficient to identify the ”patient file” aswell as rapidly browse the image database (Fig. 4).

[3] Watanabe, Y., Cassinelli, A., Komuro, T., and Ishikawa, M.:
The Deformable Workspace: a Membrane between Real and Virtual Space, IEEE Int. Workshop on Horizontal Interactive HumanComputer Systems (Tabletops Interactive Surfaces 2008) (Amsterdam, 2008.10.03) / Proc. pp. 155-162.
[4] Sticky Light: www.k2.t.u-tokyo.ac.jp/perception/StickyLight/[5] www.k2.t.u-tokyo.ac.jp/perception/VolumeSlicingDisplay/[6] Saeki, Y. and Hirota, K.: Cross-Section Projection Display using Movable Screen, 11th Japan Virtual Reality Conference, Sept.
2006.
[7] Nagakura, T.: Deskrama, SIGGRAPH 2006, E-Tech

Figure 4: Running without a projector
Although there has been related research and similar prototypes(see in particular [6],[7] and [8]), the prototype we propose here isparticularly compact, can be built with off the shelf hardware parts,and presents the advantage that the screen is a simple sheet of paperwith a set of printed markers that serve for pose estimation but that
can also be used to identify the patient in a confidential manner.

3Improvements and Future research

The present prototype uses a single projector; this reduces the working space as well as the maximum possible inclination of the screen.
We are know considering the use of multiple projectors to make the[8] Ratti, C., Wang, Y., Biderman, A., Piper, B., Ishii: H. PhoxelSpace: an Interface for Exploring Volumetric Data with PhysicalVoxels, in Proc. of Designing Interactive Systems (DIS 2004), August 1-4, 2004
[9] Han, J. Y. 2005: Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection. In Proce. of the 18th Annual ACM
Symp. on User Interface Software and Technology.
[10] Reynolds, C., Cassinelli, A., and M. Ishikawa: Metaperception: reflexes and bodies as part of the interface, Conf. on
Human Factors in Computing Systems (CHI 2008), April 10, 2008,Florence, Italy.
[11] Gupta, Sh. and Jaynes, Ch.: Active Pursuit Tracking in a
Projector-Camera System with Application to Augmented Reality,
Proc. of the 2005 IEEE Comp. Soc. Conf. on Computer Vision
and Pattern Recognition (CVPR’05), Volume 03, pp: 111, 2005.