Pre-print / Post-print version
IEEE VR 2018 Research Demonstration

Water Flow Measurement Technology Assessing Spatial User Interaction
in an Underwater Immersive Virtual Reality Environment
Shogo Yamashita

Shunichi Suwa

Takashi Miyaki

Jun Rekimoto

The University of Tokyo

The University of Tokyo
Sony Computer Science
Laboratories

The University of Tokyo

The University of Tokyo
Sony Computer Science
Laboratories

CameraLeft circular
Polarizing sheet
Tracer particle madeof clear plastic
Right circularpolarizing sheet
Display device(A) Surround-screen swimming pool providing a(B) Setup for proposed water flow tracing technology. Cameras track (C) Transparent tracer particles become
swimmer with an immersive virtual reality experience.
invisible in water. Users can viewthe movement of tracer particles distributed in a swimming pool.
displayed images through the particles.

(D) Visualized tracer particles. We employ apolarization-based method to realize thetracking of transparent particles.

Figure 1: A novel water flow measurement technology using transparent tracer particles. This technology realizes underwater spatial userinteraction in an underwater immersive virtual reality environment (A). The tracer particles become invisible in water. Therefore, tracer particles
do not stop users from viewing the content on the screen (B, C). Cameras cannot track tracer particles if they are invisible. Accordingly, thisdemonstration presents an optical solution with which polarization-based technologies are employed to resolve the technical problem (D).

A BSTRACT
Underwater immersive virtual reality (VR) environments can reproduce unique VR experiences such as swimming in the sea withbeautiful coral reefs and a cage surrounded by sharks. Underwater VR poses new technical challenges to creating user interactions
because water and the surround-screen make existing methods forrealizing user interaction irrelevant. In this research, we present a
potential water flow measurement technology aimed at accessing
human-computer interaction in underwater VR. Flow measurement
can be realized by using tracer particles that are scattered in fluids.
However, existing tracer particles are not suitable for underwater
immersive VR because the particles stop users from viewing the content on the screen. Therefore, we propose transparent tracer particles
that become invisible in water and polarization-based technologies
that enable cameras to track the movement of particles. This technology enables virtual objects in VR to react with the actual movementof water and haptic feedback by creating water flow in the swimming
pool. These additions would enhance the illusion of immersion inunderwater VR.
Index Terms: Human-centered computing—Interface design prototyping; Human-centered computing—Virtual reality
1I NTRODUCTION

We propose a novel water flow measurement technology that realizes an underwater spatial user interaction in a surround-screen
swimming pool. The swimming pool provides an immersive virtualreality (VR) experience to the swimmers (Figure 1 (A)) [2]. Underwater VR can reproduce swimming experiences such as a sea withbeautiful coral reefs and a cage surrounded by sharks [7, 11].
This configuration of underwater VR poses new technical challenges, which have not been addressed by previous studies. Userinterfaces such as gaming controllers and motion capture systems
enhance the immersion in VR by creating interaction between users

and the virtual environment. However, the surround screen and watermake existing methods for realizing user interactions irrelevant. For
example, position tracking systems using infrared (IR) are widelyused for enabling interactions in standard immersive VR environments on the ground [5, 6]. However, existing position-tracking
devices are difficult or impossible to use in a surround-screen swimming pool because of the optical distortion, reflection, and IR absorption caused by water [1, 10]. In addition, position tracking
technology using visible lights is unstable in complex backgrounds
displayed on the surround-screen [8]. In this research, we present
a potential water flow measurement technology aimed at accessing
human-computer interaction in underwater VR environments.
Tracer particles scattered in fluids are used for fluid measurements [4]. Cameras track the particles in the fluid moving along
the flow. To enable cameras to track the movement of the particles, tracer particles must be conspicuous. To render tracer particles
especially bright against the background, fluorescent materials areused as tracer particles in many cases. Laser light or parallel light
configured by a lens is emitted toward the measurement environment to make the particles glow. However, existing tracer particles
are not suitable for underwater immersive VR because fluorescenceparticles stop users from viewing the content on the screen.
In this research, we propose transparent tracer particles that become invisible in water (Figure 1 (B, C)). The tracer particles are
made of fluorine resin. The refractive index of the material is nearly
equal to water’s; hence, almost no refraction occurs on the surfaceof the material in water. Therefore, users can view the displayed
images throughout the particles. This configuration enabled higher
invisibility compared to previous methods [12]. Cameras cannot
track tracer particles if they are invisible. Accordingly, this researchdemonstration presents an optical solution with which polarizationbased technologies are employed to resolve the technical problem(Figure 1 (B, D)). The proposed water flow measurement technology
requires lights only from screens to visualize tracer particles. Thisconfiguration reduces the cost of insulation compared to previous
methods using complex optical systems.

2

WATER F LOW M EASUREMENT R EALIZING S PATIAL U SERI NTERACTION IN U NDERWATER V IRTUAL R EALITY

2.1System Configuration and Methodology

We propose a potential water flow measurement technology usingtransparent tracer particles. The tracer particles must be highly transparent so they do not prevent swimmers from viewing the content
on the screen in a surround-screen swimming pool. The technicalproblem is that the cameras cannot view the movement of particles
if they are transparent. Therefore, we employ a polarization-based
method to realize the tracking of transparent particles. We placed
circular polarizing sheets on the inside of the screens and installedcircular polarizing sheets in the opposite direction on a camera fortracing. This configuration produces a dark background and brighttracer particles for the camera. This situation is ideal for opticaltracking [9]. The dark background is the effect of light shielding
caused by circular polarizing sheets. The reason only the particles
are bright is that they are made of plastic, which has the opticalproperty of rotating the polarization plane regarding polarized light[3]. Circular polarization is used because the particle becomes dark
at certain angles if we use linear polarization.
2.2

Application

2.2.1

Spatial User Interaction

VR is a technology that provides users experiences as if they actually
were entering the virtual world made of computer graphics. Oneof the issues in underwater VR is that the immersion is reduced
because the virtual world does not produce any reaction supposed
to be caused by water flow. For example, if a real object is floating
in a swimming pool, it moves with water when a wave or waterflow is caused by people in the pool. Underwater plants also change
forms when water flow strikes them. Water flow measurements inunderwater VR can create proper redactions of the virtual world
supposed to be caused by the movement of water.
2.2.2

Haptic Feedback

Previous studies have shown the reproduction of the five senses,
such as hearing, smell, and touch, in VR is necessary to increase theimmersion. However, the water causes technical issues concerning
the reproduction of five senses in the case of underwater immersive VR. For example, the stereophonic sounds created by multiple
speakers surrounding users increase the sensation of stereoscopic
video. However, sounds travel four times faster in water than in air,and humans cannot recognize from where the sound originates in
the water. Some research shows ultrasonic wave and air cannons
can reproduce the sensation of touch for VR. Nevertheless, thesetechnologies are unusable in the water. Haptic feedback is the useof the sense of touch in a user interface design. One of the methods
to provide haptic feedback is controlling water flows in the environment. Water flow measurements help create proper water flow witha combination of pumps or screws.
2.3

Swimming Training

Underwater VR can be used for a swimming training environment,
as it can also display information valuable for swimming, such as
swimming forms and previous records of the swimmer as a 3Dcharacter. Showing swimmers visualized water flow and tellingthem how to make proper water flow should enhance swimming
training systems using underwater VR.
3D EMONSTRATIONIn the demonstration, we show transparent tracer particles that areinvisible in water and the method to make them visible to cameras.
Moreover, we show a brief video explaining the underwater immersive VR environment and possible user interactions with theproposed water flow measurement technology. To demonstrate thetechnology, we have prepared a fish tank made of glass filled withwater and proposed tracer particles. We place a monitor coated witha right circular polarizing sheet behind the fish tank. We prepare eyeglasses coated with left circular polarizing sheets so the audience cansee the visualized tracer particles physically. Additionally, a camera
equipped with a left circular polarizing sheet tracks the movementof tracer particles in the fish tank. The audience can interact with asimple application as virtual characters moving with the water flow
displayed in the monitor.
4 C ONCLUSION
We proposed a water flow measurement technology using transparent tracer particles scattered in the water. This technology has
the potential to realize user interactions in underwater VR. We employ circular polarization to configure an ideal situation for opticaltracking with a dark background and bright tracer particles. In thedemonstration, we present the water flow measurement technology
by using a small fish tank with a monitor coated with a circularpolarization sheet. Audiences can see tracer particles being invisible
in the water, but visible through eyeglasses coated with circularpolarizing sheets.
5 ACKNOWLEDGEMENT
This work was supported by ACT-I, Japan Science and Technology
Agency (JST). GCL program of The Univ. of Tokyo by JSPS funded
our attendance at IEEE VR 2018. The picture in Figure 1 (C) wastaken by Greg Goebel. Photo title: False Clown Anemonefish, Scrips
Birch Aquarium, La Jolla, California 2012 License: CC BY-SA 2.0.
R EFERENCES[1] F. Bruno, G. Bianco, M. Muzzupappa, S. Barone, and A. Razionale.
Experimentation of structured light and stereo vision for underwater
3d reconstruction. ISPRS Journal of Photogrammetry and RemoteSensing, 66(4):508–518, 2011.
[2] C. Cruz-Neira, D. J. Sandin, and T. A. DeFanti. Surround-screen
projection-based virtual reality: the design and implementation of thecave. In Proceedings of the 20th annual conference on Computer
graphics and interactive techniques, pp. 135–142. ACM, 1993. doi: 10.1145/166117.166134[3] D. S. Kliger and J. W. Lewis. Polarized light in optics and spectroscopy.
Elsevier, 2012.
[4] A. Melling. Tracer particles and seeding for particle image velocimetry.
Measurement Science and Technology, 8(12):1406, 1997.
[5] T. B. Moeslund, A. Hilton, and V. Krüger. A survey of advances invision-based human motion capture and analysis. Computer vision andimage understanding, 104(2):90–126, 2006.
[6] O. motion capture device.
Optitrack motion capture device.
http://optitrack.com/.
[7] H. Osone, T. Yoshida, and Y. Ochiai. Optimized hmd system forunderwater vr experience. In ACM SIGGRAPH 2017 Posters, p. 25.
ACM, 2017.
[8] Q. underwater motion capture. Qualisys underwater motion capture.
http://www.qualisys.com/.
[9] G. Welch, G. Bishop, L. Vicci, S. Brumback, K. Keller, and D. Colucci.
High-performance wide-area optical tracking: The hiball tracking system. presence: teleoperators and virtual environments, 10(1):1–21,
2001.
[10] S. Yamashita, X. Zhang, T. Miyaki, and J. Rekimoto. Aquacave: anunderwater immersive projection system for enhancing the swimming
experience. ICAT-EGVE, 2016.
[11] S. Yamashita, X. Zhang, and J. Rekimoto. Aquacave: Augmentedswimming environment with immersive surround-screen virtual reality. In Proceedings of the 29th Annual Symposium on User Interface
Software and Technology, pp. 183–184. ACM, 2016.
[12] S. Yamashita, X. Zhang, M. Takashi, S. Suwa, and J. Rekimoto. Visualizing water flows with transparent tracer particles for a surround-screen
swimming pool. In Proceedings of the 8th Augmented Human International Conference, pp. 11–20. ACM, 2017.