IllumiRoom: Peripheral Projected Illusions for Interactive Experiences
Brett R. Jones1,2
Hrvoje Benko1
1Microsoft ResearchRedmond, WA{benko, eyalofek, awilson}@microsoft.com

Eyal Ofek1
Andrew D. Wilson1
2
University of Illinois at Urbana-Champaign
Urbana, ILbrjones2@illinois.edu

Figure 1. IllumiRoom is a proof-of-concept system that augments the physical environment surrounding a television to enhance
gaming experiences. For example, (a) with a 3D scan of the physical environment we can (b) directly extend the FOV of the game,(c) selectively render scene elements, (d) augment the appearance of the physical environment (here as a cartoon). The images are
un-edited; showing the real-time, working prototype.

1. Abstract
IllumiRoom is a proof-of-concept system that augments the areasurrounding a television with projected visualizations to enhance
traditional gaming experiences. Our system demonstrates how
projected visualizations in the periphery can negate, include, oraugment the existing physical environment and complement thecontent displayed on the television screen. We can change theappearance of the room, induce apparent motion, extend the fieldof view, and enable entirely new physical gaming experiences.
Our system is entirely self-calibrating and is designed to work in
any room.

2. Introduction
The IllumiRoom proof-of-concept system consists of a projector
and a depth sensor that covers a wide area surrounding atelevision screen (Figure 2). The projected visuals enhance theviewing experience and blur the boundary between the on-screen
content and the surrounding room. We demonstrate how projectedvisualizations in the periphery can negate, include or augment thephysical environment, and thus enhance the content displayed on
the television screen. We call such visualizations peripheral
projected illusions.
Similar to Focus+Context displays (Baudisch, Good, & Stewart,2001), the television provides a traditional, high-resolution
gaming experience and the projector provides low-resolution
information for the user’s peripheral vision. In contrast to
previous work, we do not use flat, white projection screens, butinstead adapt the projection to the existing living room
environment. We are the first project to consider the geometry andappearance of the surrounding room and use that information tocreate novel, interactive visual experiences.

We explore the design space of peripheral projected illusions anddemonstrate eleven example illusions. For instance, we candirectly extend the game content into the living room bydisplaying a wide field-of-view rendering of the game on thesurrounding physical environment with radiometric compensation(Figure 1b). We can also selectively introduce game elements into
the living room; e.g. only displaying weapons fire and explosions
in a first person shooter (Figure 1c). Or, we can augment theliving room to match the theme or mood of the game; e.g. theroom can transform into a ‘cartoon world’ by super-saturating thecolors and adding black silhouette edges. We can also enablephysical-virtual gaming experiences by using the 3D data of theliving room; e.g. a grenade can roll out of the television, thenbounce off the coffee table, and roll on the floor of the room. Allof these effects adapt to the color and geometry of the livingroom, and are design to work in any room.
Ideally, IllumiRoom would be directly integrated into a next
generation console and new games would be designed forIllumiRoom from the ground up. We envision an API that enables
triggering illusions, changing surface appearance, controllingroom lighting, inserting objects into the physical environment, etc.
In our prototype, we demonstrate how to connect existing
commercial game content to drive illusions: using controller
input, optical flow, and source code integration.

3. SIGGRAPH Demonstration
This work will be initially presented at ACM CHI 2013 in May
2013 (Jones, Benko, Ofek, & Wilson, 2013), but will not have alive demonstration. At SIGGRAPH Emerging Technologies wewill demonstrate the full working prototype with a ‘mock’ livingroom (shown in Figure 1). Visitors will sit down on a couch andexperience a magical, immersive gaming experience. Visitors willtake turns using an Xbox controller to play video games, which
are coupled with eleven peripheral projected illusions. We canlive switch between the peripheral projected illusions for eachexample game. We believe this system will be very popularamong conference attendees.

References

Figure 2. (a) Our vision for a productized IllumiRoom system
includes an ultra-wide field of view device that sits on a coffee
table and projects content in the area surrounding thetelevision. (b) Our current proof-of-concept prototype uses anoff-the-shelf projector and a Kinect sensor.

Baudisch, P., Good, N., & Stewart, P. (2001). Focus Plus Context
Screens: Combining Display Technology with
Visualization Techniques. ACM UIST.
Jones, B., Benko, H., Ofek, E., & Wilson, A. D. (2013).
IllumiRoom: Peripheral Projected Illusions for Interactive
Experiences. ACM CHI (to appear).