THE TU-DELFT RESEARCH PROGRAM "UBIQUITOUS COMMUNICATIONS

The Ubiquitous Communications (UbiCom) program is a multidisciplinary research program at Delft University of Technology. The program aims at carrying out research needed for specifying and developing wearable systems for mobile multimedia communications. In particular the focus is on (i) real-time communication and processing of visual information for context-aware applications such as augmented reality, (ii) high bit rate communication at 17 GHz, (iii) architectural issues and performance optimization for heterogeneous communication and computation systems. In the vision of UbiCom, a user carries a wearable terminal and a lightweight see-through display. In this display the user can see virtual information that augments reality, projected over and properly integrated with the real world. The wearable system contains a radio link that connects the user to the ubiquitous computing resources and the Internet. This paper discusses the views on UbiCom, and motivates and illustrates the program-wide shared research objectives, namely low power, negotiated quality of service, and system approach. I. THE POSITION OF UBICOM I.1 Current Trends in Mobile Communications The growth and penetration of sophisticated digital communication systems, infrastructures, and services has been increasing constantly over the last decade. Examples of these services are the Internet, electronic mail, multimedia, pagers, PDA’s, and mobile telephony. From marginal penetration 15 years ago, these systems and services are becoming a commodity in both professional and consumer markets worldwide. The developments in these fields are still going strong. In particular, rapid advances – both in technology and services – can currently be observed in wireless and mobile systems that support the communication of different media, such as data, speech, audio, video and control [BULL99, WEZ98]. Current European wireless network and mobile phone services roll-out are centered around four available technologies, namely WAP, UMTS, Bluetooth, and mobile positioning systems [WAP, UMTS99]. The wireless application protocol (WAP), initially carried by second generation GSM and in the future by third generation UMTS wireless networks, will turn the mobile phone into a networked smart-phone capable of low to medium data rate Internet functionalities. Bluetooth will allow for short-range data communication between consumer appliances in a domestic This work has been funded by the Advisory Council for Research at TU-Delft (ARTD) under the Delft Interfaculty Research Center initiative [DIOC]. environment. Positioning systems will become integral part of mobile phones such that services can be made dependent on the location of the user in the network. When projecting the progress in mobile networks and services into the future, three developments are of importance. In the first place, we can observe that more and more mobile phone-like devices start to include accessories such as a small keyboard, a display, and a speech interface. Such communication and information-oriented systems are emerging as hybrids between the mobile phone and the wireless laptop personal computer. With higher bit rates supporting more advanced services, the integration of the personal computer and personal communication devices will be pushed even further. In the second place, we observe that computing resources are becoming ubiquitously – that is everywhere and at all time – available. We will soon live in an environment that supports us by providing ubiquitous computing for a wide variety of tasks and services. Daily life consumables, durable products and services already contain an ever-increasing number of sensors, actuators, processing units, and (embedded) software. Integration technology will continue to make these components smaller and more versatile. The personal computer has entered daily life as a necessary commodity, and the development of sophisticated communication systems in today’s society relies heavily on the availability of computation resources. Finally, we observe that communication and computing is becoming increasingly personal. The device (and therefore the user) is always on-line, the user is identifiable, the device can be personalized, and the system knows about the user’s position and environment. I.2 UbiCom Research Direction The combination of, on the one hand, ubiquitous computing and, on the other hand, the exchange of data, multimedia and audio-visual information in personal wearable communication systems, we call Ubiquitous Communications . In the decade to come, we will witness the emergence of Ubiquitous Communications, based on current trends in advanced signal processing, wireless networks, multimedia, and the Internet. It is, however, a mistake to ignore the potential for Ubiquitous Communications to be more than simply the convergence between mobile phones and portable computers. Systems that are personal and that have ubiquitous computing and communication abilities offer opportunities for entirely new services that are currently virtually nonexistent. One such challenge is the recently emerging but barely explored paradigm of context-aware communication and computing . By definition, the behavior of context aware systems and applications depends on the location and activities of the user at any given time. The potential of ubiquitous communications can be exploited for making communication and information-oriented products and services more adapted to the mobile human user. Examples can be found in mobile multimedia, media interaction, information on the spot about the spot, navigation systems, engineering, architecture, education, and security. Context-aware personal and wearable systems comprise a wide field of research [HUC99]. At Delft University of Technology the “Ubiquitous Communications” research program [UBICOM, TA2000] has chosen an extremely challenging view to drive research the research in context-aware mobile communication and computing. Namely, visual augmented reality has been chosen as the core feature of potential context aware applications (Figure 1). Augmented reality implies that the system is not only aware of the context; it also merges its own audiovisual virtual information with the real world. A system that anticipates the interests, intentions, and problems of the user and reacts accordingly by merging proper virtual information with the real world must continuously monitor the surroundings and actions of the user. As such, the UbiCom system processes information from sensors, combines it with information from other sources, and presents the output – for instance in visual form – to the user. One can have different views on what the multidisciplinary UbiCom program encompasses. In this paper we will first give three of those views (application view, wireless communication system, architecture view). Then we will discuss the common aspects in these views and motivate the program-wide shared research objectives. Finally we glimpse at ongoing technical research in the UbiCom program. More detailed information about factual research results can be found in the “Documents” section of the UbiCom web pages [UBICOM]. II. T HREE VIEWS ON UBICOM In the vision of UbiCom, a user carries a wearable terminal and a lightweight see-through display. In this display the user can see virtual information that augments reality, projected over and properly integrated with the real world. The wearable system contains a radio link that connects the user to the ubiquitous computing resources and the Internet. A camera captures the user’s environment. Camera images are sent to the backbone and matched to a 3-D description of the real world to determine the user’s position and to answer questions of the user that relate to the environment. Within the context of this future communication scenario, three different views on UbiCom exist that roughly correspond to the three projects under the UbiCom program. Figure 1: Position of UbiCom program at intersection of communications, computing, and context-aware and intelligent services II.1 Application Specific Context-Aware System Probably the most apparent level of viewing UbiCom is that of an application-specific context aware system for future generation personal wireless communications. Figure 2 illustrates this top-down concept in four steps. The sensor system records the environment, the context of the user and his/her environment is derived in the analysis phase (for instance a position recovering process), the application generates (synthesizes) an appropriate reply, which is then presented to the user in an overlay manner using sophisticated display techniques. Roughly speaking, the first two steps are intended to realize the awareness of the user’s position (where is he/she? ), the user’s attention (what is the user looking at, pointing at, thinking about? ), and the user’s wishes (what is the problem, what information is needed? ). The third and fourth step are intended to find and properly provide the user with the desired information by synthesizing, transmitting, and displaying it in a context adapted fashion. In essence the user “wears” the computing hardware, equipped with a variety of sensors and actuators, and has access to local and remote resources. Important in this view on UbiCom is that it contains two conflicting requirements. Namely, from a user interface point of view, a small latency between sensory input and displayed output us important. This would require large local computation power and memory requirements for video analysis, position determination, information synthesis, and database storage to be available in the user’s wearable device. A wearable system, however, will have to have a small form factor and low power consumption as it runs on batteries. Therefore, much of the processing and storage functions have to be placed at the wired backbone. Figure 3 illustrates th