System support for pervasive multimedia systems

The proliferation of multimedia-capable mobile devices, such as laptops, personal digital assistants (PDA), and cellular telephones, has lend to an explosive increase of multimedia contents. With the increasing capacities of disk storage, this trend has encouraged users to create ever-larger personal digital libraries of audios, pictures, and videos. The prevalence of small and mobile devices such as sensors and RFID tags is expected to create an ubiquitous computing environment. This ubiquitous environment can provide us pervasive locationing and identification services, and further can create sensor data streams encoding object’s location and identity which are the context of media capture. The concurrence of these two trends enable a new set of multimedia systems that we refer to as Pervasive Multimedia Systems. Pervasive multimedia systems enable users to create, access and navigate large volumes of multimedia content on a variety of personal mobile devices. Designing these pervasive multimedia systems face three major challenges: (i) achieving energy efficiency for the battery-powered mobile devices; (ii) searching and retrieving media content in a fast and human-friendly manner; and (iii) identifying and locating in scalable and maintainable fashions. In this thesis, We propose techniques that provide system support for pervasive multimedia systems to address these challenges. Due to the power constraints, battery-powered mobile devices impose requirements of energy efficiency. On the other hand, the real-time nature of multimedia applications makes it challenging to trade off application's performance with energy savings. To achieve energy efficiency without sacrificing application's quality-of-service (QoS), We have developed power management techniques using application domain-specific knowledge and time-series-based models. Our approaches can reduce the power consumption of mobile devices significantly, while at the same time still meet the QoS requirements of applications. Searching and retrieving media content is greatly enhanced by the textual annotations of the context of media capture---when, where, and who/what. Many mechanisms have been developed to generate these context manually or semi-automatically. These mechanisms are error prone and have high computational requirements. To automate the generation of highly accurate context, We have designed and implemented sensor-enhanced video annotation (SEVA), a context-aware multimedia recording system. SEVA exploits sensor technology to automatically annotate the media with identities and locations of objects. SEVA can greatly enhance user’s ability in searching and retrieving media content. Locationing and identification system is a critical component in pervasive multimedia systems. Its accuracy, deployability, and cost are crucial for the success of pervasive multimedia systems. The currently available locationing systems have higher cost due to the use of expensive and battery-powered tracking sensors. To address the issues of scalability and maintainability of locationing systems, We propose Ferret, a pervasive locationing system incorporating passive RFID technology. Ferret is cost-effective, easily maintainable and deployable compared to current locationing systems.