Special Feature

Using Metadata to Control Content
Viewing on Mobile Devices
Takeshi Sasaki†, Masahito Kawamori, Jin Hatano,and Katsuhiko Kawazoe
Abstract
This article describes technology for controlling the viewing of content on cellular phones and othermobile devices. Compressed metadata transmission and metadata handling techniques are introduced ascore technologies for achieving diverse content viewing modes in mobile environments.

1. Problems specific to mobile environments
Server-based broadcasting is gaining momentumand promises to make the idea of viewing TV programs stored on large-capacity hard disks commonplace. Mobile environments are no exception, and theinfrastructure for server-based broadcasting tomobile terminals is under active discussion. The
recent surge in popularity of personal video recorders(PVRs), which combine a DVD player/recorder and alarge-capacity hard disk, is part of a larger trend
toward offering users greater convenience. Storing
programs on large-capacity hard disks frees viewers
from time constraints, letting them enjoy the program
they want to see, any time they want. Technologies
supporting such services through the convergence ofbroadcasting and communications will become
increasingly important in the future [1].
Among the technologies supporting content viewing are those for controlling content delivery andmetadata control technology. Attempts to deploy
these technologies in a mobile environment sometimes run up against problems specific to such an environment, such as narrow transmission bandwidth andlimited terminal resources. Bandwidth limitations areon the way to being resolved through a series of steps,
starting with services in the multi-Mbit/s class such as
NTT Cyber Solutions Laboratories† Present address: NTT East Corporation
Shinjuku-ku, Tokyo, 163-8019 Japan
E-mail: takeshi.sasaki@east.ntt.co.jp

Vol. 3 No. 8 Aug. 2005

HSDPA (high-speed downlink packet access) andCDMA2000 1x EV-DO, then the so-called 3.5G services representing the halfway stage between thirdand fourth-generation services, and then 4G services
themselves, which will offer a bandwidth of 100
Mbit/s, which is comparable to today’s FTTH (fiber tothe home) service. However, there are still many problems to be overcome before 4G services are implemented and doing so may take several years. Moreover, since the mainstream business model today isdesigned around flat-rate packet communications services, carriers are looking for new profit structures nottied to traffic. Another factor is that today’s mobileterminals are increasingly feature rich and used for avariety of network-based services, making it undesirable to consume large amounts of network resources
for content viewing alone. A key issue, therefore, is
how to utilize the available bandwidth efficiently.
At the same time, while advances in terminal functionality have made available more storage space andmemory for use by applications, there is also a trend
toward lighter, more compact devices with slimmercases, which tends to limit the number of applications
that can be installed. As with bandwidth, it is important to keep terminal resource requirements to theminimum necessary.
At NTT Cyber Solutions Laboratories, we havedeveloped a new content viewing control technology
for mobile environments utilizing metadata. This
technology does more than simply control content
viewing on mobile devices. By incorporating themeans to exchange metadata among individual users
47

Special Featureand by facilitating the formation of communities
around Internet sites, it enables diverse, new forms ofcontent viewing in mobile environments.
2. Using segment metadata
Metadata is data about data; that is, it is informationthat describes the contents of data. The metadata forvideo content, for example, might include the title ofthe video, its data format, information about the service provider, and the fees. There are also many otherkinds of metadata, such as that used for digital rights
management and electronic program guides.
Our technology mainly uses segment metadata in
SegmentInformationTable
SegmentList
SegmentInformation
segmentId
ProgramRef
crid
Description
Title Synopsis Genre Keyword
RelatedMaterial CreditsList
SegmentLocator

KeyFrameLocator···SegmentGroupList
SegmentGroupInformationgroupId

fragmentId · · ·ProgramRef
cridGroupType
Description
Title Synopsis Genre Keyword
RelatedMaterial CreditsList
Segments

GroupsrefListrefListKeyFrameLocator···XML (extensible markup language) format, which has
been standardized by the TV-Anytime Forum [2] for
digital broadcasting, as shown in Fig. 1. Segment metadata applies to small units of content data divided into
individual scenes. Each unit is called a segment, and themetadata for each segment is called segment metadata.
Information about a content segment is recorded ina SegmentInformationTable element. A SegmentList
element contains SegmentInformation for each segment along with a segment ID for uniquely identifying a particular segment. The metadata for individual
segments includes such information as the title, asummary, a content reference identifier (CRID) identifying the applicable content, and a thumbnail image
Description of segments
SegmentInformation
describes one segment
Main data in the description• CRID for the main content• Title of the segment• Synopsis of the segment• Genre of the segment, if
needed• Keywords for the segment• Related material describing
data about thumbnail
graphics, promotional
information, links to relatedcontent, related
merchandise, and URLs ofrelated services• Credit information about thesegment (producer, director,cast, characters)• Segment locators
describing time points of thesegment• Time points describing thekey frames in this segment
SegmentGroupInformation
describes a group ofsegments
Main data in the description• CRID for the main content• GroupType describing thetype of group (digest,highlights, etc.)• Title of the group• Synopsis of the group• Genre of the group, ifapplicable• Keywords for the group• Related material describing
data about thumbnail
graphics, promotional
information, links to related
content, related
merchandise, and URLs ofrelated services• Credit information about thegroup (producer, director,cast, characters)• Segment locators
describing time points of thegroup• Time points describing thekey frames in this group
Element

Attribute

Fig.1. Structure of TV Anytime segment metadata.

48

NTT Technical Review

Special Featureproviding a simple representation of the segment.
These are used to create a chapter list, and enable theuser to select particular segments for viewing.
A SegmentGroupList can be used to form a segment group made up of the segments indicated inSegmentList, which enables such features as digest
viewing or highlight viewing. The segments belonging to a group can be designated by listing their segment IDs in the SegmentList. It is also possible toimplement digest or highlight viewing across multiple programs by means of CRID designation.
3. Dealing with metadata bloat
As the services to be implemented using metadata
become more advanced and diverse, more and more
information is needed. Consequently, the amount ofmetadata describing that information expands in proportion to the sophistication of the services, which canresult in large files, which in turn require more transmission bandwidth. Furthermore, high throughput anda large memory capacity will be required in the terminal that processes large-scale metadata. This is a problem that cannot be ignored in a mobile environment
having limited bandwidth and terminal resources.
What is needed is an efficient way to transmit andprocess even large-scale metadata with limited resources.
Here, we discuss a compressed metadata transmission
technology as one approach to achieving this.
4. Compressed metadata transmission technology
XML-format metadata is coded in conformance witha strictly defined schema (e.g., XML Schema) enabling
seamless distribution control across various platforms.
Since this coding was originally defined to facilitateprocessing in various uses, many of its aspects have aredundant structure. These redundant parts can be compacted using a compression algorithm.
This technology is based on the BiM format (binary format for MPEG-7 data) [3] specified in MPEG7, which standardized the description method for
effectively searching for content, such as video andaudio, as shown in Fig. 2. By converting text-format
XML to binary format, we can compress metadata toaround 10% of its original size. Table 1 gives measurement results for the compression ratio based on anumber of different samples of XML structures.
These results show that XML structures other thanreal text data can generally be compressed by morethan 99%. The compression ratio of real data varies
with the content; however, in XML documents,
where XML structures make up a large part of thetotal size, compressing just these structures makes asignificant difference to the overall data size.
Since the same compression algorithm is used byboth the sending codec and the receiving codec, it is
only necessary to extract the data that needs to beconveyed and to send the compressed binary XML as
is. Then, when the binary XML is accessed at thereceiving end, there is no need to decode it to text format (XML format). Instead, the binary XML can beaccessed directly to reference and update the data, ina manner similar to how an application at the receiving end manipulates the document object model(DOM) object that is the application programming

Textual XML<TVAMain><ProgramDescription><ProgramInformationTable><ProgramInformation><BasicDescription>
Equivalent information<Title>Advertisement</Title><Synopsis>shampoo</Synopsis><Keyword>Shampoo</Keyword><Keyword>Lalasta</Keyword></BasicDescription></ProgramInformation></ProgramInformationTable></ProgramDescription></TVAMain>
XML parsing

Binary XML
encoding

XMLInformation set

Binary XML
decoding

Compressed transmission

XML Information set

Fig. 2. Binarization of XML document.

Vol. 3 No. 8 Aug. 2005

49Special Featureinterface for XML access.
This technology not only enables efficient transmission by means of XML compression, but also allows
more information to be stored in the terminal, so it isuseful for achieving effective use of terminal resources.
5. Metadata handling technology
Next, we describe a metadata handling technology
for enabling a new kind of content viewing control. It
can process a large amount of compressed metadata
at high speed in a mobile terminal and match it withpreference metadata, as explained below.
A typical mobile user is expected to view content
not in long stretches but selectively, watching only
items of interest over short time spans. To enable
users to view only desired content of a short duration
in a limited time, we must provide some means ofmatching content to individual preference attributes.

The metadata handling technology shown in Fig. 3
automatically generates this individual preference
attributes information as metadata (called “preference metadata” here), enabling playback control tailored to the individual in combination with official
metadata provided by broadcasters or content
providers. The main things this technology can beused to accomplish are as follows.
(1) Assign an index to scenes of interest and playcontent based on it.
(2) Introduce friends and acquaintances to recommendable scenes while indicating the degree ofinterest and appending comments.
(3) Publish information about recommendable
content on Internet sites, etc.
Sometimes the official metadata may define basic
services such as digest or highlight viewing of thecorresponding content, but these are viewing modes
decided by the content provider and do not necessar-

Table 1. Some examples of metadata structure compression.
Textual XML
exampledescription
Multimedia contents 1 (group)
Multimedia contents 2 (group)
Multimedia contents 3 (group)
Still region image 1
Still region image 2
Still region image 3
Video segment 1 (frames)
Video segment 2 (frames)
Video segment 3 (frames)
Video segment 4 (frames)
Video segment 5 (frames)
Still image + video frames 1
Still image + video frames 2
Still image + video frames 3

Textual XML
data size(bytes)768 3453 847
148 847
7 258
3 208
6 826
173 627
107 338
11 801
259 252695 727
8 131
12 96261 992Binary XML
data size(bytes)542
24255
151515574
353
38860
2 3154378329Compressionrate(%)
99.9
99.4
99.8
99.8
99.5
99.8
99.7
99.7
99.7
99.7
99.7
99.5
99.4
99.5

Broadcaster
Compressed metadata transmission technology

Content server
Control
metadatantnte

oof cFlowo

Control
metadata

f contewnt
FloCommunity sites Preference
Exchanging preferences metadata

User AFlowofm

TV-Anytime
metadata

eta

dat

a

Metadata server

Metadata handling technology
adateta

fmwo
Flo• Query function on TV-Anytime metadata• Metadata compression/encoding method• Binary XML parsing• Interface bridge for TV-Anytime queryUser B• Realtime generation of preference metadata• TV-Anytime metadata engine• Function for matching official metadata andpreference metadata• Preference-based content viewing• Preference metadata exchange• Dynamic generation of graphical user interface

Telecommunications

Fig. 3. Use scenario of metadata compression/handling technology.

50

NTT Technical Review

Special Feature

ily present a collection of scenes that the viewer
wants to see. The technology introduced here gives
the viewer the means to view content as he/she
desires while protecting the provider from the risk of
the content being used in unauthorized ways. This isdone by controlling the official metadata and preference metadata based on certain definitions, so that theuser can enjoy the convenience of making private useof the content within the allowed scope, without
infringing on the rights of the content provider.
In addition, preference metadata can be exchanged
among individual users or published on Web sites.
The recipient of such metadata can then view the samecontent in different ways by applying the preference
metadata received from friends. One example is viewing by popularity ranking, based on multiple sets ofpreference metadata taking into account the degree ofinterest indicated by friends or their comments about
the scenes being viewed. This will lead to interactive
services that offer new ways of viewing content.

tered on their interests. Such virtual communities tend
to form naturally around sites such as blogs, wheredetailed information is published in specific fields andusers interested in the same field gather. By exchanging preference metadata in forums of this nature, people will be able to express “points of interest” regarding various images related to a given theme, and thiswill give rise to new kinds of content uses.
To enable the various systems for achieving diverse
services like those described above, to be incorporated into mobile terminals, a variety of technologies
such as metadata compression and storage are being
devised for compactness.
7. Developing new content services
NTT Cyber Solutions Laboratories is carrying out
research and development on core technologies
applicable to various communications-based services
being undertaken by NTT, such as video delivery services and the OnQ Project of NTT DoCoMo [4].

6. Extensive service scenarios
References
Web log services, commonly known as blogs, haveadvanced remarkably in a short time. Along with services such as social networking, they provide rapidly
growing opportunities for individuals to create andpost information. These phenomena attest to thestrong desire of people to form communities, not only
ones in real society but also virtual communities, cen-

[1]

[2]
[3]
[4]

K. Kawazoe and S. Hayama, “Cooperative Broadcasting and Communication Technology for a New Way of Viewing TV,” NTT Technical Review, Vol. 2, No. 8, pp. 50-51, 2004.
http://www.tv-anytime.org/
http://www.chiariglione.org/mpeg/standards/mpeg-7/mpeg7.htm#3.5_BiM_(Binary_Format_for_MPEG-7)
http://www.nttdocomo.co.jp/new/contents/05/whatnew0311.html (inJapanese).

Takeshi Sasaki

Jin Hatano

NTT East Corporation.
He received the B.E. degree in communicationengineering from Tokai University, Kanagawa in
2002. He joined NTT Laboratories in 2003. Heworked in research areas such as technology fortransmitting MPEG-2 images encrypted by contents encoding technology and technology for
distributing advertisements that performs filtering and recommendation according to viewer
attributes. In addition he researched image contents control technology using metadata inmobile terminals, such as cellular phones andPDAs. He moved to NTT East Corporation in2005.

NTT East Corporation.
He received the B.S. and M.S. degrees in computer science from Keio University, Kanagawa
in 1994 and 1996, respectively. In 2001, hejoined NTT Cyber Solutions Laboratories, Kanagawa and engaged in research on metadata andcooperative broadcasting and communication
technology. He moved to NTT East Corporationin 2005.

Masahito Kawamori

Katsuhiko Kawazoe

Senior Research Engineer, NTT Cyber Solutions Laboratories.
He joined NTT Laboratories in 1989. He has
worked in research areas such as artificial intelligence, language processing, and interactive agentsusing speech recognition. His current research area
is metadata and rights language, media delivery forbroadcasting and broadband communications, andthe convergence of mobile and fixed communications services. Since 2000, he has been the leader ofthe Metadata Task Group of the Association ofRadio Industries and Businesses (ARIB) Working
Group for Broadcasting Systems based on a Home
Server. He is also a convener of the TV-Anytime
Forum, chairing the Metadata Working Group.

Senior Research Engineer, Supervisor, NTT
Cyber Solutions Laboratories.
He received the B.E. and M.E. degrees in engineering from Waseda University, Tokyo in 1985
and 1987, respectively. Since joining NTT in1987, he has mainly been engaged in R&D ofradio communication systems, satellite communication systems, and the personal handy-phonesystem (PHS). His specialty is forward error correction systems. He is currently a co-chairman ofARIB Working Group for Broadcasting Systems
based on a Home Server. He is a member of theInstitute of Electronics, Information and Communication Engineers (IEICE) of Japan and received
the Young Engineer Award from IEICE in 1995.

Vol. 3 No. 8 Aug. 2005

51