Visual Interface－ Moving Toward the Future －September 11(Mon), 2006
Arcadia Ichigaya
4-2-25 Kudan-kita, Chiyoda-ku, Tokyo
Japan

Sponsored byIEEJ:
The Institute of Electrical Engineers of JapanIEICE:The Institute of Electronics, Information and Communication Engineers
ITE:
The Institute of Image Information and Television Engineers
HCECI:The Historical Committee of Electric, Communication and Information
130th Committee on Optoelectronics (Japan Society for the Promotion of Science)

Supported by:
JSPS:
Japan Society for the Promotion of Science
NICT:
National Institute of Information and Communications Technology

映像インターフェースの未来へ
Visual Interface－ Moving Toward the Future －2006年9月11日（月）September 11(Mon.), 2006

アルカディア市ヶ谷
Arcadia Ichigaya

東京都千代田区九段北 4-2-25

4-2-25 Kudan-kita, Chiyoda-ku, Tokyo, Japan（主催）
IEEJ (International Exchange Grant) : 電気学会
The Institute of Electrical Engineers of JapanIEICE : 電子情報通信学会
The Institute of Electronics, Information and Communication Engineers
ITE: 映像情報メディア学会
The Institute of Image Information and Television Engineers
HCECI : 電気電子・情報関連技術史委員会
The Historical Committee of Electric, Communication and Information
130th Committee on Optoelectronics (Japan Society for the Promotion of Science): 日本学術振興会光エレクトロニクス第130委員会（協賛）
Takayanagi Foundation for Electronics Science and Technolgy: 髙柳記念電子科学技術振興財団（公的支援）
JSPS :日本学術振興会
Japan Society for the Promotion of Science
NICT : 独立行政法人 情報通信研究機構
National Institute of Information and Communications Technology

ペラ++.indd 1

2006/08/29 14:42:19

Welcome Address
To support the forthcoming “information society” for the next generation of people,
who will probably obtain most of their information through electronic video, the electronic
display technology (i.e., video interfaces) of the future is likely to become a highly
important one with rich display capabilities and ultra fine detail, surpassing even that ofworks of art. Electronic display technology began with breakthroughs in television
technology and subsequently took on the role of video interfaces as computers weredeveloped. With the development of the Internet, it has come to play a crucial role insocial activities as a video interface that allows people all over the world to gather andshare information. Although much progress has been made, the telecommunications
networks of the future will probably depend on functions that contribute to fostering ashared consciousness whereby people all over the world can share their local problems
and work together for solutions and continued development. Hopefully, many
developments and innovations will continue to be made in the future towards theimplementation of user-friendly and richly featured video interfaces.
With this understanding and goal in mind, by welcoming the world’s authorities inthese and other fields, we hope to reflect on the early breakthroughs in television thatwere made in Europe, the United States, and Japan, and to work on the future
prospects of technologies aimed at video interfaces. It is said that new ideas can be
developed based on study of the past, so by reviewing the origins of electronic television,
we hope to develop stronger strategies for its future development.
We would like to thank everyone who agreed to contribute to this forum on the futureof video interfaces, and we would like to thank everyone in the coordinating committee,
for their considerable efforts in promoting and operating this forum. We would also liketo acknowledge the event’s sponsors: the Japan Society for the Promotion of Science’s
130th Committee on Opto-electronics; the National Institute of Information andCommunications Technology; the Institute of Image Information and Television
Engineers; the Institute of Electrical Engineers of Japan; the Institute of Electronics,
Information and Communication Engineers; all the members of the Electrical, Electronic
and Information-related Technology History Committee; and the Takayanagi Foundation
for Electronics Science and Technology for their help and cooperation.
September 11th, 2006

Yasuharu Suematsu
General Chair

Kazumasa Enami
Steering Committee Chair

Program
Welcome Address
9:30-10:00 Opening Ceremony
Steering Committee Chair : Kazumasa Enami (NHK)Plenary Talk

General Chair: Yasuharu Suematsu (NII)

10:00-12:30 Invited Session
Theme : Early Works of Video Display Research
Session Chair : Eiju Matsumoto (SHMJ)・ Early Developments of European Television
Oskar Blumtritt (Deutsches Museum)・ Early Works in Hamamatsu
Hidenori Mimura (Shizuoka Univ.)・ A Visual Vision －100 years and Counting －Joseph A. Flaherty (CBS)12:30-13:30

Lunch

13:30-15:50 General Session

Session Chair : Haruo Okuda (Shonan Inst. of Technology)
・ “Super Hi-Vision” future television
Masayuki Sugawara (NHK)・ Visual Content Production and Digital Technology
Mitsuru Kaneko (Tokyo Univ. of Technology)
・ Free Viewpoint Television
Masayuki Tanimoto (Nagoya Univ.)
・ Color Reproduction in the Image and Video Display “ Beyond RGB ”
Masahiro Yamaguchi (Tokyo Inst. of Technology)
・ Electronic Paper : Display for Comfortable Reading
Makoto Omodani (Tokai Univ.)

15:50-16:10 Coffee Break
16:10-18:30 Panel Discussion
Theme : Visual Interface toward the Future
Session Chair : Shinji Ozawa (Keio Univ.)
Panelist
Kenkichi Tanioka (NHK)Yuichi Matsushima (NICT)
Minoru Etoh (NTT DoCoMo)
Kazuo Okamura (Panasonic)
Shunsuke Kobayashi (Tokyo Univ. of Science, Yamaguchi)

18:30-

Reception

Committee Member

Plenary Talk

“Visual Interface: From the Beginning into the Future”
Yasuharu Suematsu
National Institute of Informatics
2-1-2 Hitotsubashi, Chiyoda-ku,
Tokyo, 101-8430

Introduction
The development of telecommunications networks has made it possible to reachhuman beings anywhere in the world. Video display devices - in other words, videointerfaces - are playing an increasingly key role in bridging the gap between networks
and people. These interfaces are evolving in diverse ways, not only to assist in theacquisition and recognition of data archived in databases throughout the world, but also
to offer human beings new ways to connect across large distances.
In this forum on the future of video interfaces, we will start by looking back at the
breakthroughs in electronic video devices that occurred with the starting point of videointerfaces: television technology. Next, we will review the current state-of-the-art videointerface technology, which is currently being developed for video display devices suchas computers and television receivers. We will then discuss how video interfaces arelikely to develop in the future.

Television breakthroughs
A long history of breakthroughs has accompanied the development of electronic
technology for televisions, starting with the invention of the cathode ray tube by KarlFerdinand Braun in 1897. This was followed in 1907 by Lee De Forest’s invention of
the triode, which made it possible to amplify electrical signals at high speed. The
invention of electronic television systems required two breakthroughs: imaging
technology and electronic display technology. The first televisions used a mechanicalsystem developed by John Logie Baird (1925); this system was used for trial broadcasts
in the following year. Analytical background was also shown by Herbert E. Ives.
Meanwhile, trials and patent applications on the use of cathode ray tubes (the prototype
of modern electronic television sets) as electronic display devices were conducted byresearchers all over the world; these researchers included Vladimir K. Zworykin, Philo
Farnsworth, Kenjiro Takayanagi, Alexander Douvillier, Max Dieckmann, Camille A.
Sabbah, Charles Francis Jenkinns, Boris Rosing, Grabovsky, Reginald S. Clay, Robert C.
Mathes, Frank Gray, Cambell Winton, et al. During this time, Takayanagi, who was an
assistant professor at Hamamatsu Technical High School, had been conducting researchon television and had been placed an ordered for a prototype cathode ray tube by his

1won design with Toshiba Co. (1924). Takayanagi pointed out that mechanical televisionsystems would only be capable of limited picture quality, and that it would be necessary
to develop electronic systems that would be capable of overcoming this drawback. He
claimed that he successfully displayed an electronic video image of a Japanese character
called “i” on a cathode ray tube system (1926). It is said that Gray demonstrated a
television system. And Farnsworth applied for a patent on an imaging tube anddemonstrated the use of a cathode ray tube to display video images (1927). Zworykin
had been made constantly breakthrough by Braun tube display and developing a
excellent imaging tube. In this way, the development of television came about due topioneering work by researchers in Europe, America, Japan, and elsewhere. Their efforts
laid the foundations for the development of electronic television and electronic video
display equipment. TV broadcasting eventually began in the 1930s.

From video display devices to video interfaces
Computer technology underwent considerable development in the 1940s, and in 1950
the Whirlwind computer was developed at MIT. This computer incorporated a humaninterface based on a cathode ray tube. Further breakthroughs included the invention of
the computer mouse connected to a computer display screen (1957) and Alan Kay’s
work on personal computers (1970). As a result, electronic video display devices cameto play an important role in interactions between humans and machines.

Solid-state technology and flat-panel displays
The invention of the transistor (Bardeen, Brattain and Shockley; 1947) was followedby the development of large-scale integrated circuits (LSIs, 1958) and microprocessors(1972). As a result, electronic equipment became smaller in size, consumed less power,
and became more reliable.
Progress was also made in the field of video tape recording (VTR), and significant
progress was made in the scale of memory capacity based on the implementation ofvertical magnetic recording and due to the development of technologies such as
semiconductor memory, ferroelectric memory, atomic-force-prove memory, andlarge-scale buffer memory.
With the appearance of liquid crystal displays (Heilmeier and Realte; 1968), compact
and lightweight low-power panel display devices were incorporated into wristwatchesand electric calculators, and later became widely used in portable television receivers
and personal computers. Other display technologies have subsequently appeared,
including plasma displays (PD), organic electroluminescent displays (OELD), fieldemission displays (FED), projection displays, and light-emitting diode displays (LED).

2

These technologies have made it possible to use electronic displays in a wide variety of
applications.
Imaging devices have also become smaller and more stable due to the appearance ofsolid-state technologies such as charge-coupled devices or CCDs (Boyle and Smith;1970) and high-speed complementary-metal-oxide-semiconductor devices or CMOS
device. Solid-state imaging devices have made video capture devices much simpler andmore durable. The field of photography has been revolutionized by the appearance ofdigital cameras based on this technology, which is now being used widely in mobile
telephones.

Advent of super high definition TV
Traditional National-Television-System-Committee (NTSC) standard andPhase-Alternation-Line (PAL) standard television pictures consisted of 500–600 scanlines. This was followed by the development of high definition television with 1,080
lines. Later advances in electronic display technology led to the development of Digital
Cinema systems with 2,000 lines and Super High-Vision Television systems with 4,000
lines; the picture definition of the latter technology rivals that of printing technology
and works of art. Progress has also been made in three-dimensional video technology,
which allows the creation of three-dimensional virtual spaces, and efforts have been
made to implement completely new virtual spaces in which human beings can interact
with three-dimensional video objects. Unlike the traditional media of handwritten andprinted material, this electronic video display technology has the potential to provide
societies with new ways of working with information. The latest display technology,
Super Hi-Vision, produces displays that surpass the quality of ordinary printed pages.
Video display technology has also entered the domain of the expressive power of movie
films, where it is used to display scenes in three dimensions (including the passage oftime). The attraction of electronic video technology is in its ability to intertwine thevideo being projected with the people that are watching it, and there are many latentpossibilities in the applications of four-dimensional virtual spaces depicting the realworld. The spread of this sort of electronic display technology is likely to lead to thedevelopment of many new fields in the future. For example, it could be used inapplications such as the four-dimensional design of assembly lines together with theflow of objects in a factory.
Super High-Vision displays, for example, also offer rich capabilities both in terms ofpresenting information and in terms of providing quality entertainment. Like the opera
glass effect, users can individually view a large amount of information in the details ofvideo content they are interested in by enlarging it either with eyeglasses or with anelectronic eyeglass effect. In addition, this technology is expected to have a significant

3effect on e-learning, and a wide range of fields by bringing such benefits as promoting
the development of video-supported surgery, enhancing the performance of intelligent
robots by increasing their field of view, and ensuring the safety of public streets bymeans of closed circuit cameras.

Future of network development and video displays
The ARPANET network, which was started in the United States (1968), later
developed into the Internet with the spread of personal computers in the 1980s. This
globally extensive telecommunications network has been supported by the developmentof technology for communicating large volumes of traffic over large distances. Later,
the growth of the World Wide Web from SERN (Tim Berners-Lee; 1989) gave rise to an
era of information explosion by revolutionizing the functions for exchanging
information and the means of sharing it between people in the world. Great advanceswere made in search engine technology, which grew into a giant business. Mobile
telephones also underwent rapid development and became highly popular items,
furthering the global spread of telecommunications networks.
In the world’s telecommunications networks, video interfaces are widely used as coretechnologies that bridge the gap between humans and networks. The development ofelectronic video technology has been brought about through the combination of a wide
range of fields in science and technology, including fundamental physics and chemistry,electronics, photonics, broadcasting, telecommunications network technology, and data
storage technology. The driving force behind the technological breakthroughs and thegrowing popularity of ultra high definition large screen displays has been the necessityof making revolutionary developments in systems that can provide content withoutstanding entertainment value, such as television broadcasts, and this is likely toremain the driving force in the future. In addition to making it possible for consumers to
get high quality video in an economical way, these developments have brought spillover
effects to other aspects of society, such as education and medicine. For
telecommunication networks to fulfill their potential benefits to society, it is alsoessential to have functions that contribute to fostering a shared consciousness whereby
people all over the world can share their local problems and work together for solutions
and continued development. Video interfaces should continue to develop to achieve thisimportant goal.

4

Invited Session

Early Developments of European Television
Oskar Blumtritt, Member, IEEE
Deutsches Museum
Museumsinsel 1, 81669 Muenchen, Germanyo.blumtritt@deutsches-museum.de

Abstract

1.

The early history of television technology has threedifferent roots. First, as soon as the first telegraphsystems came into being in the 1840ies some experts
started with experiments in order to transmit
facsimiles of texts and still images. Second, since the1870ies mainly physicists investigated the relations
between light and electricity and came up with somedevices to convert light into electricity and vice versa.
Third, the first patent for transmitting moving pictures
was applied for in 1884, partly recurring on
mechanic-optical traditions such as the fantascopes
and praxinoscopes presented at fun-fairs.
But only after the establishment of soundbroadcasting in industrialized countries around 1925,
did the idea of developing broadcasting of moving
images start to gain acceptance in engineering
circles. At that time, well known electromechanicaland optomechanical components were used in orderto develop an effective television system. Despite thevisions of cultural and economic values of an
omnipresent, and even color television system, thetechnical development of the scanning, transmitting,and reproducing sets proceeded hesitantly. Different
cultural and political orientations of thetelecommunications industries, combined withdifferent regulations of the telecommunications
markets and with the competition amongst theindustrialized nations, led to differences in theirdevelopment strategies.
These heterogeneousdevelopments continued when it became obvious thatmechanical components had to be substituted byelectronic devices. Due to the political situation in
the 1930ies, the establishment of a ‘first regulartelevision program’ dominated the European scene,
while in the USA the development of electronic
components won the overhand. Especially the radardevelopment, that is above all the dealing with veryhigh frequency systems, influenced the war and postwar technology of television.

Introduction

Stories about television often start with the legendthat a young German student of physics andmathematics sat alone in his little room in Berlin on
Christmas eve in 1883. He yearned for his girl friend
and thought about of how to communicate with her.
Being acquainted with some knowledge ofphysiological optics and electrical physics, he planned
to scan the flickering light of a candle by using arotating disc with little holes in it. The light passing
through these holes would be converted into anelectric current which could be transmitted. On thereceiving side the current had to be reconverted by aglow lamp into light of different intensity. A
synchronously rotating disc would eventuallyreconstruct the image of the candle light on a screen.
The young student, Paul Nipkow, successfully applied
for a patent of his so-called electrical telescope on
January 6, 1884. Nipkow never constructed such atelescope, and for the next forty years scarcely any
scientist, engineer, or industrial manager wasinterested in developing sets or systems fortransmitting moving pictures.

Figure 1: Advertisements and news on large screens,
as suggested by Albert Robida in his book: Le
Vingtième Siecle, 1883. (Deutsches Museum)5But writers, such as the prominent French science
fiction author Albert Robida, had already depicteddifferent scenes of how to apply television and picture
telephony for communication, entertainment, andadvertisement – without, of course, knowing
Nipkow’s patent specification.
The path from an idea to its realization apparently
is not the way in which the history of television
technology can be approached. Instead, we have toconcentrate on the intertwining developments ofscience and technology by integrating their economic,
political, and cultural presuppositions and effects.
Therefore, we have to investigate communications
and media technologies as far as they paved the way
to create systems able to scan, transmit, and reproduce
moving pictures. Concomitant with this, we have toinquire as to which political/economical and cultural
reasons forced or hindered the development of
television. Moreover, we have to start with theestablishment of the electric telegraph system in the
1840ies where the idea arose that it should be possible
not only to transmit digital signs but also analogue
images by means of electricity. But, as it is wellknown, the electric telegraph, later accompanied by
the electric teletypewriter, remained the most widelyused communications means until the middle of thetwentieth century when the telephone received acomparable acceptance. Fax machines developed intoa widely used equipment only in the late 1970ies, due
to an increasing demand in Japan and to the followinginternational efforts in standardization.
Picturetelephones – as we recently could experience – were
only accepted in the form of mobile phones.
As regards television, we can observe a different
situation because of the overlapping developments incommunications and – as we call it today – media
technologies and their respective cultures. The
initially unexpected rapid acceptance of soundbroadcasting since the mid-1920ies led industrial andpolitical circles to take into consideration thattelevision broadcasting could produce added value.
Although efforts in both communications and media
technologies were genuinely oriented towards global
enterprises, local, national, and international
competition influenced the spread and the shape of
television systems.
The adaptation of different
systems of color television in the 1960ies might serveas an example.
Tracing the early history of television in Europe –the central topic of this paper – has to reflect on bothglobal and local aspects as well as on the tension
betweentechnical
feasibilities
andactualestablishments of television technologies and systems.
In doing so, we will recur to technical and historicalliterature as well as to archival materials and artifacts
which have survived. The Deutsches Museum has alarge collection of television components and sets
which can exemplarily serve as material sources andas means of illustration.

2.

Some roots of image transmissionin Europe

In 1844, the painter Samuel F. B. Morse finished thefirst public-use electric telegraph line between
Baltimore and Washington in the USA. Despite
various developments in other countries, especially incentral and West-Europe, his telegraphy system
would become the model from a technical point ofview all over the world. One year earlier, in 1843, theScottish clockmaker Alexander B. Bain, interested inapplying electric phenomena to various technicalfields, had suggested using his electrochemical
telegraph also for the transmission of the characters of
the alphabet. The raised parts of the characters could
be mechanically scanned in lines and converted into
electric impulses. Bain emphasizes in his patent
specification that ‘telegraphed’ copies can be madefrom any image as long as it is constructed out of
conductive and non-conductive segments.

Figure 2: Bakewell’s picture telegraph as of 1849 –with tin-foil covered cylinder C, clock mechanism M,
metal stylus T. (Deutsches Museum)6

In the following years, several scientists andtechnicians, mostly from Great Britain, France, orItaly, tried to improve Bain’s facsimile transmission.
Some of them used metallic cylinders or plates
whereby texts or line drawings were placed by a nonconductive ink on the transmitter side. Well-known
were the apparatuses of the Scottish physicist
Frederick C. Bakewell and of the Italian Abbé
Giovanni Caselli.
Although one could receive
facsimiles over a distance of about 200 km in theearly 1860ies there were economically unjustifiable
restrictions due to mechanical components: the slow
speed of scanning and the faults because of imperfectsynchronization. As we shall see later, similar
technical problems arose during the early
development of television.
Although Caselli’s ‘pantograph’ was used inseveral European countries and the French Post Officeeven installed lines for its public use in the late
1860ies, the public did not accept this quite expensive
communications service. Therefore the French Post
Office discontinued the service after a few years. One
also has to take into account that both the professional
and amateur photography was well established at thattime and had influenced the public’s visual
perception. The digitally operating picture telegraphs,
which could not transmit the graduations of graytones in photographs, seemed to be unable tocontribute to the often expressed needs for ‘distantvision’.
The vision of ‘distant vision’ was enforced for ashort time when the British telegraph expert
Willbough Smith discovered in 1873 that the electric
resistance of selenium could be influenced byexposing it to different light intensities. Smith was
looking for a material of high electric resistancewhich he thought to have found in selenium. But itturned out that the behavior of selenium – an element
known since 1818 – was unstable due to light effects.
However, European and US-American scientists
immediately became interested in investigating
selenium because this topic fitted into the then widely
discussed relationship of light and electricity. And
most of them speculated about the use of selenium forconverting light into electricity for distant-vision
technologies. But material investigations of selenium
in different physical states – from amorphous to
crystalline – could not entirely overcome one
characteristic of selenium, i.e. that the resistance only
slowly returned to its initial value when the lightsource was removed. Nevertheless, physicists as wellas technicians produced a great variety of selenium

cells and developed new schemes for transmitting still
and moving pictures. Some of them suggested amosaic of small selenium cells at the transmitter side
whereby each of these cells were connected by a wire
to a light emitting cell on the receiver side. To thisgroup belonged the English physicist and telegraphy
expert William Edward Ayrton, who had held aprofessorship at the Imperial College of Engineeringin Tokyo in the mid-1870ies.
Although such circuits were in accordance withphysiological investigations on the human opticalnerve system and avoided the problem ofsynchronization, they caused difficulties in finding
appropriate light-emissive elements and enormous
costs in wiring. Others, beginning with the FrenchlawyerConstantin-Louis
Senlecq,
preferredmechanical scanning systems, for instance with aselenium stylus.
But the reproduction of thetransmitted signals remained a problem here too. In
this context the already mentioned efforts of Paul
Nipkow from 1883, or of the French industrialist
Lazare Weiller from 1889, can be seen. Weiller had
transferred Nipkow’s idea of synchronized rotating
discs into synchronized rotating mirror wheels. Eachof the mirrors on the perimeter of the wheel was
deflected in a different angle in order to scan andreproduce the entire picture.

Figure 3: Mirror wheels were replaced by mirror
screws (left) from 1927 on. (Deutsches Museum)The list of names of individuals involved in thesolution of detailed problems of distant-vision
technologies in the second half of the 19th century
could be enlarged to a great extent. Although many
components were developed in laboratories nearly7none of the suggested television systems – as wewould call them now – was really constructed andtested. Their inefficiencies were too obvious, notonly on a technical but also on an economical level.
Nevertheless, the culturally established visionary ideaof distant vision, especially in the form of picture
telephony after Alexander Graham Bell’s telephonebecame worldwide known around 1880, continuously
encouraged scientists and technicians to overcome
some of the deficits that hindered a development on alarge scale. This kind of modern thinking was
legitimated by large sections of the population andsupported by a development of the natural sciences,which easily related phenomena or effects found in
the laboratory to devices which could be oftechnological use. The other way round, that is thescientific use of experiences made with technicaldevices and systems, proved to be effective in thesame way. Along with this, there was an international
exchange of scientific ideas, especially amongst thewestern nations. At this stage of the development of
distant-vision technologies, the restrictions recurring
to protection by patent were of minimal repercussion.
The industries were not very enthusiastic about newcommunications technologies; but this implied alsothat they were not willing to sponsor these activities
to a great extent – only as far as some of thecomponents might have promised advantages in othertechnical fields, too. Finally, media technologies suchas photography and cinema, which would become
industrialized in the late 1890ies, were consideredseparate enterprises.

3.

service, and the public press envisaged thetransmission of moving pictures as the next step inmedia technologies in the very near future.
Nonetheless, scientists and electrical engineers weremuch more cautious in their predictions.
The English physicist Shelford Bidwell publisheda short article in the reputable scientific journal‘Nature’ in 1908, where he presented his estimations
on the requirement of receiving an image of 5.5 cm x
5.5 cm: One would need to transmit at least 150,000
image points ten times per second.
The
synchronization of these 1,500,000 operations and therespective frequency band needed could not be
obtained by the means available at that time. The
alternative, that is using several rows of sensitive
selenium cells separately wired, would require about
120 line wires. Both systems, so his conclusion,
would be highly impracticable – and extremely
expensive. This statement, which was a response to
the optimistic public press, shows on the one hand,that ideas of the technologies for television systems
had not changed in the minds of popular writers. On
the other hand, this kind of dealing with traditional
ideas on television expressed the crucial problems ofsuch systems more precisely than it could have been
observed in the late 19th century. Therefore, mostly
academically trained people who were interested intechnical developments tried to convert physicalinstruments (or their knowledge of it) into technicallyusable devices. One of these instruments was thecathode-ray tube which would become a central partof television systems.

Aiming television systems

The term ‘television’ was coined by the Hungarian
Constantin Perskyi in the title of his paper given at theInternational Electricity Congress in Paris in 1900.
The German version of ‘television’, i.e. ‘Fernsehen’,already became known to the public via the book ofRaphael Eduard Liesegang ‘Beiträge zum Problem
des elektrischen Fernsehens’ in 1891. In these works
nothing entirely new was said about television
technology; but the authors summed up the state of
the art in the late 19th century. In the first decade of
the 20th century, the efforts to develop televisionsystems changed insofar as they were accompanied by
the establishment of a reliable picture-telegraphsystem, which allowed the transmission andreproduction of half-tone photographs. The picture
telegraphy became widely used as a newspaper

Figure 4: Scheme of the Braun tube in a physicaljournal, 1897. (Deutsches Museum)The cathode-ray tube had its origins in theinvestigation of the effects of partly evacuateddischarge tubes in the second half of the 19th century.
One could find these kind of tubes in nearly all
physical laboratories because they were regarded as acentral instrument in researching the relationship, or
the nature respectively, of negative and positiveelectricity, magnetism, polarization of molecules, and,8

eventually, the construction of matter.
Suchexperiments resulted, for instance, in thedetermination of the charge, mass, and velocity of theparticles of cathode rays by the English physicist
Joseph John Thomson in 1894. The concept ofelementary charges, named electrons, was widely
accepted since then, and led to a variety of new
investigations of the discharge tube. The tube, whichThomson used, consisted of a long, closed glass tube.
On the one end, a cathode and an anode provided theenergy so that the cathode rays could travel throughthe tube. Guided by metal apertures and deflecting
electrodes, the cathode ray could finally be madevisible on a fluorescent screen on the other end. The
German physicist Ferdinand Braun was able todevelop this tube into a standard measuring
instrument by slightly, but effectively varying someof its components. So the tube made its way from a
specially designed physical device to a commercially
produced instrument for various purposes. Braun wasespecially interested in measuring and visualizing
alternating currents by avoiding the then usedsluggish mechanical components. For that reason hewidened the fluorescence screen of the tube, andinstead of deflection plates he employed coils byapplying their magnetic effect.
Jonathan Zenneck, an assistant of Braun, utilized aslightly varied form of this tube for his measurementsof radio circuits from 1899 on. It soon became clear
in scientists’ and engineers’ circles that the (nearly)
inertia-free measuring instrument could be used forvery high frequencies. The German physicists Max
Dieckmann and Gustav Glage applied for a patent in1906 in which they described a method oftransmitting hand-written letters and line drawings bymeans of cathode-ray tubes. Whereas the so-called
Braun tube (Braunsche Röhre) was used on thereceiving side, the scanning was still donemechanically. One year later, the Russian physicist
Boris Lvovich Rosing was the first to suggest
integrating the tube as a receiver into a televisionsystem. He successfully applied for a German patentin 1908 and probably built his first televisionapparatus in 1911 – with the assistance of Vladimir
Kosma Zvorykin.
Although most of the physicists and engineers
were hesitant in developing television systems
because of reasons mentioned above, it soon becomecommon sense that inertia-free components at thescanning and reproducing side could at least solve
some of the problems. This had already been true forthe use of the so-called Kerr cell – invented by the

Scottish mathematician John Kerr in 1875 and appliedin picture telegraphy since the late 1890ies. The Kerrcell could directly vary the intensity of light by being
influenced in its polarization by electrical signals.
And, of course, some suggestions and experiments
were made to employ the (nearly) inertia-free Kerrcell in television receivers. As far as it is known, thefirst suggestion to produce an all-electronic televisionsystem was made by Alan Campbell Swinton in 1908.
Campbell
Swinton,a
Scottishautodidact,enthusiastically experimented with all kinds ofcathode-ray tubes, including x-ray tubes and Braun’soscilloscope tube, in his private laboratory since themid-1890ies. The pessimistic article of Bidwell madehim think about solutions of how to proceed in orderto receive a practicable television system. In a letterto Nature, published there only two weeks after
Bidwell’s article, he stated that the development of
the cathode-ray tube into a reliable receiver would notcause any serious problems. Two orthogonally placed
coils should provide the varying deflection of thebeam. A transmitter, with the receiver synchronized,
had to be of the same construction. Here somedifficulties might be encountered relating to thevariation of the electrical current in harmony with thechanges of the intensity of light and to the speed ofthis action, i.e. 160,000 variations per second inminimum.
None of the known opto-electricalcomponents would be able to perform such a rapid
action, but he was confident that ‘something’ would
be discovered in a not too distant future. However,
his own efforts undertaken in this field over the next
years with a Braun tube, which he had obtained from
Germany, did not result in any decent solution for theconstruction of an electronic camera. But he soon
came up with a suggestion of how a Braun tube mightwork as a camera: The image of a scene was to be
projected through a lens on the screen of the tube
where it would first traverse a metallic gauze screen,
then pass through a gas, and finally meet a mosaic ofphoto-electrically active cubes consisting of metals
such as rubidium. The cathode ray of the tube had toscan the backside of these metallic cubes. In the casethat the cubes were illuminated, the cathode ray
would pass through the ionized gas and meet themetallic gauze and be led to the receiver. Campbell
Swinton proposed an image of 200 lines, the line scanbeing at a rate of 1,000 Hz and the frame scan at a rateof 10 Hz.
This far-sighted suggestion of an all-electronic
television system was presented to the RöntgenSociety (nowadays the British Institute of Radiology)9

in 1911 and published in its journal one year later.
But there did not exist any industry which would havebeen able to manage all the problems such asproviding a sufficient vacuum technology, producing
effectively working photoelectric cells, or supply thenecessary transmission lines and amplifiers. The
electrical industry, for instance, in nearly every
industrialized country had just started to develop
electronic components in the form of rectifying andamplifying tubes to be used in telephone andradiotelegraph circuits. Mass production of suchelectronic devices was established during the FirstWorld War. Some investigation in material research
was done by the German teachers and scientificinstrument makers Julius Elster and Hans Geitel fromabout 1890 to 1920. They came up with some newamalgams, such as of alkali elements, which led to the
production of reliable photoelectric cells . Their work
also provided a scientifically based photoelectricmeasurement. The German professor for technicalphysics, Arthur Rudolph Wehnelt, successfully
improved the Braun tube during the first five years ofthe 20th century. Well known and widely used sincethen were the oxide coating of the cathode and theimplementing of a negatively charged cylinder, thelater so-called Wehnelt cylinder, around the cathode.
This allowed lower voltages for operating the Brauntube and a quicker control of the cathode ray.

well as the material sciences, belonged to the prime
movers. During this process the physical sciences
radically changed and, as it is well known, created
new research fields such as the theory of relativity andthe quantum theory. In communications technology,
radio and telephony, including sub-domains such as
high-frequency technology and filter design, became
the preeminent branches in both research andeconomy. The vision of television still existed in themind of the public and of some individual engineersand private inventors. They even got patents forsuggesting technologies of color television, forinstance. In industry, such kind of development andresearch was neither technically nor economically
legitimated. The political control over technologicaldevelopments, performed either by the military or bynational Post Offices, also enforced the extension of
traditional and the building of first of all reliablecommunications systems.

4.

Mechanical versus electronic
television

In the 1920ies, the discrepancy between scientific andengineering circles on the one hand, and the public
and private inventors on the other hand, continued
insofar as a realization of television systems wasconcerned. Although some components and circuitdesigns now seemed to perform some features ofimage transmission better, some crucial problems still
existed. So it is not surprising that mostly men, whowere not closely attached to academia or industry,tried to develop television systems. One of the mostprominent men in this field surely was the Scotsman
John Logie Baird. Being a gifted tinkerer withelementary background in mechanical and electricalengineering and with some experience gained in anelectrical power company, he tried to earn his moneywith inventions and the trade of unusual things –ranging from borax-sprinkled undersocks to ‘Speedy
Cleaners’. Although he was not unsuccessful, he did
not become a wealthy man, because his activities
were often interrupted by long phases of illness. In
the winter of 1922 he became interested in television
by reading some visionary articles in specialist
publications. He was wondering why no one had yet
produced a television system, which seemed to be aneasy task.
Baird’s first patent was filed in Great Britain in
1923. It describes the scanning with a Nipkow disc

Figure 6: Wehnelt cylinder around the cathode (right).
(Deutsches Museum)
Those and similar developments originated in aclosely interwoven research in science and technology
whereby the radio science and radio technology, as

10and the reproducing of the image with an array ofsmall incandescent lamps. The conversion of lightinto electricity was obtained by a common selenium
cell and the transmitted electrical signals were
distributed to the respective lamps by a metal arm
rotating around a ring of contacts. This television set,
operating with 18 lines and 20 frames per second,
might just have produced very crude images. And it
did not contain any new elements. But Baird
managed to give a demonstration of his set to thepress in early 1924. The following article in the Daily
News about this first public demonstration of
television brought him the first sponsor and somemore interest from the public press, too. He now
developed his system by using serrated Nipkow discs
of up to 2,000 revolutions per second. Additionally,
he surmounted the problem of the sluggishness of theselenium cell by providing circuits, which added tothe output current of the cell the first differential ofthis current. Herewith he obtained electrical signals
which were in their shape very close to those of the
optical input signals. Further-on, the synchronism
between transmitter and receiver was supported by asimple spindle with which the receiver could be tuned
manually. Baird also mentioned the replacement of
the selenium cell by a cell with a colloidal fluid, but
he did not give any details. Nevertheless, there werestill serious problems to overcome, such as the very
poor signal-noise ratio.

Television Ltd., was founded by Baird together withassociates in 1925. Baird did not try to hire renowned
engineers or scientists; most of the technical work
was done by himself and stayed on the level ofmechanical television on both the transmitter andreceiver side. Of course, the company tried to expand
its networks nationally and internationally. Although
the demonstrations of Baird’s ‘real’ television were
mostly appreciated by the press in other countries,
too, the demand for his system with still crude images
was not well received by the relevant circles in GreatBritain and not accepted by most of the companies inother countries, which tried to compete with Baird’s
system.
Especially, the British Broadcasting
Corporation (BBC), which was converted into a nonprivate incorporation in 1927, did not show muchinterest in the poor performance of television until
1932. Only the British General Post Office provided
some support by allowing the Television Ltd. to use
its transmitter lines for several tests from 1928 on.
This resulted, for instance, in the experience that it
was of advantage to transmit image and sound on
different frequency bands. In 1929, the BBC agreed
to participate in just one of the tests, but at theexpenses of the Television Ltd..
Although Baird’s system laid down the basis for areliable mechanical television broadcasting system inGreat Britain in the mid 1930ies, the competition withother companies and the competition with electronic
television systems – or just the use of some electronic
components – led Baird and his company to lose theirrole as the prime mover in television. Further-on,
Baird’s many anticipations as regards the possible
development of television – for instance, nightlighttelevision, phonovision, two-way television, color
television, and large-screen television – and hissuggestions of how to solve the respective problems
were successively regarded as ineffective sidelines.
Others, such as the Hungarian mechanical engineer
Dénes von Mihály, who had moved to Germany andwas supported in his television work by the electricalcompany AEG-AG from 1924 on, came up withsimilar systems, but he concentrated on practical,
reliable television transmission. All this indicates thattelevision was increasingly seen as an earnest but notvery profitable enterprise, especially in addition tosound broadcasting, in most of the industrializedcountries at the end of the 1920ies. Half-heartedly
supported by the management, renowned engineers
such as the Swedish emigrant Ernst Frederik
Alexanderson at General Electric in the United States,
were engaged in varying components and circuits in

Figure 7: Baird’s television apparatus, ca. 1925.
(Science Museum)
To make it short, the technological andeconomical development in accordance with a steadypublic propaganda would continue in the same styleuntil the 1930ies, i.e. even after the first company, the11order to achieve a better control of light sources andsynchronization. At another US-American company,
the American Telephone and Telegraph Company(AT&T), a short but intensive research resulted indemonstrations of television from 1927 through 1930,
that revealed both the maximum qualities and thelimitations of mechanical television. The research
was done in the famous Bell Laboratories, generously
financed by their mother company.
The well-known limitations of the mechanicaltelevision systems encouraged some individuals tostart with investigations in electronic televisionsystems. Zvorykin, who had assisted Rosing – asalready mentioned – with the development of atelevision receiver tube in 1911 and who hademigrated to the USA after the Russian Revolution,
experimented on an electronic camera from 1923 on.
While his initial ideas – although patented – did notwork out in practice he was required by the research
department of Westinghouse, where he was
employed, to develop cathode-ray tubes for different
purposes, including television.
David Sarnoff,
General Manager of the RCA (Radio Corporation ofAmerica) which was responsible for the research andmarketing of radio equipment of General Electric andWestinghouse, eventually encouraged Zvorykin toconcentrate on the development of his iconoscope
which should become the most widely used television
camera tube from the mid-1930ies on. Others, suchas the autodidact Philo Taylor Farnsworth in the USA
or the engineering teacher Kenjiro Takayanagi inJapan, had devoted their life to an all-electronic
television system. They came up with similar ideas
and components; eventually, these three gifted
inventors exchanged many of their ideas despite theindustrial and national competition that was involved.
But coming back to the situation in Europe, one
has to note that Zvorykin had been asked by Sarnoff
to visit France before he could develop his
iconoscope. This indicates that in France much work
was done on improvements of the features of cathoderay tubes, i.e. especially of oscilloscopes. Allelectronic television in the narrow sense was not oneof the primary research topics in Europe. To
understand this situation, the development of Manfredvon Ardenne’s first so-called all-electronic television
set shall serve as an example.

Figure 8: Zvorykin with his iconoscope. (Deutsches
Museum)5.

All-electronic telecinema in
Germany

In Germany, like in other European countries, thetraditional communications industry was hesitant ininvesting in a research program oriented towards
television systems. But the Imperial Post Office,regulating and operating the communication
networks, showed interest in and supported such
efforts already from 1927 on. It kept an eye oncompetitive developments, especially in Britain andthe USA, and favored a practical mechanicaltelevision system rather than a visionary electronic
one, which did not seem to be feasible in the near
future. In 1929 the German Post Office and theGerman broadcasting authorities agreed on adoptinga, at that time, rather low scanning standard: 30
horzinontally scanned lines and 12.5 pictures per
second; the aspect ratio being 3:4 (vertical/
horizontal). The industry, dependent on the orders
from the Post Office, had to keep this standard to alarge degree. At the Berlin radio fair, the showcase of12the communications industry and the Post Office, thefollowing mechanical television systems were on
display in 1929:
Karolus-Telefunken, Mihaly’s
Telehor, a joint project of R. Bosch AG, Radio-DS
Loewe, and Zeiss Ikon (under the name of the justfounded company Fernseh AG), an experimental setof the Post Office, and Baird’s devices.
Siegmund Loewe, who ran a small radio company
and held a share in the Fernseh AG, encouraged
Manfred von Ardenne to think about electronic
television in 1930. Ardenne, a gifted tinkerer andautodidact in physics and chemistry, had established aprivate laboratory for scientific electronic instruments
and had made an accommodation with Loewe to
produce devices in mutual interest because of his
debts. His specialties at that time were oscilloscopes
and even oscillographs as well as tubes and circuits
for the entire new fields of ultra-high frequency andbroadband amplification. So Loewe’s demand did notseem a big challenge for him. Additionally, theproduction of ‘electronic’, i.e. ‘inertia free’, technicalsystems fitted the current ideology of progressivenessand modernity which the young Ardenne desired to bepartaking in, due to his status as an outsider in scienceand (industrial) technology.
Ardenne had just to combine some of thecomponents fabricated in his laboratory in order toachieve a set, which he thought would be a fullelectronic television system. On the transmitter side amains connection fed a cathode-ray tube which
produced a small, bright light spot. The light went
through a lens and a transparent slide and finally met
a low-inertia photoelectric cell. The cell converted
the intensity of the light into respective electricalsignals, which were amplified by a broadband
amplifier and transferred to the cathode-ray tube on
the receiver side. Two saw-tooth generators wereconnected in parallel to the deflection plates of thetwo cathode-ray tubes. This provided synchronization
as well as 25 images of 100 lines per second.
Ardenne had oriented himself on the quality ofcinema films which he saw in accordance with thephysiological investigations in visual perception.
Additionally, his television set allowed the electricaltransmission of the then produced cinema movies atfairly high quality. Ardenne’s ensemble met the highend of the unofficial television standard inengineering circles in 1931. The demonstrations of
his set were internationally appreciated.
It isinteresting to learn from contemporary commentaries
that emphasis was laid on the advantages first of thereceiver and afterwards of the transmitter, whichbecame worldwide famous as ‘flying spot scanner’.

Figure 9: Announcement of the presentation ofArdenne’s flying spot scanner, 1931. (Deutsches
Museum)The images on the receiver side initially impressed
television experts and authorities because of theirbrilliance, which was mostly due to the relatively hightransmission standard used. The experts soon realized
this and, animated by emphatic articles in theAmerican press, they adopted the American term
‘flying spot scanner’ and praised the fully electronictransmission system for television purposes.
Nevertheless, Ardenne did not reach his initial goal,
i.e. to demonstrate that his system would have thepotentiality to successively introduce better and better
television standards. His system proved itself to be
useful only for scanning movies, not for live shooting.
The Zvorykin-type of camera tubes and, to a certain
extent also the Farnsworth-type of tube, wereintroduced in Germany from the mid-1930ies on.
Both types used a storage process before scanning theimage with the cathode ray and provided thepotentiality for technological advancements andeconomical advantages. The scanning method ofArdenne was used for the transmission of movies
until the 1980ies.

13While the German industry had to importtelevision camera technologies it developed, on the
one hand, mechanical television systems and, on theother hand, various procedures to improve the qualityof television images, such as the line-jump scanning
by Fritz Georg Schröter of the Telefunken company.
Being aware of the fact that the European industry
could not economically compete with the USAmerican companies, some European companies,
especially in Great Britain and Germany, developed
variations of Zvorykin’s iconoscope and came up withthe supericonoscope in 1934. In Germany, theintermediate-film process was introduced in 1932, i.e.
live shooting with a movie camera, immediately
developing the film, and scanning it electronicallywith a flying spot scanner or mechanically with
Nipkow discs. The economical competition between
companies on national and international levels wasoverlapped by political competition of nation-states.
Germany and Great Britain mobilized their technicaland structural resources in order to present the first
regular public television program.
Both states
claimed to be the winner in 1935 but the smoothtransition from test to regular transmissions let suchstatements seem absurd. While industries and Post
Offices tried to push the development of television
after 1935, in Germany, for instance, the ministry of
propaganda pushed the cinema because of thetechnically better quality of the images. Although the
production of an inexpensive receiver similar to the‘Volksempfänger’ started in 1939, the Second World
War stopped such undertakings. But the developmentof television was not stopped at all during the Second
World War. Television became important for militarydevices such as guided missiles. Additionally, theresearch in radar technology enforced the research on
higher and higher frequencies which in turn improved
the quality of television transmissions.

6.

Britain even insisted on its low 405-lines norm. The
same was true when the different color-televisionsystems came into being (NTSC in the USA, 1954;
SECAM in France, 1957; PAL in Germany, 1963).
Summarily stated, Europe could hardly compete with
the innovations coming from the USA. Videotechnology, stereo transmission, or satellite
transmission in the 1950ies and 1960ies, respectively,
may serve as examples. Another competition for
Europe arose in Japan. Sony was the first company toequip receivers with transistors, and European
countries oriented themselves towards Japaneseefforts in video technology and high-definitiontransmission, for instance. Philips, a global player –as we would express it today – situated in the
Netherlands, and which had introduced the newcamera tube ‘plumbicon’ in 1965, was the only
European company able to compete with America andAsia in the long run.

REFERENCES
Oskar Blumtritt is senior curator at the Deutsches
Museum, Munich, Germany. He is responsible for thetelecommunications and microelectronics departments. His
research covers the history of electrical science andtechnology.
[1]. Abramson, Albert. The History of Television, 1880 to
1941. Jefferson, N.C. and London 1987.
[2]. Blumtritt, Oskar. The flying-spot scanner, Manfred vonArdenne and the telecinema. Presenting Pictures. Ed. by
Bernard Finn. London 2004, pp. 84-115.
[3]. Burns, Russell W. Television – an international historyof the formative years. London 1998.
[4]. Chandler, Alfred D. Inventing the Electronic Century –The Epic Story of the Consumer Electronics and Computer
Industries. New York etc. 2001.
[5]. Fisher, David and Fisher, Marshall. Tube – The
Invention of Television. Washington, D.C. 1966.
[6]. Goebel, Gerhart. Das Fernsehen in Deutschland bis
1945. Archiv für das Post- und Fernmeldewesen 5 (1953),
pp. 259-393.
[7]. Hugill, Peter J. Global Communications since 1844 –Geopolitics and Technology. Baltimore, MA and London
1999.
[8]. Takayanagi, Kenjiro. Pioneering Television – The
autobiography of Kenjiro Takayanagi. Translated by
Mayumi Yoshida. San Francisco, CA 1993.

OutlookAfter the Second World War, all industrialized
nations were able to establish public televisionsystems in black and white and they did it
successively in accordance with their economical andregulatory conditions. Whilst the USA had agreed ona 525-lines norm already in 1941, Germany
introduced a norm of 625 lines in 1948 and France anorm of 819 lines in the same year. In 1951,
international efforts were undertaken to standardize
the global enterprise television, but they failed. Great14

Early Works in Hamamatsu*
Hidenori Mimura
Research Institute of Electronics Shizuoka University
3-5-1 Johoku Hamamatsu 432-8011 Japan
mimura@rie.shizuoka.ac.jp
Abstract

graduation, Dr. Nakamura, who was the Dean of
the Department of Electrical Engineering ofTokyo Technical College, gave graduates a
discourse. The discourse stimulated him to devotehimself to pursuing one important research theme
for the next 10 to 20 years. At that time radio wason the eve of coming into home use, following theappearance of the wireless telephone. He did notchoose radio as a research theme, and was excitedby the consideration of how scenes could be
reproduced at a great distance, in real time, frommoment to moment. He coined the term “wirelessdistance electric vision” to describe what heenvisaged at that time. One day he found a
cartoon involving future television in a Frenchelectrical magazine, as shown in Fig. 1. In the
cartoon, a screen mounted on a box like a radio
set was drawn, with a girl singing on the screen.
The cartoon was titled “Télévision”. He
understood that, what he named “wirelessdistance electric vision”, was called “television”
in France. He felt that television was already inthe experimental stage, and he made up his mind
to choose television as a theme to be pursued forthe future. He had already heard the news that
Baird was also working on television in GreatBritain, Mihaly in Austria and Jenkins and otherresearchers in the U.S., but all their systems weremechanical. He racked his brains to determine inwhich direction he should proceed. At last hecame to realize that the mechanical system of
television was not suitable and would be
extremely hard to realize, if a high definition
image was to be pursued. It was in the autumn of1924 that he decided to work on an all-electric
television system instead of a mechanical one.
When he made this decision, he had alreadyenvisaged the use of electron beams in both thetransmitter and receiver.

Prof. Kenjiro Takayanagi joined Hamamatsu
Technical College and commenced work on
television in 1924. From the beginning he decided
to work on an all-electric television system, using
electron beams in both the transmitter andreceiver. He devised the first television receiving
Braun tube in 1925, however he failed to
manufacture his design of pick-up tube because ofpoor vacuum technology. He compromised anddecided to use a Nipkow disk in the transmitter.
Following his inventions relating to televisionsynchronization and deflection methods, it was on
December 25, 1926 that he succeed in
reproducing Japanese character “ イ ”. The
experiment was carried out using 40 scanninglines and a television picture rate of 14 per second,and it constituted the first success in electronic
television in the world using a cathode ray tube
(CRT). Prof. Takayanagi invented, in succession,a wide band amplifier, an accumulating system,
and an electric interlace system. In 1935 hesucceeded in manufacturing the iconoscope as anaccumulating-type pick-up tube. At last hesucceeded in reproducing highly detailed pictures,using 245 scanning lines, a television picture rateof 30 per second and a field rate of 60 per second.
It was on February in 1936 that he accomplished
the al1-electric television system which he had
initially envisaged.

1. Start of the Work on “All-electric
Television System”
In May 1924 Prof. Kenjiro Takayanagi joined
the Department of Electrical Engineering ofHamamatsu Technical College, (which later
became the Faculty of Engineering, Shizuoka
University) as an associate professor andcommenced work on television with theunderstanding and assistance of President
Sekiguchi of the College. He graduated from the
Department of Electrical Engineering of Tokyo
Technical College in April 1921. At the time of

2. Concept of the First Pick-up Tube
and
Failure
in
its
Trial
Manufacture
Prof. Takayanagi conducted investigations and15

3. First Braun Tube

surveys concerning selenium and selenium cells,
because the photo-conductive effect was wellknown in selenium cells. He acquired basic
techniques for using thin selenium films,
transparent metal films and electron beams.
Making use of these techniques and materials hedevised his first pick-up tube as shown in Fig. 2.
This pick-up tube corresponds in its designconcept to the modern vidicon. The first pick-up
tube was manufactured in the laboratory ofHamamatsu Technical College, but his vacuum
technology could not cope with this challenge.

Prof. Takayanagi modified the Braun tube
which was used in physical measurements anddevised his first television receiving Braun tube as
shown in Fig. 3. His first Braun tube featured (1)the use of thermal electron emission, (2) anelectron flow control electrode, (3) horizontal andvertical deflection plates arranged in square toeach other, and (4) gas focusing of the electron
beam. Figure 3 shows a schematic of his firstBraun tube, and a picture of the first trial Braun
tube. This was manufactured in the autumn of1925 with the cooperation of Dr. Asao at the
Research Institute of Tokyo Shibaura Electric Co.
(later Toshiba Electric Co). This was the firstBraun tube of this shape in the world.

Fig. 1 Cartoon of the television of the future in a
French electrical magazine (1923).

Fig.3 First television receiving Braun tube (1925).

4. Compromise System

Fig. 2 First pick-up tube designed by Prof. Takayanagi(1924).

The Braun tube was now at hand, but
experiments on television could not proceed forwant of the pick-up tube. To continue with his
research, he decided to use the Nipkow disk in thetransmitter until a suitable pick-up tube became
available. He called this combination acompromise system. An experiment was
successfully carried out, using 40 scanning lines
and a television picture rate of 14 per second. His
inventions relating to television synchronization

16

and deflection contributed to the success of theexperiment. Figure 4 shows the operating
principle of his invention. His idea consisted of
automatic generation of horizontal and verticaldeflection voltage with the correct waveform in
the receiver and included the generating in thetransmitter of a horizontal synchronizing pulse
after each scan and a vertical synchronizing pulse
after each sweep. These two pulses from thetransmitter were sent to the receiver tosynchronize the deflection generator. These
synchronization and deflection methods still form
the basis of the modern television receiving
system. The Japanese character “イ” appeared on
the screen using 40 scanning lines. It was on
December 25 in 1926 that he achieved the firstsuccess in electronic television in the world, using
a cathode ray tube (CRT). The picture of theJapanese character written on the mica plate of
the transmitter is shown in Fig. 5.
In the years following this success, thetransmitter was improved. By increasing thephotosensitivity, success was achieved in thetransmission of a human face or hands by early
1928. The pictures 6, 7, 8, and 9 show thetransmitter, video signal amplifier, receiver, andreceived pictures, respectively.
Nipkow disk (Sending side)
photocellFig. 5 Picture of the Japanese character “イ” written on
the mica plate of the transmitter (1926).

Braun Tube (Receiving Side)

lamp
holes for synchronization
holes for image

circlessec

Fig. 6 Transmitter of the television system (1926-1927).

saw-toothed wave-form deflector signal

sec

sec

Fig. 4 Operating principle
synchronization method (1926).

of

the

television
Fig. 7 Video signal amplifier of the television system(1926-1927).

17

Fig. 8 Receiver of the television system (1926-1927).

Fig. 10 High vacuum, multi-electrode Braun tube
(1929).

6. Imperial Inspection of Television
Demonstration(a)

In May 1930 the Emperor visited Hamamatsu
Technical College, and Prof. Takayanagi washonored with the Emperor’s inspection of thedemonstration of his television unit. Figure 11, 12,
and 13 show pictures of the television receiving
set, the received picture and the transmitter,
respectively. After the demonstration to theEmperor, his electric television researches wererecognized and significantly supported by NHK(Japan Broadcasting Corporation).

(b)
Fig. 9 (a) Real pictures, and (b) images reproduced on
the receiver (1926-1927).

5. Invention of the Television BraunTube
To eliminate various defects of the gas filled
Braun tube, Prof. Takayanagi invented ahigh-vacuum, multi-electrode Braun tube, as
shown in Fig. 10. He asked Dr. Asao forcooperation in the trial manufacture of thishigh-vacuum, multi-electrode Braun tube. Trial
Braun tubes were manufactured in two types withphosphor screen diameters of 15 cm and 30 cm.
With this invention it became possible to increase
the anode voltage from 240 V, used in the gas
filled Braun tube, to 2,400 V at a stroke. This
improvement produced a bright and distinct
image.

Fig. 11 Receivers of the television system. 15cm (left)and 30cm (right) Braun tubes (May 1930).

18

operated at 100 scanning lines and 25 pictures per
second. As a result, a vivid image, with fine detail,was reproduced by this equipment. He shouted“hurray!” in spite of himself. The basic wide band
amplifier circuit is shown in Fig. 14.

Fig. 14 Basic circuit of the wide band amplifier (1930).

8. Accomplishment of the all-electric
television system

Fig. 12 Images reproduced on the receiver (May 1930).

8-1 Invention of an Accumulating System
Great strides has been made in the research anddevelopment of the Braun tube and video
amplifier so that it became possible to receive
high definition pictures, but the problem with thetransmitter could not be solved so long as themechanical system, such as the Nipkow disk, wasused in the transmitter. To solve this problem,
Prof. Takayanagi resumed his work on the pick-up
tube in 1930. He compared the human eye withthe conventional transmitting system. As a result,
he realized that, whereas the human eye received
the total flux from the picture being sent at all
times, the television system did not use the total
flux from the picture, but only the flux from each
picture element. Appreciating this concept, hedevised an accumulating system that utilized thetotal flux from the picture at all times as in thehuman eye. The principle of the accumulating
system is shown in Fig. 15.

Fig. 13 Transmitter of the television system (1931).

7. Wide Band Amplifier
High definition television requires a wide band
amplifier of the electrical signal derived from an
optical image. In 1928, Prof. Takayanagi set towork on the wide band amplifier. Afterpainstaking experiments and trial manufactures, awide band amplifier was, at least, completed in
1930. In the course of experiments he came toappreciate that the stray capacitance, present inparallel with the anode resistance in the amplifier,attenuated the signal in the high frequency range.
He reduced the stray effect and anode resistanceto a minimum. As a result, a wide band amplifierthat operated over an adequate frequency range
was brought about, although the gain per stage
was sacrificed to some extent. This wide band
amplifier was tested in the television set which

8-2 Success in the Trial Manufacture of the
Accumulating-type Pick-up Tube
Prof. Takayanagi heard the that Dr. V. K.
Zworykin in RCA of the U.S. had invented theiconoscope and succeeded in producing highly
detailed pictures from outdoor scenes by the useof the iconoscope. He visited Dr. Zworykin at
RCA and saw the iconoscope in the summer of1934. When he returned to Japan from his trip to
the U.S., he and his research associates redoubled
their efforts to develop the accumulating-type

19

pick-up tube. On October 10 in 1935 theysucceeded in the trial manufacture of theaccumulating-type pick-up tube, producing highly
detailed pictures. Figure 16 shows a schematicand a picture of the iconoscope. Figure 17 shows
the first reproduced image, obtained from theiconoscope camera. The experiment was carriedout, using 220 scanning lines and a television
picture rate of 20 per second.
To improve the picture quality, especially toeliminate flicker noise, Prof. Takayanagi invented
the electrical interlaced scanning system. At last
he succeeded in producing highly detailed
pictures, as shown in Fig. 18, using 245 scanninglines, a television picture rate of 30 per secondand a field rate of 60 per second. It was on
February in 1936 that he accomplished the
all-electric television system which he hadenvisaged from the beginning.
In August 1937, Prof. Takayanagi and his
associates moved to the Technical ResearchInstitute of NHK and undertook a variety ofexperiments in preparation for the television
broadcasting of the Tokyo Olympic Games in
1940. In May 1939 an experimental television
broadcasting station was set up in the TechnicalResearch Institute of NHK. Test pictures werebroadcast at 441 scanning lines and 25 pictures
per second, using the interlaced scanning system.
Prof. Takayanagi described in his manuscript that
he felt proud of this experimental television
broadcasting station which had been built andoperated, solely by the Japanese.

Fig. 15 Principle of the accumulating system.

* The original of this article was written by K.
Takayanaji on November 8th, 1981. The author(H. M.) revised the original manuscript byreferring to the following:
1) K. Takayanagi, “Research and development of
all electric television system”, 1981. The original
was own by the Foundation of the Promotion forElectronics
inHamamatsu(Hamamatsu
Denshikougaku Syoreikai).
2) K. Takayanagi, “Terebi Kotohajime” inJapanese, 1981 Yuhikaku Publishing Co.
3) “History of the television technology inHamamatsu” in Japanese, edited by theFoundation of the Promotion for Electronics inHamamatsu(Hamamatsu
Denshikougaku
Syoreikai) 1987.

Fig. 16 Iconoscope manufactured in Hamamatsu
Technical College (1935).

20

Fig. 17 First image reproduced from the iconoscopecamera (1935).

Fig. 18 Image reproduced with all-electronic televisionsystem, 245 scanning lines, a television picture rate of
30 per second and a field rate of 60 per second (1936).

21

A Visual Vision
100 Years and Counting
Presented at
The Visual Interface Forumin
Arcadia Ichigaya
Tokyo, JapanBy
Dr. J. A. Flaherty FIEE FRTS
Senior Vice President Technology
CBS Broadcasting Inc.

Tokyo, Japan

September 11, 2006

23

A Visual Vision
100 Years and Counting
A Visual Vision! Vision is the art of seeing that which will be, and vision and imagination
are essential to all engineering and scientific enterprises. In 1936 David Sarnoff, thenPresident of RCA said: “Progress in technology calls for imagination of the highest orderand the courage to follow where that imagination leads.” Such vision, imagination, andcourage brought television to its present state of excellence. Today, television is man’s
window on his world, and, indeed, throughout the World more people watch television thanare literate.
It all began with a vision over 100 years ago in 1875, before the advent of televisiondevelopments, when Victorian cartoonist Robida foresaw the large, wide screen HDTV set in
the English home. Figure 1 shows a Victorian family, in the comfort of their living room,
watching a distant colonial war. Indeed Father is even equipped with a microphone with which
he can shout interactive encouragement to the Imperial forces.
Today, television has become such a wide screen, high definition, and interactive reality as
foreseen in 1875.
This reality began in 1911 when Campbell-Swinton proposed an all-electronic televisionsystem. By 1926 John Logie Baird introduced his Nipkow disc mechanical scanning systemof 30 lines, Figure 2, and he called it high definition television. In 1927, H. E. Ives
demonstrated a 50 line TV system, and so it all began.
At the same time, electronic television was beginning to emerge with the Audion triode tube,
patented by Lee Deforest in 1907 enabling very weak signals to be amplified. Following thisbreakthrough in signal processing, Farnsworth introduced the image dissector, the firstelectronic camera tube in 1927, Figure 3.
Yet, not all foresaw the successful growth of television. A journalist in 1930 in a trade magazine
entitled “Television” wrote:“The over-enthusiastic televisionists are making their big mistake in thinking thattelevision will repeat the glamorous history of radio broadcasting, when every
sign indicates that it will not and cannot. Conditions are now altogether different
from what they were ten years ago. Today we have a Federal Radio Commission,an aggravating patent situation, an over crowded ether, an overabundance ofradio factories, a lot of politicians with radio axes to grind, and worst of all, asophisticated buying element that has been spoiled by high quality radio
broadcasting and high quality talking motion pictures.
If not for the ‘talkies’, the present crude televisors might stand a slight chance, as
the mere novelty of a sight-and-sound combination would be enough to sell a lotof radio apparatus. However, the ‘talkies’ have entirely erased this possibility.”

224

Such doubters notwithstanding, television developments pushed on undisturbed!
The work of Zworykin in developing the all-electronic television system using the newkinescope enabled RCA to demonstrate a 120-line system in 1932 followed by NBC in 1934
with a 343-line television transmission demonstration. In 1936 the BBC introduced a 405-line
service and called it HDTV.
By now, the high definition ball was beginning to roll with the introduction by Zworykin of theimage iconoscope, Figure 4. Later the 1940 orthicon, and the 1946 image orthicon, Figure 5,
with its great improvement in sensitivity and resolution it became widely used by broadcasters in
the US and in Europe.
In 1950 Philips developed the photoconductivity based Vidicon tube and later the Plumbicon.
Year-by-year, the continuum of technological advances increased the size of displays and thequality of the monochrome pictures.
The monochrome television world was doomed when CBS and RCA developed color televisionsystems in the 1950s. When RCA developed a color kinescope or CRT, having a matrix ofpositioned red, blue and green colored phosphor dots combined with the RCA 525 line color
television system, color TV was launched. This RCA system was standardized as the US NTSC
color standard in 1953, and this standard has been firmly embedded as American color TV for 50
years; enabling the growth of the color TV set market to reach all the 109 million televisionhouseholds in the USA.
Thus, in 1935 high definition was 343 lines, in prewar England it became 405 lines, by the 1939
New York World's Fair it was 441 lines. In postwar America it became 525 lines and wascolorized as NTSC in 1953. In latter day Europe it became 625 line PAL and SECAM color, andeach of these quality steps was introduced as "high definition television".
Among most noteworthy modern developments were Video Tape by Ampex in 1956, thetransistor, solid-state electronics, VLSI components, the VCR, the Trinitron display, andElectronic News Gathering (ENG) in 1972. Developments in digital video, digital image
sensors, digital VTRs, and digital TV and HDTV enabled today’s all-digital broadcasting.
Today digital technology dominates radio and television broadcasting, computers, the Internet
and virtually all electronic systems. Analog broadcast equipment is being relegated to museums,
and yet digital techniques in communications are not new. In fact, in 1623, Sir Francis Bacon, in
his treatise: "The Dignity and Advancement of Learning", proposed to encode the alphabet by abinary communications system. He suggested that:“Provided only that the matter included be five times less than that which
includes it, without any condition or limitation, the alphabet can be
resolved into two letters only, which by repetition and transposition
through five places could represent all the other letters of the alphabet.”

325

With his five-bit "byte", he could compress and encode 32 different characters, or the letters of
the alphabet, and he concluded:"The contrivance shows a method of signifying and expressing one's mind
to any distance by objects that are either audible or visible, provided onlythat the objects are but capable of two differences; e.g. fireworks, bells, orcannon."
Today it is two differences of ones and zeros, but hats off to Sir Francis Bacon.
In short, high definition has always been, and will always be, the best quality achievablewith a given state-of-the-art. HDTV is always the best, not the second best, not the third
best, and certainly not the previous best. Today, digital, wide screen HDTV has escalated
television quality to a plateau never before imagined.
More importantly, there has never been a significant improvement in television technologythat has not become a part of everyday American life, and HDTV is just the latest “musthave” technology in viewers' homes, and so it will likely be in Europe and around theWorld.
The accelerating pace of television technology has not only served broadcasting, but has
spawned a host of competitive delivery systems such as cable, direct-to-home satellitedelivery systems (DTH), package video, VCR, DVD, TIVO, etc - many without the
spectrum constraints of broadcasting. The broadcasters' monopoly of video channels tothe home is gone, and gone forever. The television of abundance places broadcasters inan ever increasingly competitive environment.
So, as we evaluate tomorrow's digital TV and HDTV systems and plan for theirimplementation, we must bear in mind that today's standard of service enjoyed by theviewer will not be his level of expectation tomorrow. Good enough is no longer perfect,
and may become wholly unsatisfactory. Quality is a moving target, both in programs andin technology. Our judgments as to the future must not be based on today's performance,or on minor improvements thereto.
Today television is 1920/1080/16:9 HDTV as first devised and developed in Japan, andintroduced in North America by NHK and CBS on February 7, 1981. Figure 6 pictures
the key participants in this first US demonstration of full HDTV, and includes motion
picture director, Francis Ford Coppola and Dr. Takashi Fujio – a principal NHK
developer of HDTV.
This HDTV system, with its common image format of two million pixels, wasstandardized by the International Telecommunications Union (ITU) in 1999, as
Recommendation BT.709 and it became the single worldwide high definition production
and program exchange standard. Without this ITU BT.709 high definition production
and international program exchange Recommendation, the rapid growth of HDTV would

4

26

have been severely delayed and confused. Thus, without the ITU, there would be no
BT.709 high definition Recommendation, and no global HDTV standard.
The International Telecommunications Union (ITU) is a vital link in the development andimplementation of communications technology. Among the advantages of global ITU
communication Recommendations are:• Easier information transportability through the definition of interfaces and protocols
among system elements, leading to interoperability and interconnectivity;• Reduced costs for operators and users of broadcasting systems through economies ofscale in capital and operating costs, including training and maintenance;• Increased flexibility and functionality for users through user equipment and interfaces
capable of operating across a number of broadcasting and other systems;• Easier access of produced programs to the international market.
Following a host of HD tests and demonstrations in the 1980s, the FCC began a rulemaking
process that would make DTV and HDTV the broadcast standard of the United States. TheFCC Chairman, Alfred Sykes, launched the project in saying:"I understand the concerns of those who believe in an incremental,
step-by-step progression toward full HDTV, but pursuing Extended Definition
options would tend to maximize transition costs for both industry andconsumers. Stations presumably would need to make a series of sequential
investments, as they inched toward full HDTV. At the same time, consumers
almost certainly would be confused, and would probably resist buying
equipment which, in relatively short order, might be rendered obsolete."..."TheFCC cares enough about broadcasting and the service it provides the public towant the very best - full HDTV - and not some incremental solution to thisformidable challenge.”
America had a goal – full HDTV at the top of its visual hierarchy. Multi-step schemes"sneaking up" on HDTV as a final goal, were seen as unwise, a waste of time and money,
and would do great disservice to the viewing public who would have to buy several versionsof “enhanced” receiving equipment, and struggle through years of "evolution" on the way toreal HDTV.
Darwinistic evolution had no place in HDTV! America would transition directly to HDTV
in a single giant step.
The FCC, on Christmas Eve, December 24, 1996, approved the digital ATSC DTV andHDTV system as developed by the FCC Advisory Committee on Advanced Television
Service (ACATS). The FCC decision allotted a second 6 MHz television channel to eachanalog TV station, but this spectrum was to be for the transmission of both digital SDTV
and HDTV and was not limited to SDTV only. Thus, America future-proofed its digital
HDTV transition.

5

27

The wisdom of this FCC dual SDTV and HDTV decision provided the maximum utilizationof the limited broadcast television spectrum so vital to the very concept of terrestrial overthe-air broadcasting and, thus, future proofed American digital broadcasting.
Spectrum, spectrum, spectrum; terrestrial broadcasting is all about spectrum, and America is
now well supplied with the spectrum to support full HDTV broadcasting for every TV
station in the Nation.
As of today, 1,566 of a total of 1,640 local television stations have completed thetransition to DTV and HDTV transmission. 99.99% of the 109 million American
television households in all 211 U.S. markets can receive at least one HDTV signal, and
91% of the TV households can receive five or more HDTV signals.
Four American TV networks, CBS, NBC, PBS, and UPN, broadcast all their scripted
primetime entertainment programming and all major sports programming in full
1920/1080/16:9/24p and 60i high definition.
Two networks down-convert the 1920/1080/16:9/60i primetime entertainment programs
to an intermediate quality of 1280/720/16:9/24p and 60p, a format with less than one
million pixels.
Figure 7 shows the current CBS weekly HDTV broadcast schedule. These programs
include all the scripted primetime evening entertainment programs, both the daily latenight talk shows, a daily daytime serial drama, and virtually all major sporting events.
All of them broadcast in full HDTV of 1920/1080/16:9/24p and 60i in accordance with
the ITU-R Recommendation BT.709.
Figure 8 lists the live sporting events covered by CBS Sports with most in 1920/1080/60i
HDTV. Many of these have been broadcast in HDTV for the last five years.
Including primetime, daytime, and late night programs, and sports, the broadcast
networks produce over 7000 hours of original high definition programming per year.
Moreover, such HDTV programs have an important after-broadcast sale in the expanding
national and international HDTV syndication markets where SDTV “enhanced” program
sales are fading.
The coverage of local ENG news in 1920/1080/60i HDTV is now growing rapidly. For
example, this year CBS is converting 17 of its 35 owned TV stations to HDTV news withan initial purchase of 100 HD ENG cameras to be followed by 450 HD ENG cameras
next year with all related editing and transmission equipment. A growing number of
independent local stations and other networks are also making the transition to 1920/1080
HDTV news as SDTV news begins to vanish.
Cable systems are also moving heavily into HDTV. Today, there are 67 million cable
subscribers in the USA, and the cable systems to which they are connected are equippedto carry high definition programming. These television households actually receive their
628

local terrestrial SDTV and HDTV television programs by cable. In these cable homes
70% of the viewing time is devoted to programming from the four major TV networks.
As shown in Figure 9, Cable TV subscribers in the USA can receive 1920/1080 HDTV
programming from 35 cable channels that originate and distribute over 1,900 hours ofhigh definition programming each week.
The two major satellite-to-home systems provide a total of 57 channels of 1920/1080 HD
programming, and typically operate 20 hours per day, for a total of over 2000 hours of
HDTV programming-per-week.
Taking broadcast, cable and satellite distribution together, and recognizing that not all
cable channels listed are available to all consumers and that many programs distributedby satellite are also distributed by cable, the American viewer has over 2500 hours-perweek of high definition news, sports and entertainment programs available and thisHDTV content is increasing at an accelerating rate.
Consumer electronics companies are also in synchronism with the growth of HDTV.
With the plethora of HD programming becoming available, the consumer electronics
industry has made great strides in expanding the consumer DTV and HDTV receivermarket.
The annual unit sales of integrated HD receivers and the projections through 2009 areshown in Figure10. There were 9.8 million DTV/HDTV units sold in 2005 and 18.2
million sales are forecast in 2006.
It is forecast that the market penetration will rise to 26% in 2006, and to 83% by 2008 in
final preparation for the 17 February 2009 shutdown of the NTSC analog service.
In short, consumer digital DTV receiver sales are booming, and among other things theboom is in HD receiver sales. This is due to the strong consumer appeal for 1920/1080
high definition quality, and to the steady drop in the sales prices of HD receivers, as
shown in Figure 11.
Figure 12, shows that the revenue earned by consumer equipment manufacturers on thesale of DTV/HDTV receivers in 2005 was $17 billion with a steady forecast growththrough 2009 to $33.2 billion, as the HD receiver market penetration nears 100%.
It is interesting to compare this DTV/HDTV growth with the corresponding decline in thesales of analog receivers, with the crossover point having taken place in 2003.
A similar pattern is seen in Figure13 with the growth of DTV/HDTV unit sales and thedecline in the analog unit sales, the revenue crossover point occurring in 2006.
Another factor is the appeal of flat panel displays such that kinescope receivers are
disappearing from the market. At the 2006 Consumer Electronics Show in Las Vegas,
729there were over 100 flat panel HD models shown in sizes up to 102 inches with only few
small screen CRT sets.
Television is HDTV, and HDTV is flat panel displays. Broadcasters and manufacturers,
adapt!
Today HDTV is alive, well, and growing in Japan, the USA, Australia, Brazil, Canada,
Mexico, and in many other nations. Standard Definition television, or SDTV, isrumbling on into oblivion and will be completely replaced by high definition.
However exciting this HDTV broadcasting rollout is, it is not the whole story. More
competitive HD delivery systems are coming and coming fast!
The onset of the high definition DVDs in the BlueRay and HD-DVD formats will bringthe viewer full 1920/1080 high definition quality, equal to, or better than, the best
broadcast HDTV and far outclassing any digital SDTV or intermediate 1280/720p
broadcasts.
Added to this DVD high definition competition is the recent announcement by both Sonyand Microsoft that they are upgrading their new Play Station 3 and X-box 360 gamesystems to full 1920/1080 HDTV.
Keeping pace with this trend, the large full quality 1920/1080/60i&p flat screen displays
now commercially available will sweep the HD consumer market and HD displays will
no longer be low-pass filters as they have been until now. Finally, full HDTV canactually be seen, thus increasing the quality gap between full quality HDTV and all lesser
formats!
In addition to the above, a growing number of programs are being distributed to the homeor to mobile viewers through cellular telephony, pod-casting and by Internet streaming,
thus adding to the total content distribution market, and tightening the terrestrial
broadcasters’ competition for viewers. Moreover, before long these new distribution
media will also be able to transmit high definition content.
Further, we are at the threshold of even greater television developments in Large Screen
Digital Imagery (LSDI) with its D-Cinema versions, and beyond these lie ExtremelyHigh Resolution Imagery systems (EHRI) at 3840/2160 with 8 million pixels and7680/4320 with 33 million pixels.
The digital revolution is a momentous event in the history of our industry. As H.G.
Wells put it:“The past is but the beginning of a beginning, and all that is and has been,
is but the twilight of the dawn.”

830

Such is the state of television art and science today. The tomorrows of television
technology are now up to you. The Visual Visions of television’s future rest withthis new generation of scientists and engineers worldwide.
As these new engineers and scientists drive the inexorable march of visual technology,
and the miracle of television quality throughout this 21st century, I commend you to the1940 counsel of Dr. Alfred Goldsmith of RCA in his treatise on Engineering Ethics when
he wrote:“Engineering is a profession, not a trade. Engineers are members of an
intellectual fellowship rather than of a competitive commercial group. As
highly trained professionals over a long period of years, the knowledge ofthe past has been placed before us in carefully written textbooks,
painstakingly prepared lectures, the facilities of universities, and theexperience of great engineers and organizations. Our education, our lust
for learning, and our dedication to excellence brings to us a highprofessional responsibility, a responsibility well beyond that of everyday
trade and business disciplines.”
We, finally, need to treat television engineering as more than a business, or it will
become nothing more than a business.
As you pursue this engineering responsibility in furthering the art and science of
television in all its forms, remember in your work the observation of the philosopher
Santayana:“We must welcome the future, remembering that soon it will be the past,
and we must respect the past, remembering that once it was all that washumanly possible.”

9

31

10

32

11

33

12

34

13

35

14

36

15

37

16

38

General Session

"Super Hi-Vision" future television
Masayuki Sugawara
NHK Science and Technical Research Laboratories
1-10-11 Kinuta Setagaya-ku Tokyo 157-8510 Japansugawara.m-fq@nhk.or.jp

generation television and started research on every
part of its broadcasting chain in 2002 [4].

Abstract“Super Hi-Vision” is the nickname of an ultrahighdefinition television system with 4000-scanning-line
video and 22.2 multichannel sound. NHK isdeveloping it with the aim of creating a broadcasting
system that conveys a strong sensation of presence.
This paper briefly describes its concept, history, R&D
status, and human factors.

2. Image format of Super Hi-Vision
When we started our research, we faced a Chicken
and Egg dilemma about the image format. That is,unless we created the equipment, we could notconduct the experiments that would give us data to
determine a suitable image format. Yet without adecision on the image format, how could we make theequipment? Finally, we decided on a preliminary
image format by taking past research into account.
Table 1 lists the major specifications of the Super HiVision signal format. The pixel count is 7680 x 4320,
which provides a viewing angle of 100 arc-degrees
horizontally when the viewer watches the screen at
the standard viewing distance. Here, the standard
viewing distance is defined as the shortest distance atwhich the scanning line structure is unrecognizable bya person with 20/20 vision.

1. Introduction
Increasing the number of scanning lines was one of
the most important issues during the early stages of
television development because viewers found thatearly TVs with fewer scanning lines were
unsatisfactory. In fact, when a 200 scanning line
system was developed in 1932, it was called "highdefinition". After 525 and 625 scanning line systems
were developed, they became standards that havelasted for several decades [1].
The research on HDTV started at NHK’s Science andTechnical Research laboratory about 40 years ago [2].
Since then, broadcasting engineers all over the world
have worked to make HDTV a reality. Each piece oftechnology and equipment along the broadcast signal
chain had to be developed anew, and psycho-physicalinvestigations of HDTV’s effect on viewers andinternational standardization activities to make HDTV
more effective and efficient had to be undertaken. As
a result of these efforts, we are now witnessing theevolution from SDTV to HDTV.

Table 1: Signal format of Super Hi-Vision
Parameter

Value

Picture rate

60 frames/sec

Scanning

Progressive

Sample per active line

7680

Active lines per picture

4320

Picture aspect ratio

16:9

We realized that interoperability with the present
HDTV system would be desirable, and the use ofcommercially available HDTV equipment would
make system implementation easier. Hence we madethe pixel count of Super Hi-Vision exactly four times
larger (both horizontally and vertically) than that of
HDTV, namely 1920 x 1080. Figure 1 compares thespatial and temporal resolutions of Super Hi-Vision
with those of the media that are currently used orunder study. Large screen digital imagery (LSDI) has

We believe that it is not too early to start the researchon the next generation of television now. As thehistory of HDTV suggests, we believe it would take along time to develop and deploy a new televisionsystem. Although we started research on an ultrahighdefinition camera in 1995 [3], at that time, we were
not thinking of it as part of the Super Hi-Vision
television system. The original goal was simply to
explore new media for HD and SD program
production. We set our goals for developing a next-39

On November 2, 2005, NHK conducted the first relaytransmission of uncompressed Super Hi-Vision image
and 22.2 multichannel sound signals. The experiment
was conducted over a 260-km distance via an opticalfiber network in real-time. We connected the
Kamogawa Sea World aquarium (production site) andthe NHK STRL (presentation site) by dark fibers
belonging to several telecommunication companies.
Such a large-capacity data transmission was madefeasible through dense wavelength divisionmultiplexing (DWDM). The picture and sound should
be sent with a small delay so that newscasters andreporters can conduct a smooth conversation during
live broadcasts. In this regard, the delay intransmission of Super Hi-Vision was only 4 ms,
which is shorter than the frame period.

been
under
study
at
the
International
Telecommunication Union Radio-sector (ITU-R).
Some of the research results on Super Hi-Vision were
contributed to this organization’s international
standardization activities, and they are now reflectedin an ITU-R Recommendation and Report [5, 6].
㪜㫏㫇㪸㫅㪻㪼㪻㩷㪣㪪㪛㪠㪪㫇㪸㫋㫀㪸㫃㩷㫉㪼㫊㫆㫃㫌㫋㫀㫆㫅㩷㩿㫇㫀㫏㪼㫃㩷㫅㫌㫄㪹㪼㫉㪆㪽㫉㪸㫄㪼㪀㪣㪸㫉㪾㪼㩷㪽㫆㫉㫄㪸㫋㩷㪽㫀㫃㫄㩷㩿㪼㫊㫋㫀㫄㪸㫋㪼㪻㪀
6+9



㪋㫂㩷㪛㫀㪾㫀㫋㪸㫃㩷㪚㫀㫅㪼㫄㪸
㪉㫂㩷㪛㫀㪾㫀㫋㪸㫃㩷㪚㫀㫅㪼㫄㪸



+'79

࡮Coding system [11]A video codec based on the MPEG-2 coding scheme
has been developed for efficient transmission andrecording of Super Hi-Vision signals. The codec
system consists of a video format converter, a video
codec, and an audio codec. The video format
converter converts the 7680 x 4320 (G1, G2, B, R)
format from/into sixteen 1920 x 1080/30 PsF (Y/C
4:2:2) images, where the Super Hi-Vision image isdivided spatio-temporally. The video codec consistsof four sub-codecs for 3840 x 2160 images. A subcodec contains four single-unit MPEG-2 HDTV
codecs and a multi-channel frame synchronizer.

6'79


㪇

㪉㪇

㪋㪇

㪍㪇

㪏㪇

㪫㪼㫄㫇㫆㫉㪸㫃㩷㫉㪼㫊㫆㫃㫌㫋㫀㫆㫅㩷䋨㪽㫉㪸㫄㪼㩷㫆㫉㩷㪽㫀㪼㫃㪻㪆㫊㪼㪺㫆㫅㪻䋩

Figure 1: Resolutions of Super Hi-Vision and othersystems

3. Current R&D status
Figure 2 shows the current state of our R&D effortson the broadcasting program chain.
࡮Basic equipment
The first Super Hi-Vision camera system was basedon four-CCD image-acquisition technology: two 2.5in. 8M-pixel CCDs are used for green, and one eachfor red and blue [7]. The four-CCD beam-splitting
system incorporates a two-dimensional pixel offset toform a Bayer color-sampling pattern with the CCDs.
The second version of the Super Hi-Vision camera isequipped with four 1.25-in CMOS sensors [8]. The
display system is similar to the imaging system; ituses four 8M-pixel LCDs to realize a 4000-scanningline display, one LCD for red, one for blue, and twofor green with diagonal pixel offset [9]. Recording
equipment is based on parallel operation of highdefinition hard disk drives. The component devices
are connected using 16 optical fiber or coaxial cables
based on the HD-SDI standard.
࡮Live transmission of uncompressed
through optical fiber Network [10]3URGXFWLRQVLWH&DPHUDKHDG

6WXGLR&&86037(FDEOH':'0
7UDQVPLWWHU+DUGGLVNUHFRUGHU':'0
5HFHLYHU'LVSOD\YHQXH
03(*(QFRGHU+' 6',[PHWDORUILEHURSWLFV76WR,3
FRQYHUWHU+DUGGLVNUHFRUGHU,3WR76
FRQYHUWHU

03(*'HFRGHU'LVSOD\)LEHURSWLFV
03(* 76
*LJDELW,3QHWZRUNV

*+] EDQG%66,QGRRUH[SHULPHQW

Figure 2: Super Hi-Vision system under development

material

40

view large displays. We verified that symptom of
motion sickness induced by vibrating images could be
detected through electrocardiography (ECG).

࡮21-GHz-band satellite transmission experiment[12]
The next-generation transmission systems for 21GHz-band satellites have been investigated in the
expectation that they can achieve higher bit-ratetransmissions than present systems. An indoortransmission experiment was conducted using anexperimental satellite transponder. A Super Hi-Vision
signal compressed to a bitrate of 200 Mbps was
divided into four channels. Each TS generated by the
sub-encoder was modulated with a TC-8PSK, andthen up-converted and amplified with a travelingwave tube (TWT) in the 21-GHz band. The TWT
successfully transmitted signals for over 10 hours intotal. The results suggested that Super Hi-Vision
broadcasting via a 21-GHz band satellite is apossibility.

Ultrahigh-definition images, which are most effective
in large, wide-screen video presentations, will alsopresent a new possibility for a relatively small display.
To clarify this idea, let us consider the relationshipbetween viewing distance and display size with thehelp of Figure 3. The figure depicts the relationshipbetween viewing distance and display size, with thestandard viewing distance condition being the straightline. For a given display size, the pixel structure
cannot be distinguished at any viewing distance that
falls on the right side of the straight line.
6WDQGDUGYLHZLQJYLHZLQJGLVWDQFHFRQGLWLRQ$QH[DPSOHRIODUJHVFUHHQ

4. Researches from the aspect of humanfactors'LVSOD\VL]H LQFK

A thorough knowledge of human factors, especiallythe response of the human visual system to such asystem, will be indispensable to the design of thesystem. We consider that our research scheme should
embody the following aspects of human factors.
Firstly, subjective evaluations are indispensable when
we design the system. Secondly, objective(physiological) evaluations are useful as acomplement to subjective ones because subjective
evaluations may be time-consuming. Furthermore,
physiological evaluations would provide us with auseful perspective for studying the human visualsystem in regard to the design of such a futuristic
audio-visual system. Thirdly, negative effects such as
motion sickness induced by video should be taken
into account when designing the system.

$QH[DPSOHRI+RPH+'79VL]H
6L]HRIXQIROGHGQHZVSDSHU

3L[HOVWUXFWXUHQRWGLVWLQJXLVKDEOH

6XEVWDQWLYHO\QRYLHZLQJGLVWDQFHOLPLWDWLRQ






9LHZLQJGLVWDQFH FP$FFRPPRGDWLRQOLPLW(DV\ YLHZLQJGLVWDQFH
GLVWDQFHRIGLVWLQFWYLVLRQ

Figure 3: Viewing distance and display size for Super
Hi-Vision
It is said that the natural distance for a human toobserve an object distinctly, such as when reading apiece of printed material held in the hand, is around
40 cm, which is called the distance of distinct vision.
For Super Hi-Vision, the display size at which thisdistance of distinct vision becomes the standard
viewing distance is approximately 40-inches. This isequivalent to the size of a present home HDTV
display, or an opened-up newspaper. Assuming that adisplay of such a size could be observed from a
distance of 40 cm raises a number of potential
applications. One possibility is that by watching theentire screen from a distance of about 2 m, the viewer
would experience an effect similar to viewing HDTV,

Recent researches at NHK STRL have revealed
following facts [13, 14, 15].
̒To determine the relationship between viewing angle
and sensation of presence, we conducted subjective
evaluations and body-sway measurements. An
experiment using a large display (over 300 inches)
showed that the sensation of presence increases withthe widening of the viewing angle. The perceived
sensation improvement tapers off as the angle exceeds
70 to 100 arc-degrees.
̒We also studied an issue related to motion sickness
induced by video, which affects some people who

41Jefferson, North Carolina (1987).

but getting closer to the display, to about 40 cm(maintaining the same image) the viewer could viewof a portion of the image with much more clarity.
Such freedom in selecting the viewing distancewithout any concern for picture quality is not possible
with conventional video systems, and it may haveuseful applications in medicine, education, or art
appreciation.

[2] NHK Science and Technical Research Laboratories,"HIGH DEFINITION TELEVISION Hi-Vision
Technology", Van Nostrand Reinhold, New York(1993).
[3] K. Mitani et al., "2k x 2k Image Acquisition System
Using Four CMD Imagers", ITE Technical Report, vol.
19, no. 7, pp. 51-56 (1995).
[4] M. Sugawara et al., "Progress on Large, Wide-screen
Image Presentation - 4000-scanning-line TV aims at a
medium that is provocative yet pleasing to the senses",
Broadcast Technology, no. 18, pp. 5-9 (2004).

A display size of 20-inches would have a standard
viewing distance of 20 cm, which is close to theaccommodation limit (near-point of accommodation)
at which no pixel structure can be distinguished at any
distance. Thus, Super Hi-Vision would effectively
have a free viewing distance capability. This type ofvideo presentation on a flexible display device would
be the equivalent of a gravure printed magazine that isable to show motion pictures.

[5] Recommendation ITU-R BT.1769 Parameter values
for an expanded hierarchy of LSDI image formats for
production and international programme exchange(2006).
[6] Report ITU-R BT.2053 Large screen digital imagery(2005).
[7] K. Mitani et al., "Ultrahigh-definition color camera
system with 4K-scanning lines", Proceedings of SPIE,
vol. 5017-18 (2003).

5. Conclusion
This paper described the Super Hi-Vision televisionsystem, which we are researching as part of our planof developing a future broadcasting system. The goalof Super Hi-Vision is to provide a better visualexperience, especially a stronger sensation ofpresence, to viewers. For that purpose, the display
has a viewing angle of up to 100 degrees and its
picture quality is excellent. Super Hi-Vision is still inthe early stage of research and development. This
paper described some of the work that is being
conducted from diverse aspects, including thedevelopment of equipment, system integration, andhuman factors.

[8] H. Shimamoto et al., "An 8k x 4k Ultrahigh-Definition
Color Video Camera with 8M Pixel CMOS Imager",
SMPTE Motion Imaging Journal 2005(July/August),
pp.260-268 (2005).
[9] M. Kanazawa et al., "A wide-screen projector of 4k x
8k pixels", IDW'02, pp. 1241-1244 (2002).
[10] T. Nakatogawa et al., “Long-haul Transmission
Experiment using Dense Wavelength Division
Multiplexing with 16 Carriers for Uncompressed Super
Hi-Vision Signal”, ITE Technical Report, BCT200618 (2006).
[11] Y. Nishida et al., "MPEG-2 Codec for Super HiVision", IEICE General Conference, D-11-26 (2006).

If we consider the system from the aspect of its worth
as a "visual interface", we can conclude that Super HiVision satisfies the fundamental human desire towatch what exists at distant places and that it isamong the most visual of any media developed so far.
It would expand the possibilities of visual interface
technologies. Besides its suitability for large and wide
screens, if Super Hi-Vision could be shown on rather
small but high-pixel-density screens, it would be avideo medium with virtually invisible pixels.

[12] H. Sujikai et al., “Indoor Transmission Experiment ofSuper Hi-Vision in the 21GHz Band”, ITE Technical
Report, BCT2006-23 (2006).
[13] K. Masaoka et al., "Contrast Effect in Evaluating the
Sensation of Presence for Wide Field-of-View", IEICE
Technical Report, IE2006-15, pp. 19-24 (2006).
[14] M. Emoto et al., "Viewing angle effects from wide
field video projection images on the humanequilibrium", Displays, vol. 26, no. 1, pp. 9-14 (2005).
[15] M. Emoto et al., "Viewing angle effects from widefield video system on visual-induced motion sickness",
ITE Technical Report, vol. 30, no. 22, pp. 25-28(2006).

References[1] A. Abramson, “The History of Television, 1880 to1941”, McFarland & Company, Inc., Publishers,

42Visual Content Production and Digital Technology
Mitsuru Kaneko, Ph.D.
To kyo University o f Technology
1404-4 Kata kura , Ha chioji, To kyo, Japa n
mits@media .teu.a c.jp

Pro cesses of Production stage and Planning stag e h ad

Abstract

been fil m and analog vid eo oriented until recen tly.

Cine ma was in vented al mo st over 100 years ago as

Although word processo r and paint too ls are applied to

the first g eneration of the v isu al con ten ts . No w its for m

Screenplay writing and d es igning at Planning stag e, th e

is transfor ming fro m Fil m to Digital. The in fluence ofusage of d igital technolog y was li mited only with in th e

digital transition affects almo s t all the production

individu al pro cess so th at any data or in for matio nprocess.

generated by a screenplay writer or design er have to be

In this repo rt, I speak abo ut th e major chang es

carried manually fo rward to th e follo wing p rocess.

derived f ro m Digital Techno log y, esp ecially on the

Mail bo ys, assistan ts to prod ucers and directors areprocess wh ich has no t b een chang ed f ro m the earlyrunning around produ ction tea m to convey the latest

days. A Scen ario Engin e, can forward keywords of achang es and inf or mation .

screenp lay to the p rodu ction staff to create designs,
storybo ards, s ettings , and to the assistan t d irecto rs forarrang e ments.

A

DesignEngin ecanCurrent Process of Visual Content Production

designIdea
Story and adapt plan
Direction plancharacters and backg round models u sing th e relatin g

scenario
Storyboard

designLayoutimage d atab ase. A Dio ra ma En gine will allo w directo rs

Production budget and schedule
Art planto si mulate h is direction p lan without askin g acto r ’s

Production management

Background Drawing
Key Animation Drawing

Casting

Exposure Sheet

Shooting script

In-between Animation

Rehearsal

participation . And f inally, co nnecting all th e engin es,
to make the best of digitalization, development of

Costume, make up, props,

Ink and Paint

Lighting, Camera Shooting

Camera Shooting

Live Action

Animation

Effects

production con trol syste m called a Dig ital Production

Dialog, effects and Music
Editing

Assistance is about to co mplete

Dubbing
Mastering
Packaging

1. Introduction
Accord ing to th e visu al co ntent p roduction p rocess
Current Process of Visual Content Businessanalysisund erresearch

at

Tokyo

Univ ersityofPackaged Contents
Promotion idea

Techno log y (TUT), the whole produ ction procedu re ofa typical cin e ma can be divided to almo st

Plan and budget
Casting and sub contracting

50
Public relations

processes. Th ese processes can be categorized to th ree

Advertisement

Merchandising

Theaters

Special reportmajo r stages, which are Planning (Pre-Production )Television

EventsBook and music publishingstag e,Produ ction

stag eandPost-Produ ction

including

ed iting ,co mpositing ,Other merchandising

Various media(Activation ) s tage. Wh ile the mo st of Activation
processes

Retailing

News release

Evaluation

mixing ,Save and store
Re-exploitation

dubbing are d ig itally transfor med sin ce man y years ago,

㩷
43

Shops
Networks

(Process list of Plann ing, Production and Activ ation

to make easy p rep aration fo r their jobs.

stag es of a visu al content)

Common Structure of a Screenplay
Background world

A mong at Pro duction Stag e, dig ital techn ology h aschang ed an i mation p rocess almo s t to tally. It is no w all
done b y co mp uter graphics, no celluloid o r fil m are

Act I

Theme
Act II

Act III

Set upDevelopment

Solution

Story

L.P.1

M.P.

L.P.2 䌃䌬䌩䌭䌡䌸

䃂

䃂

䃂

used du ring the produ ction stage an y more. Live action䃂 䃂 䃂Charactershooting was a little late o n dig italizatio n. Digital

SubSub-story and episodeca mera has been taking over film ca mera only recently.
ReminderMan y directo rs, cin e matograp hers are still suspiciou son dig ital ca mera on reso lution, colo r and lightin g( Stru ctu re of a Screenplay)

expression. At Activation Stage, most p ro cesses aredone dig itally, but all of Cin ema con ten t is transfor med
to fil m when it is exhibited th rough theaters except for

A Sample Interface of Scenario Engine
䉺䉟䊃䊦a few cases.

૞ᚑ⠪ෳᾖ䊒䊨䉾䊃

䌓No w, d igitalization moves quickly into all th ree
stag es as૞ᚑᐕ᦬ᣣᤨ㑆䊄䊤䊙䉦䊷䊑

䌍

䊁䊷䊙

䌆

䉣䊏䉸䊷䊄
╙৻᐀
⊒┵

co mp uters beco me f aster, n etwo rks get

ಽ䊶⑽ᢙ
⢛᥊਎⇇

䊋䉾䉪䉴䊃䊷䊥

䋱䋭䋱㪈 䈇䈧䉅䈱↢ᵴ䋨⴫␜䊨䉾䊃ฬ䋩

cheaper, wider and b etter.

႐ᚲ ᤨ㑆䋨ᧄ䉲䊷䊮/
䋨ᧄ䉲䊷䊮/⚥⸘/⚥⸘/ᱷ䉍䋩

⋧㑐࿑⊓႐ੱ‛䉰䊑䉴䊃䊷䊥

䉰䊑䉣䊏䉸䊷䊄䋨ᩇ䋩႐ᚲ䇮ᤨ㑆䇮ᄤ୥╬䈱⹦⚦⺑᣿䋨䊃ᦠ䈐䋩⊓႐ੱ‛䊶䉥䊑䉳䉢䉪䊃䈱㈩⟎䈫േ䈐ෳᾖ䊒䊨䉾䊃䊶
⾗ᢱ⴫␜ᨒ义䷸䷮丬丶个乊

Impact of Transition
Analog to Digital

Production

Planning䌌

Activation⊓႐ੱ‛ฬ 䇸䋨䈞䉍䈸䋩⊓႐ੱ‛ฬ 䇸

䋨䈞䉍䈸䋩

䇹
䇹

⊓႐ੱ‛䊶䉥䊑䉳䉢䉪䊃䈱⒖േ䈫േ䈐⊓႐ੱ‛ฬ 䇸䋨䈞䉍䈸䋩⊓႐ੱ‛ฬ 䇸

䋨䈞䉍䈸䋩

䋱䋭䋲

႐ᚲ ᤨ㑆

䇹
䇹

䋨ᩇ䋩

(Sa mple in terface of a Japanese screenplay writing
Film, Cel toDigital Media

Traditional Creative Processing toComputer Aided Creative Processing

Storyboard to
Dynamic Storyboard

software)

Cost

Paper,Pencil,Running to

Quality

Digital Production
Assistance

IncomeCharacter d esigning is a ma tter of talen t. If you

Digital Production Assistancecannot d raw, you cannot b e a design er. Th is had b een
tru e until Co mputer got enou gh power to help hu mans.

2. Digital Changes at Planning stage

Design Eng ine is a result of analyzing desig ners’ work

Scenario writing esp ecially in Japan has not used

process on p ap er, pen cil and brain. Design ers can only

digital techno logy as in Eng lish sp eak ing countries

create designs when he or sh e has seen or ex perien ced

except for wo rd pro cessing. No screenwriti ng software

so meth ing si milar before. His or her brain reme mb ers

such as Dra ma tica o r Fin al Draft had been develop ed

the i mag es and instructs to han d to d raw. Co mputer can

and used in Japan. A Scen ario Engin e in develop men t

help th e user b ringing sa mp les b y way of d atabase and

at

job

by way of d rawing sof tware. Not everyone wh o use thep roduction

Design Engin e can be as good as desig ners, but

process. Using XML, keywo rds and mean ings are

everyone can create exa mple images to show his or her

carried forward to d esigners, art staff and to directo rs

idea to th e des igners .

TUT

is

instruction

mo re

than

so ftwaretowritingthesoftwarefollo wing

but

㩷
44

Designers’ creation process

Think
Think
Knowledge/Skill
Knowledge/Skill

Experience
Experience

Express
Express
Input
Input
AdjustAdjustOutputOutput䋨Wo rk An alysis of a character/backg round designer䋩

(Sa mple Interface of Dio ra ma Engin e)
In co mpu ter graph ics p rodu ction , motion data are
obtain ed b y motion cap turing liv e p erfor mers and areCharacter is formed from information on
Actors/Images, Story, Setting and Episodes

integ rated to co mputer models. Anoth er n ew method toActors䊶Images
Actors䊶Images

si mulate

goo d

old

cartoo naction

data

is

in

develop ment at TUT.

Story
Story

Setting
Setting

Character
Character

EpisodesEpisodes( Sa mp le Interface of Design Engin e)

Motion Capturing of Cartoon animation

Dyna mic Storyboard 㩷 h as b een in dev elop me nt for the
purposeofd irectors

si mulation .

In

liv e

actionproduction , a directo r concretes his o r h er plan b y
reh earsal, which is costly and ti me li mited . With th e
Dio ra ma Engin e a director can put b ackg rou nd objects( Sa mp le mo tion d ata of an o ld cartoon )on the virtu al floor, position the cast, pu t ca mera andlights and th en move ch aracters and ca mera as h e o r

Copyrigh t of the cartoon foo tages like this in so me

she p lans. If it is no t good , try th e o ther way.

countries is still v alid and in so me coun tries not. Wh at
about the motions and action s of th e ch aracters which

3. Digital Changes at Production stage

the an i mators created for these ani mated footages many

By using dio rama engin e, a directo r ’s job of direction

years ago?

beco mes ti me effective. Ca mera and ligh t positions are

Good old movies ’ data on ligh ting can b e d atabas ed as

pre-deter minedthethe sh eet sho ws here. Nu mber of lights, strength ,cine matog raph er and his tea m as si mu lated images . Set

location and direction of ligh ts are 3 D an alyzed so th at

prep aration will be also time effective since art

this sheet can be used both f or liv e shootin g and CG.

directo r kno ws well in adv ance about wh at th e d irecto r

The question of cop yrigh t arises h ere too .

and

are

info r med

towan ts.

㩷45

Visu al outpu t by a d igital p rojector is always clean ,clear (d epends upon th e resolu tion) and b righ t. If it is
not b right enou gh because of distan t screen or a mbient
light, two p ro jectors can p roject exactly the sa me
fra me i mag es for p inpoin t p ro jection .
Audio ou tput by mu ltiple speak ers creates mo rerealistic sound becau se a screen with many sp eak ers
printed will b e availab le shortly. Sound mix in g p rocess
for this typ e o f screen exhib ition, though , h as to b e( Sa mp le data sheet of ligh ting of old mov ies)

mo re co mp licated.

4K reso lution

5 Digital changes at Production Control

ani mation Test Produ ction see ms to

be too mu ch for An i me. Bu t it is good to p rovid e

A concept of Digital Production Assistan ce (DPA) has

information of how to deploy characters and models injust started at research and test level. Eli mination ofa fra me, ho w the action can b e ex aggerated or li mited ,fil m accelerates the move men t of applying the con cep twhat k ind and nu mb er of colo rs should be selected forto th e who le en tertain men t p ro duction industry.

high qu ality i mages.

All

pro cessofvisualcontent

produ ction

fro mPlann ing stage to Activ ation stage can b e aud ited daily
just as the o ther p roduction indu stry so that tru ecreativity stan ds on th e establish men t of health yacade mic groun d of p roduction .

6 Future of Visual content production
To fulfill th e increasing de mand for n ew visualconten ts fo r Ubiquitous, ho me, classroo m, auditoriu mand th eater media, th e produ ction has to p repare formo re reso lutio n, mo re co lo r expression, mo re detailed
sound sou rce expression using digital equip ments.
(Test Prod uction of 4 K

Ani me p roduction)

Regardless of Interactiv e and non -in teractive, th equality of visual content has to match with the de mand

4 Digital Changes at Activation stage

for all med ia exhibition . Digital screenplay writing ,Digital post-p roduction p rocesses su ch as co ntrast anddigital d esigning, co mputer graph ics si mu lation , 4 K

colo rca meracorrection,sound

relocation

andmixing ,visualinput,

multi- microphone

input,

co mposition , editing , dubbing , and mastering h as been

separated audio-visu al track record ing and mastering,industrial stan dard for several years. Soon the current

network suppo rt for produ ctio n and d istribu tion will be

fil m convers io n will be eli mi nated as th e th eaters hav e

the key techn ologies fo r

equipped with the 2 K and 4 K digital pro jectors. The

industry.

the future dig ital con ten t

cost of fil m p rinting, tran sporting and maintaining will
be gone becau se of rep lacing fil m to dig ital h igh sp eed( Mitsu ru Kaneko, 2006 /09/11 )data trans mission.

㩷
46

Free Viewpoint Television

Masayuki Tanimoto
Graduate School of Engineering, Nagoya University
Furo-cho, Chikusa-ku, Nagoya, 464-8603 Japan
tanimoto@nuee.nagoya-u.ac.jp
it opens new applications in the fields of communication,entertainment, advertising, education, medicine, art, archives
and so on. As information infrastructure for secure society, it
increases security of public facilities, roads, vehicles, schools,
factories and so on.
We proposed FTV to MPEG [2] and have been making
contributions. It is considered the most challenging scenario in
3DAV (3D Audio Visual) of MPEG and the standardization of
MVC (Multi-view Video Coding) has started [9], [10].
The FTV Committee was organized to promote MPEG
standardization and to develop FTV applications in JEITA(Japan Electronics and Information Technology Industries
Association) under the support of METI (Ministry of Economy,
Trade and Industry). About 20 organizations including
industries, carriers, broadcasters, content providers and so on
participate this activity.
FTV can be a platform for various important 3D systems andthe FTV signal can be a common signal format for the 3D
systems.
It should be noted that FTV is not a conventional pixel-based
system but a ray-based system. We have been developing thetechnologies of ray capturing, ray processing, and ray display[11].

ABSTRACTFTV (Free viewpoint TV) enables us to view a distant 3D world
freely by changing our viewpoints as if we were there. FTV willbring an epoch-making change in the history of television since
it is an ultimate 3D-TV and positioned as the top of image
media because of its largest space of representation. We
proposed the concept of FTV and verified its feasibility by theworld's first real-time system including the complete chain from
capturing to display. The international standardization on thecompression of FTV is in progress. FTV is not a pixel-based
system but a ray-based system. We are creating ray-based image
engineering through the development of FTV.

1. INTRODUCTION
Television has a long and successful history since it realized ahuman dream of seeing a distant world in real-time. It stands as
the most important visual information system. However, TV
shows us the same scene even if we move our viewpoints infront of the display. It is quite different from what we
experience in the real world. In TV, users can get only a single
view of a real 3D world they want to see. The view isdetermined not by users but by a camera placed in the 3D world.
Although many important technologies have been developed,
this function of TV has never changed.
TV has been developed by pixel-based technologies. However,
the most essential element of visual systems is not a pixel but aray. We have been developing ray-based 3D information
systems that consist of ray acquisition, ray processing, and ray
display. FTV (Free Viewpoint Television) [1] - [3] is a typicalexample of such a system.
FTV will bring an epoch-making change in the history of
television because it has a new function to view the 3D world
freely as if we were there. The ray-space method [4] - [7]enables it.
We proposed the concept of FTV and verified its feasibility by
the world's first real-time experiment [8], where a real moving
object was viewed using a virtual bird’s eye camera whose
position was moved freely in a 3D space controlled by a 3D
mouse.
FTV brings a new frontier to the field of signal processing since
the signal processing of FTV such as coding and view image
generation is performed in the ray-space that is a new domain
with higher dimension than that of the current TV. A new userinterface is also needed for FTV to make full use of 3D
information of FTV.
As an ultimate 3D TV, FTV needs higher performance for all
related devices, equipments and systems, and accelerates thedevelopment of electronic industry. As the next-generation TV,

2. RAY-SPACE REPRESENTATION
2.1. Ray-space method
We developed FTV based on the ray-space representation [4] [7]. In the ray-space representation, one ray in the 3D real
space is represented by one point in the ray space. The rayspace is a virtual space. However, it is directly connected to thereal space. The ray-space is generated easily by collecting
multi-view images with the consideration of camera parameters.
Let ( x, y , z ) be three space coordinates and θ,φ be theparameters of direction. A ray going through the space can be
uniquely parameterized by its location ( x, y , z ) and the
direction (θ,φ ); in other words, a ray can be mapped to a pointin this 5-D ray parameter space. In this ray parameter space, we
introduce a function f whose value corresponds to an intensityof the specified ray. Thus, all the intensity data of rays can be
expressed byf ( x, y, z;θ , φ ).

− π ≤ θ < π ,−π / 2 ≤ φ < π / 2.

(1)

We call this ray parameter space the ray-space.
Although the 5-D ray-space mentioned above includes all theinformation viewed from any viewpoint, it is very redundantdue

47

u=tanθyxHorizontal
sectiontan θ

Ray-Space f (x, θ)xFigure 3. An example of orthogonal ray-space.

Figure 1. Definition of orthogonal ray-space.

ySection １
Section ２xu

Section Image 1

Ray-Space f ( x, y, u)Section Image 2

Figure 4. Generation of view images

ξFigure 2. Acquisition of orthogonal ray-space.

Horizontal
section 0ηto the straight traveling of rays. Thus, when we treat rays that
arrive at a reference plane, we can reduce the dimension ofparameter space to 4-D.
We use two types of ray-space for FTV. One is the orthogonal
ray-space, where a ray is expressed by the intersection of the ray
and the reference plane and its direction. Another is thespherical ray-space, where the reference plane is set to benormal to the ray [7]. The orthogonal ray-space is used for
FTV with linear camera arrangement whereas the spherical rayspace for FTV with circular camera arrangement.
The definition of the orthogonal ray-space is shown in Fig. 1.
The ray-space is 4-dimensional and 5-dimensional including
time. If we place cameras within a limited region, the obtained
rays are limited and the ray-space constructed from these rays is
a subspace of the ray-space. For example, if we place cameras
on a line or a circle, we have only one part of data of the rayspace. In such cases, we define smaller ray-space.
For the linear camera arrangement, the ray space is constructedby placing many camera images upright in parallel as shown in
Fig. 2, forming the FTV signal in this case. The FTV signal
consists of many camera images and the horizontal cross-section
has line structure as shown in Fig. 3. The line structure of the
ray-space is used for the ray-space interpolation andcompression.

π

θ

ξ
θ

π
ξ

0

θ

Figure 5. An example of spherical ray-space.

Vertical cross-sections of the ray space give view images at thecorresponding viewpoints as shown in Fig. 4.
For the circular camera arrangement, the spherical ray space isconstructed from many camera images and its horizontal crosssection has a sinusoidal structure as shown in Fig. 5. The
sinusoidal structure of the ray-space is also used for the rayspace interpolation and compression.
The hierarchical ray-space [6] is defined for scalability.

48Video Capture

Multi-Camera

MVC
Encoder

Correction

Captured
Images

View
Generation

MVC
Decoder

Corrected
ImagesReconstructed
Images

Ray
Space

2D/3D
DisplayFree
Viewpoint
ImagesFigure 6. Configuration of FTV system.

3. FTV SYSTEM
3.1. Configuration of FTV System
Fig. 6 shows the configuration of FTV system.
At the sender side, 3D scene is captured by multi-cameras. The
captured images contain the misalignment and luminancedifference of cameras. They should be corrected to construct
ray-space. The corrected images are compressed fortransmission and storage by the MVC (Multi-view Video
Coding) encoder.
At the receiver side, reconstructed images are obtained by the
MVC decoder. Ray-space is constructed by arranging thereconstructed images and interpolating them. Free viewpoint
images are generated by cutting the ray-space vertically anddisplayed on a 2D/3D display.
The function of FTV was successfully demonstrated bygenerating photo-realistic free viewpoint images of the moving
scene in real-time.
Each parts of Figure 6 are explained in more detail below.

Figure 7. 1D arc capturing system.

3.3. Correction
The rectification [14] and luminance compensation [15] ofcamera images are performed by measuring the correspondence
points of camera images. This measurement is made once after
the cameras are set.

3.2. Capturing

3.4. MVC Encoding and Decoding

We constructed a 1D arc capturing system shown in Fig.7 for areal-time FTV system covering the complete chain from video
capturing to display [12]. It consists of 16 cameras, 16 clientsand one server. Each client has one camera and all clients areconnected to the server with Gigabit Ethernet.
A “100-camera system” has been developed to capture larger
space by Nagoya University (Intelligent Media Integration COE
and Tanimoto Laboratory) [13]. The system consists of one hostserver PC and 100 client PCs (called ‘nodes’) that are equippedwith JAI PULNiX TM-1400CL cameras. The interface between
camera and PC is Camera-Link. The host PC generates asynchronization signal and distributes it to all the nodes. Thissystem is capable of capturing not only high-resolution video
with 30 fps but also analog signal up to 96 kHz.
The camera setting is flexible as show in Fig. 8. MPEG test
sequences “Rena” and “Akko&Kayo” as shown in Fig. 9 weretaken by the camera arrangements of (a) and (c), respectively .

Standardization of compression of multi-camera images is
progressing as MVC (Multi-view Video Coding) in MPEG. The
detail is described in chapter 5.

3.5. View Generation
Ray-space is formed by placing the reconstructed images
vertically and interpolating them. Free viewpoint images
are generated by making the cross-section of the rayspace.
Examples of the generated free viewpoint images are shown in
Fig. 10. Complicated natural scenes including sophisticatedobjects such as small moving fishes, bubbles and reflection oflight from the aquarium glass are reproduced very well.

49

(a) linear arrangement
Figure 10. Examples of generated FTV images
at various time and viewpoints.

(b)

circular arrangement

Figure 11. Display of FTV.

The quality of generated view images strongly depends on theinterpolation. The ray-space interpolation is done by detecting
depth information pixel by pixel from the multi-view video. We
proposed several interpolation schemes of the ray-space [16][18].
The free viewpoint images are generation by a PC cluster in[12]. Now, they can be generated by a single PC and FTV on
PC is realized in real-time [19].

(c) 2D array arrangement
Figure 8. 100-camera system.

3.6. 2D/3D Display(a) “Rena”

FTV needs a new user interface to display free viewpoint
images. Two types of user interface were made using 2D andautomultiscopic 3D displays with a head tracking system asshown in Fig. 11.
Many head tracking systems have been proposed using
magnetic sensor, various optical marker, infrared camera,
retroreflective light from retina, etc. Our head tracking system
uses only a conventional 2D camera and detects the position ofa user’s head by image processing. The user doesn't need toattach any markers or sensors.
In the user interface using a 2D display, the location of theuser's head is detected with the head tracking system and thecorresponding view image is generated. Then, it is displayed on
the 2D display as shown in Fig. 12.
Automultiscopic displays enable a user to see stereoscopic
images without special glasses. However, it has two problems:(b) “Akko&Kayo”

Figure 9. MPEG test sequences.

50

Figure 12. 2D display with eye tracking.

Figure 15. Mirror-scan ray capturing system.

eyes is provided by automultiscopic display and motion parallax
is provided by head tracking and changing image adaptively as
shown in Fig. 13.

(a) without head tracking

4. EVOLUTION OF IMAGE SYSTEM(b) with head tracking

4.1. Progress of 3D Capturing and Display

Figure 13. 3D display with and without head tracking.

Fig. 14 shows the evolution of image system. In the past, image
systems such as photograph, film and TV were individual
systems. At present, they are digitized and can be treated on thesame platform as pixel-based systems. These pixel-based
systems are developing toward more pixels. It is symbolized bySuper High-Definition TV [20]. Although Super HDTV has
about 100 times pixels of SDTV, view is still only one. In the
future, needs for more pixels will be saturated and more views
will be needed. This is an evolution from a pixel-based system
to a ray-based system. We have been developing FTV according
to this scenario. Roughly speaking, we can realize SD-FTV byusing technologies of HDTV or Super HDTV and balancing
pixels and views.
Figure 14. Evolution of image system.

4.2. Ray Reproducing FTV
We are developing the ray reproducing FTV to create ray-based
image engineering. The ray reproducing FTV consists of ray
capturing, ray processing, and ray display.

limited viewing zone and discreteness of motion parallax.
Because the width of viewing zone for each view equals theinterpupillary distance approximately, the view image does notchange by the viewer's movement within the zone. On the other
hand, when the viewer moves over the zone, the view imagechanges suddenly.
In the user interface using the automultiscopic display, thefunction of providing motion parallax is extended by using thehead tracking system. The feeded images change according to
the movement of the head position to provide small motion
parallax, and the view channel to feed image is switched forhandling large motion. It means that binocular parallax for

We developed a new ray capturing system [21] as shownin Fig. 15. It acquires a dense ray-space without
interpolation in real time. In this capturing system, a highspeed camera and a scanning optics system are usedinstead of multiple cameras. The important feature of thisconfiguration is that the spatial density of a multi-camera
setup is converted to a density in time axis, i.e. frame rateof the camera. This means that we can increase the density

51

Fig. 17 shows the progress of 3D capturing and display. In thisfigure, the ability of 3D capturing and display is expressed by afactor of pixel-view product that is defined as (number ofpixels)×(number of views). In Figure 11, ① denotes the factorof the 100-camera system mentioned earlier. We have also beendeveloping new types of ray capturing and display systems.
Their factors are indicated by ② and ③ in Figure 14. ② is amirror-scan ray capturing system[21] and ③ is a 360 degree ray
reproducing SeeLINDER display [22]. They are explained in
the next section.
In Fig. 17, the progress of space-multiplexing display follows
Moore's Law because it is achieved by miniaturization. The
factor of the time-multiplexing display is larger than that of thespace-multiplexing display. The difference is given by timemultiplexing technology. The progress of capturing does notfollow Moore's Law because it depends on the resolution of acamera and the number of cameras used.
The pixel-view product increases very rapidly year after year inboth capturing and display. This strongly supports our scenario.
We can see the three-dimensional image naturally and theimages have strong depth cues of natural binocular disparity.
When we move around the display, we see the corresponding
images in our viewing position. Therefore, we perceive theobjects just as if they were floating in the cylinder.
Figure 16. 360 degree ray reproducing SeeLINDER display.

5. INTERNATIONAL STANDARDIZATIONMPEG established AHG on 3DAV (3D Audio Visual) andstarted 3DAV activities in December 2001 as shown in Fig. 18.
First, many topics on 3D were discussed. Especially, thefollowing 4 topics were studied intensively as EE’s (Exploration
Experiments).
EE1 ： omni directional video
EE2 ： free viewpoint-FTV (Ray-Space)-FVV (model based)
EE3 ： stereoscopic video
EE4 ： 3D-TV with depth disparity informationof the camera interval equivalently by increasing the frame rateof the camera. The scanning optics system is composed of adouble-hyperbolic mirror shell and a galvanometric mirror. The
mirror shell produces a real image of an object that is placed at
the bottom of the shell. The galvanometric mirror in the real
image reflects the image to the camera-axis direction. The
reflected image observed from the camera varies according to
the angle of the galvanometric mirror. This means that thecamera can capture the object from various viewing directions
that are determined by the angle of the galvanometric mirror. To
capture the time-varying reflected images, we use a high-speed
camera with an electronic shutter that is synchronized with theangle of the galvanometric mirror. We capture more than 100
view images within the reciprocation time of the galvanometricmirror. The collection of the view images is then mapped to the
ray-space.
A 360 degree ray producing display SeeLINDER [22] thatallows for multiple viewers to see 3D FTV images is shown in
Fig. 16. It consists of a cylindrical parallax barrier and onedimensional light source arrays. Semiconductor light sources
such as LEDs are aligned in a vertical line for the onedimensional light source arrays. The cylindrical parallax barrier
rotates fast, and the light source arrays rotate to the opposite
direction slowly. If the aperture width of the parallax barrier issufficiently small, the light going through the aperture becomesa thin flux, and its direction is scanned by movement of theparallax barrier and the light source arrays. By synchronously
changing the intensity of the light sources with the scanning,
pixels whose luminosity differs for each viewing direction canbe displayed.

The discussion was focused on FTV and MVC (Multi-viewVideo Coding) in January 2004 because FTV got a strong
support from industries in response to “Call for Comments on
3DAV”. Then, the standardization of MVC started in January
2006.
MVC activity moved to Joint Video Team (JVT) of MPEG and
ITU for further standardization process in July 2006.
The FTV system is not realized by multi-view video coding
only. We have proposed to define FTV reference model thatconsists of several modules and to standardize the interface
between modules under the support of SCOPE-I of MIC(Ministry of Internal Affairs and Communications) [23].

52Figure 17. Progress of pixel-view product at 3D capturing and display.

2001/
12

2002/
12

2003/
10

2004/
10

2005/
07

2006/
01

First proposalof 3D Video

3DAV
seminarCfC
on 3DAVCfE
on MVCCfP
on MVC

Start MVC
standardizationtime

Ray is the most essential element of visual systems. We havebeen developing ray-based 3D information systems that consistof ray capturing, ray processing, and ray display. FTV is atypical example of such systems.
The processing of FTV can be performed in real-time on asingle PC. Thus, we can enjoy FTV on PC if the new standard
of MVC is available and compressed FTV contents aredelivered with package media.

3DAV activities
Requirements on 3DAVEEs on 3DAVFTV and MVC

Many topics
discussed

6. CONCLUSION

Furthermore, the technologies of ray capturing anddisplay are making rapid progress. It accelerates theintroduction of FTV into market as an ultimate 3D TV, a
new tool for art and content creation, an information
infrastructure for secure society, etc.

Focused on FTV and MVC(Multi-view Video Coding)

FTV is opening the way to ray-based image engineering thatgives us frontier technologies to treat rays one by one.

Figure 18. MPEG 3DAV activities and standardization of
FTV coding

53

REFERENCES[1]Masayuki Tanimoto, “Overview of Free Viewpoint Television,”
Signal Processing：Image Communication，vol.21，no.6，pp.454-461，
July 2006．
[2] Masayuki Tanimoto, “Free Viewpoint Television - FTV”, Picture
Coding Symposium 2004, Session 5, December 2004.
[3] M. Tanimoto and T. Fujii, "FTV － Free Viewpoint Television",
ISO/IEC JTC1/SC29/WG11, M8595, July 2002.
[4] T. Fujii, “A Basic Study on the Integrated 3-D Visual
Communication”, Ph.D thesis of engineering, The University of Tokyo,
1994(in Japanese).
[5] T. Fujii, T. Kimoto and M. Tanimoto, “Ray Space Coding for 3D
Visual Communication", Picture Coding Symposium '96, pp. 447-451,
Mar. 1996.
[6] M. Tanimoto, A. Nakanishi, T. Fujii and T. Kimoto, "The
Hierarchical Ray-Space for Scalable 3-D Image Coding", Picture Coding
Symposium 2001, pp.81-84, April 2001.
[7] T. Fujii and M. Tanimoto, “Free-viewpoint Television based on the
Ray-Space representation, ” Proc. SPIE ITCom 2002, pp.175-189,
August 2002.
[8] M. Sekitoh, T. Fujii, T. Kimoto, and M. Tanimoto, "Bird’s Eye View
System for ITS", IEEE, Intelligent Vehicle Symposium, pp. 119-123,
2001.
[9] “Introduction to Multi-view Video Coding”, ISO/IEC JTC 1/SC
29/WG11, N7328, July 2005.
[10] “Call for Proposals on Multi-view Video Coding”, ISO/IEC
JTC1/SC29/WG11, N7327, July 2005.
[11] Masayuki Tanimoto, “FTV (Free Viewpoint Television) Creating
Ray-Based Image Engineering”, Proc. of ICIP2005, pp.II-25—II-28,
2005.
[12] Na Bangchang, T. Fujii and M. Tanimoto, "Experimental System ofFree viewpoint TeleVision", Proc. IST/ SPIE Symposium on Electronic
Imaging, Vol. 5006-66, pp. 554-563, Jan. 2003.
[13] Toshiaki Fujii, Kensaku Mori, Kazuya Takeda, Kenji Mase,
Masayuki Tanimoto and Yasuhito Suenaga,"Multipoint Measuring
System for Video and Sound -100-camera and microphone system -",IEEE 2006 International Conference on Multimedia & Expo (ICME), pp.
437-440 July 2006.
[14] K. Matsumoto, T. Yendo, T. Fujii and M. Tanimoto, "MultipleImage Rectification for FTV", Proc. of 3D Image Conference 2006, P-19,
pp.171-174 (2006).
[15] Kenji Yamamoto, Tomohiro Yendo, Toshiaki Fujii, Masayuki
Tanimoto and David Suter, “Color Correction for Multi-camera System
by using Correspondences,'' ACM SIGGRAPH2006, Posters, 30 July - 3
August 2006.
[16] A. Nakanishi, T. Fujii, T. Kimoto and M. Tanimoto, "Ray-Space
Data Interpolation by Adaptive Filtering using Locus of Corresponding
Points on Epipolar Plane Image", The Journal of the Institute of Image
Information and Television Engineers (ITE), Vol. 56, No. 8, pp. 13211327, 2002.
[17] M. Droese, T. Fujii and M. Tanimoto, "Ray-Space Interpolation
based on Filtering in Disparity Domain", Proc. 3D Conference 2004, pp.
213-216, Tokyo, Japan, June 2004.
[18] M. Droese, T. Fujii and M. Tanimoto,"Ray-Space Interpolation
Constraining Smooth Disparities Based On Loopy Belief Propagation",
Proc. of IWSSIP2004, pp. 247-250, Poznan, Poland, Sept. 2004.
[19] Norishige Fukushima, Tomohiro Yendo, Toshiaki Fujii and
Masayuki Tanimoto, "An Effective Partial Interpolation Method for
Ray-Space", Proc. of 3D Image Conference 2006, pp. 85-88, July 2006.
[20] M. Sugawara, M. Kanazawa, K. Mitani, H. Shimamoto, T.
Yamashita and F. Okano, "Ultrahigh-Definition Video System with
4000 Scanning Lines”, SMPTE Motion Imaging, 112, pp.339-346,
October/November 2003.
[21] T. Fujii and M. Tanimoto, "Real-Time Ray-Space Acquisition
System", SPIE Electronic Imaging, vol. 5291, pp. 179-187, Jan. 2004.

54[22] T. Endo, Y. Kajiki, T. Honda and M. Sato, "Cylindrical 3-D Video
Display Observable from All Directions", Proceedins of PacificGraphics 2000, pp.300-306, 2000.
[23] M. Tanimoto, T Fujii, S. Sakazawa and H. Kimata,"Proposal onStandardization of Free Viewpoint TV (FTV),” ISO/IEC
JTC1/SC29/WG11, M13612 (JVT-T140), July 2006.

Color reproduction in the image and video display "beyond RGB"
Masahiro Yamaguchi
Imaging Science and Engineering laboratory, Tokyo Institute of Technology4259-R2-56 Nagatsuta, Midori-ku, Yokohama 226-8503
guchi@isl.titech.ac.jp

RGB 3-primary color systems.
To breakthrough the limitation, there have been reported
that the use of multispectral imaging significantly improves
the color accuracy [1-4]. Moreover, "Natural Vision" (referredas NV hereafter) system [5-7] has been developed aiming at
an innovative video and still-image communication
technology, which enables high-fidelity color reproduction,
based on spectral information.
NV is an industry-government-academy joint project [5]performed at Akasaka Natural Vision Research Center(NVRC) from 1999 through 2006, conducted by NICT
(National Institute of Information and Communications
Technology, formally TAO) under the support of the Ministryof Internal Affairs and Communication. Researchers fromacademic institutes (Tokyo Institute of Technology and Chiba
University) and industry (NTT Data, Olympus, NTT, etc.)
were participated in the project. The activity of NV iscurrently being continued. In NV project, the experimental
systems for both still-image and video have been elaborated
and shown the effectiveness of spectrum-based color
reproduction scheme. This paper introduces the motivation,
methodology, systems, and applications of spectrum-based
color reproduction technology, mainly referring the results ofNV project.

Abstract
This paper introduces the innovative color reproduction
technologies in the display of images and video, going beyond
conventional RGB-based systems. In the recent applications ofvisual communication systems, the image data are utilized invarious purposes, such as medical diagnosis, electroniccommerce, product design, and image database, and it isexpected that the image content faithfully characterize thesubject in the image. However, the RGB-based color
representation is usually device-dependent, illuminationdependent, and application-specific, and it is difficult toreproduce the original color of subject. The spectrum-based
color reproduction is the breakthrough of the RGB-based
systems, because it deals with quantitative color information,
which enables the high-fidelity color reproduction, thesubstantial improvement of color accuracy, the expansion ofcolor gamut, and the color reproduction under different
illumination environment. In addition, the quantitative color
information enhances the value of image data, namely it
expands the use of image content. In this paper, after thelimitations of current RGB-based systems are addressed, theconcept and technologies of spectrum-based color
reproduction technology are introduced. The impact to theapplication is also presented along with the examples ofexperimental investigations.

1. Introduction
The issue of color is now receiving considerable attention in
the applications of image and video technology. The quality ofcolor becomes significantly improved in the recent displays
and prints; better color tone, higher color contrast, and wider
color gamut. Then it is desired to reproduce more and more
realistic image on the advanced high-quality displays. For therealistic image display, the reproduction of the original color,i.e., natural color reproduction, is quite important [Fig.1]. The
natural color reproduction is also one of the key issues in thetelemedicine, electronic commerce, and electronic museum.
However, the conventional image and video systems do not
reproduce the original color of subject, and it obstructs theapplications of visual communication systems. The color
management technology is greatly progressed especially incolor printing, but there still remain limitations imposed by

(a)

(b)
Figure 1: The concept of natural color reproduction.
(a) Reproducing the color as if the observer were at the
remote site, (b) reproducing the color as if object wereplaced at the site of observer,

55

Color space
Conversion profile

Multispectral
cameraMultispectral
scanner

SourceprofileImage
data

Trichromatic
cameraSpectrum-based color
reproduction system
Colorimetry
Spectral Radiance
Spectral Reflectance /TransmittanceMultiprimary
display
Trichromatic
display
printer

ProfileDatabase
Transmission

Image
data

involved in the imaging systems. For example in theimage retrieval using color information, target object
cannot found if the illumination condition is different.
The target of spectrum-based color reproduction is to
overcome these problems in the image and videocommunication systems.

Destination
profile3. Spectrum-based color reproduction

Data analysis etc.

The concept of the spectrum-based color reproduction system[6,7] is depicted in fig.2. To obtain sufficient spectral
information, multispectral camera (MSC) is desired as an
input device. The profile of the input device including thespectral sensitivity, tone curve, and dark current level of thecamera, and the spectral energy distribution of illuminant are
attached to multispectral image (MSI) data. In image
processing, transmission, and storage, the image isaccompanied with the profile data, so that the spectral
radiance, reflectance or transmittance can be retrieved. In
image display, the image of tristimulus values or spectral
radiance is reproduced on 3-primary or multiprimary color
display with device calibration.
The architecture of spectrum-based color reproduction
system is similar to the ICC (International Color Consortium)
color management system, but the profile connection space(PCS) is based on physical model, i.e., spectrum-based PCS(SPCS) instead of color appearance model. This can be any of
CIEXYZ under arbitrary illumination, spectral radiance, or
spectral reflectance. The information required for thetransform to SPCS is held in the profile data. The profile data
format was defined as NV image data format, which was
proposed to CIE TC8-07. The XML version NV format wasalso developed for easier handling.
In this system, the numbers of channels in the image
capture and the display are independent, and the input andoutput devices with the arbitrary numbers of channels can be
employed. Three-channel devices can also be used in thissystem with proper device characterization, with certain
restrictions in the color reproduction capability or accuracy.
Spectrum-based system offers the solution to the problems
presented in the section 2, as follows:(a) For the natural color reproduction of real scene as if theobserver were at the site as shown in fig.1(a), the colorimetricor spectral information is accurately captured and reproduced
on a screen.
(b) In the situation shown in fig.1(b), the spectral reflectanceof the object is captured and the color under the illuminationof observing environment can be reproduced on a screen.
Fig.3 depicts the example of color accuracy estimated fromthe 16-band camera shown in fig.5 and commercial 3-banddigital still camera (DSC) with spectral characterization. Itshows the advantage of using spectral information for
accurate color reproduction.

Figure 2: The scheme of spectrum-based color reproduction
system

2. Limitation of RGB-based systems
Let us briefly review the limitations of conventional
RGB-based systems, to make clear the advantage ofspectrum-based color reproduction.
(1) The RGB values obtained in conventional systems often
have different meanings, depending on the device
characteristics or color processing. For example, many ofconventional color imaging systems are designed for user
preference, RGB values does not represent the objective
color information.
(2) The spectral sensitivity characteristics of color camera
are different from the human vision; RGB signal does nothave one-by-one correspondence to the tristimulus values
perceived by human vision. If the spectral sensitivity iscloser to human vision, the color fidelity is improved, but
noise behavior is known to become worse.
(3) The RGB or any other 3-primary color signals thatcomply with the color space such as sRGB or YCbCr,
which are defined under the white point such as CIE D65
or D50 standard illuminant. To reproduce the color as if
the object were placed in the presence of the observer[fig.1(b)], the color under different illuminant is required,
but the standard color space does not support the color
under arbitrary illuminant.
(4) For the color reproduction shown in fig.1(b), the spectralreflectance of object and the illuminant spectrum areessentially required for calculating the color under
different illuminant. White balance adjustment in RGB
space does not give high colorimetric accuracy.
(5) The color gamut of a normal RGB display is limited, andsome of high-saturation colors cannot be reproduced.
Even if the display device allows the display of wider
color gamut, conventional color signals such as sRGB or
ITU-R BT.709 do not support wider gamut color signal.
(6) The observer metamerism effect, due to the observer
dependence of color matching functions, cannot beignored in high-accuracy color reproduction, ex., in thecolor proofing of printed materials with color monitors.
(7) In the image archive, database, or analysis, the utilizationof color information is limited, since the RGB signal
depends on the devices, illuminants, and preprocesses

56DL
16 bandMSC
3 band
DSC

A
F2
DL
A
F2
0

5

10

15

∆E *ab20

Figure 3: The color estimation accuracy of 16-band MSC
and 3-band DSC, evaluated using GretagMacbeth Color
Checker, under daylight (DL), CIE A and F2 illuminants,
where DL was used in the image capturing.

(a)(b)
Figure 5: (a) 16-band MSC with rotating filter wheel. (b)6-band HDTV camera for capturing motion picture.

Color Gamut presented by multiprimarycolor display
Color Gamut presented by RGB display system

y

(a)
(b)
(c)
Figure 6: Muliprimary displays developed in NV project. (a)6-primary DLP projector (b) 4-primary flat-panel LCD
display with LED backlight, and (c) 6-primary rearprojection display.

x

Figure 4: The color gamut obtained by 6-primary color
display shown in fig.7 in comparison with conventional
RGB display.

Multiprimary color displays and wide gamut systems
Although there were few previous works on multiprimarycolor displays for larger color gamut [11], the systemdevelopment and evaluation of muliprimary display had beenoriginally started by NV project. The multiprimary projection
displays were developed in NV as shown in fig.6 to confirm
the advantage of wide gamut natural color reproduction
system and to evaluate its component such as multiprimarycolor conversion techniques. As in fig.4 and table 1, 6-primary
DLP display almost covers the gamut of natural objects andthe gamut volume in CIELAB space is about 1.8times larger
than the normal RGB projector. The gamut is enlarged in thedark red, cyan, purple regions and bright orange regions.
There have been reports of multiprimary displays recently forcommercial products from various companies [12,13]
Since the conventional color space for images and video
does not cover the whole color gamut, it is required to use
expanded color space for wide gamut display. Conventional
color spaces are defined according to the narrower gamut
color display, but it is not required to depend on the display
characteristics in the definition of color space from the aspectof spectrum-based color reproduction. The wide gamut video
signal can be directly encoded with CIEXYZ, CIELAB,
though the efficiency is not optimized if considering thequantization. Recently, standards for wider gamut video signal
become a reality, de facto or de jure, such as ITU-R BT.1361,
xvYCC, and adobeRGB. It is expected to establish the content
creation environment for wider gamut images and video, andto realize the broadcasting of wider gamut video for thewidespread use of the benefit of latest display technology.

Table 1: Comparison of color gamut with conventional
RGB display and natural objects (Pointer+SOCS gamut [7]).
Volume(×106)
Relative
volumeNatural color
coverage (%)

Pointer+SOCS

RGB-DLP

6-P DLP[Fig.6(a)]

0.890

0.908

1.647

1.00

1.02

1.85

100.0

78.8

99.2

(c) The wider color gamut becomes available by multiprimarycolor display, i.e., using more than three-primary colors. Fig.4
and Table 1 show the color gamut obtained by the 6-primarycolor displays developed by NV project [8,9]. Also, the image
and video data obtained from spectrum-based system canrepresent the wider gamut signal.
(d) The original attribute of object that generates color, ex.,spectral radiance, reflectance or transmittance information of
the object, is captured and preserved by means ofmultispectral imaging. Such information is useful for the
analysis or the recognition of object, in the digital archives ofcultural heritage, artworks, and clinical cases in medicine.

4. Multispectral and Multiprimary Technology
Multispectral Image Capture
Fig.5(a) shows the 16-band MSC developed in NVRC. As in
fig.3, the accuracy by MSC is high and better than thediscriminable level of human vision, while visually apparent
error is observed in 3-band DSC. Fig.5(b) shows a 6-band
HDTV camera for the acquisition of motion picture [10]. In
image capturing, the illumination light spectrum is alsomeasured, and then the spectral reflectance of the object canbe calculated using the multispectral pixel value, illumination
spectrum, and the spectral sensitivity of the camera.

Spectral color display
There exists variability in color matching functions of57

human observers; originated from the macular pigments, lensabsorption, and cone sensitivity. Due to the individual
difference of color matching functions, the color difference
may appear to a certain observer when two color stimuli areshown, even if they are perceived as the same color byanother observer, or CIE standard observer, called observer
metamerism. It causes the color mismatch between different
media, such as color printed materials and displays, eventhough the colorimetric match is achieved for CIE XYZ
values. Based on the multispectral and multiprimary
technology, spectral color display becomes possible, and theobserver metamerism effect can be disappeared. The
advantage of spectral color display was experimentallydemonstrated using 6-primary display [14].

6-band HDTV camera

CCU

Dual-link HD-SDI×2
Dual-link SDI × 2
4:2:2 × 4

Uncompressed 6-channel signal
4:4:4:4:4:4
Color conversion

Network
Video editor

6-primary display

Video recorder

Database

3-primary signal

TMDS SXGA × 2 6-primary signal or
HD-SDI 4:2:2×2 6-primary signal

3-primary display

Parallel encoder

NV data format

software

3-6 colorconverter

Input frame buffer

Output frame buffer

3-primary signal

Parallel decoder

Dual-link HD-SDI×2

Figure 7: The experimental prototype system
multispectral and multiprimary video reproduction.

for

6-primary display
ObjectMultiprimary color conversion
The multiprimary color signal is generated from the imageof tristimulus values or multispectral data, called multiprimarycolor conversion. For colorimetric color reproduction,
tristimulus values are transformed to M-dimensional
multiprimary color values (3toM), where a degree of freedom
is involved. Several methods have been developed for 3toM
color conversion [15,16]. For the spectral color reproduction
explained in the previous paragraph, it becomes NtoM
conversion, where N is the number of channels of MSI. In the
proposed NtoM conversion method, the spectral error isminimized under the constraint that the colorimetric match isattained for the standard observer [14].

HDTV master monitor

3-band camera
6-band camera

Figure 8 Experimental live reproduction using 6-band
HDTV camera, 6-primary display, and real-time color
conversion, comparing with conventional HDTV. Note thatthe difference can only be seen in the real situation.

5. Applications
It is expected that realistic images and video display withhigh-fidelity natural color reproduction technology will beapplied in various fields. There have been experimentallydemonstrated the effectiveness of spectrum-based system;
In medicine, the use of digital color images is extremely
valuable for teleconference, teleconsultation, education,
training, image analysis, and reference database, in pathology,
endoscopy, and dermatology. It is also required to reproduce
the complexion of a patient through visual communication
system for the telediagnosis and homecare. If the reproduced
color is not accurate, it may cause misdiagnosis, thus thereliability of reproduced color is critical [19]. In dermatology,
MSI systems of both still-image and video were tested and thecolor reproducibility of multispectral system was rated
sufficiently high from the visual evaluation by dermatologists[20]. In the pathology application, the spectrum-based systemenables not only the accurate color reproduction of stained
specimen, but also the correction of variation in staining
condition, with adjusting the amount of stain through digitalprocess [21] [Fig.8]. It was also shown that spectral
information is useful for the image analysis that supports
diagnosis in both pathology and dermatology; the grading ofdisorders or the quantitative evaluation of treatments [20].

Multispectral video system
The prototype of the multispectral video transmission was
also developed in NV project. By using multiprimary color
conversion hardware, real-time colorimetric reproduction isrealized with 6-band MSC and 6-primary display in HDTV
resolution [7], as shown in figs7 and 8
The compression and encoding are the other issues on MSI
transmission. As an MSI compression technique considering
the colorimetric accuracy, modified KL transform calledweighted KL transform was proposed and combined with
JPEG2000 scheme [17]. For video compression, several
methods that support multichannel images were tested,
including a coding method in which MSI signal is converted
into visible and invisible components, considering thecompatibility to the conventional video signal [18]. JPEG2000
is one of the suitable formats for high-quality encoding ofboth video and still MSI's. The profile data for NV format
described above can be easily implemented in JPEG2000 as aextended ICC profile or metadata with using XML.
MSI can be considered as a value-added color image, as
every pixel has quantitative spectral data. MSI databasesystem has been developed, which supports image processing,analysis, and retrieval that utilize spectral data, and applied to
the various fields as explained in the next section.

58

(a)
(b)
(c)
Figure 9 Application to pathology: standardization ofstaining condition. (a) An image of understained specimen,(b) Corrected image from (a), (c) Example of optimal stain.
In the field of electronic commerce (EC), high-fidelity color
reproduction is very important. In the online-shopping andtrading of textiles, apparels, cosmetics, vehicles, furniture, etc.,
the color difference causes the return of purchased items. Alsoin the color matching of samples between designers, factory,
buyers, and sales, the quantitative color information is quite
beneficial. High-fidelity natural color reproduction system
will contribute the further progress in EC. A web-based
server-client prototype of electronic catalog system wasdeveloped in NV, and its effectiveness was demonstrated.
In the digital archive and electronic museum for art andcultural heritage, it is undesirable that the color of the archive
depends on the imaging device and illumination condition
with the use of conventional imaging devices. There havebeen many reports on the multispectral imaging for digitalarchive. The result of color reproduction on a display using
16-band MSI was shown to be satisfactory for the art
management staffs in NV project [22]. The image data can bealso employed for the high-fidelity color prints. Moreover, thespectral information in the image will be useful for the
analysis of pigments or dyes, or the selection of material for
restoration.
As for the application to color prints, it has been also
proved that multispectral imaging greatly improves the color
fidelity. In the test assuming the printing of catalogs for
product promotion, a 16-band MSC and professional RGB
DSC's were compared. From the printed images obtained
from the RGB DSC, a professional print director put 4-5
instructions for color corrections on average per printed
sample. On the other hand, no instruction for color correction
was given to any of resultant samples from 16-band MSC.
The print director also commented that the printed resultsfrom MSI is even better than the proof produced from RGB
DSC after the color correction [23]. Furthermore, the spectral
color display can be useful for the color proofing applications
using softcopy monitors, since the color matching between
display and print is considerably improved thanks to thespectral approximation.
Digital prototyping is an important technology for
expedition and efficiency of product development, andcomputer graphics (CG) enables the realistic rendering ofvirtual products, using Bidirectional Reflectance Distribution

59

Function (BRDF) or Bidirectional Texture Function (BTF),
but the color disagrees with real products. Multispectral
BRDF/BTF measurement system and multispectral rendering
technique were developed for high-fidelity color digital
prototyping [24].
The expanded color gamut provides a new tool or new
color for graphics expression. A CG system for coloring withmultiprimary colors, named "IRODORI" [25] in NV project.
Through the experiences of graphic expression using
6-primary display, the significance of wide-gamut graphics;(a) rendering of reality in vivid or splendid colors, (b) the
enhanced reality based on "memory color," which often shifts
to higher saturation with time, (c) strong impact or fantastic
impression, and (d) rich color tone thanks to the expansion ofcolor range, providing the faint color change with dark colors,
or the expression of gloss, metallic colors, or emissive colors.
The application of high-fidelity color reproduction is also
beneficial in multimedia education such as instructional
materials and encyclopedia, theater or electronic museum forrealistic color reproduction of nature, artworks, and historicsubjects. It can be also applied to the high-quality color
management in the contents production from the material ofrecorded picture, editing, distribution, and playback of CG,
animation, and other visual programs. In NV project, video
contents were produced using 6-band HDTV camera andmultiprimary CG, and the feasibility of spectrum-based
systems was demonstrated. The application to digital
broadcasting is strongly expected to facilitate the widespreaduse of innovative color imaging technology.

6. Conclusion
The concept, technology, and applications of spectrum-based
color reproduction system are introduced in this paper. It
enables not only the high-fidelity color reproduction but also
the application of image analysis based on the quantitative
spectral information. Moreover, multispectral information willbe also of great utility in the image editing for preferable color,and other various image processing such as object extraction
or image synthesis, though those were not the main topic ofNV project that basically aims to natural color reproduction.
Going beyond RGB, great benefit emerges in advanced
imaging applications. Multiprimary printing is alreadyavailable in commercial product, but to make better use ofdevices that support innovative color reproducibility, aplatform for spectrum-based system is expected, i.e.,
multispectral and wide-gamut video contents creation,
management, distribution, and utilization.
Along with the introduction of information technology into
our life, the quality of information that flows through thesystem becomes very important. For example, the image isused for diagnostic decision in the telemedicine system, and inelectronic commerce, purchase or order items based on thedigital information including images. If inaccurate images

(in Japanese)[12] S. Roth, I. Ben-David, M. Ben-Chorin, D. Eliav, and O.
Bben-David, “Wide gamut, high-brightness multiple primaries
single panel projection displays,” SID '03 Digest 118-121(2003)[13] M-C. Kim, Y-C. Shin, Y-R. Song, S-J. Lee, and I-D. Kim,
"Wide gamut multi-primary display for HDTV," Proceedingsof CGIV 2004, 248-253 (2004)[14] Y. Murakami, J. Ishii, T. Obi, M. Yamaguchi, and N.
Ohyama, "Color conversion method for multi-primary display
for spectral color reproduction," J. Electronic Imaging, 13, 4,
701-708 (2004)[15] T. Ajito, K. Ohsawa, T. Obi, M. Yamaguchi, and N.
Ohyama, "Color conversion method for multiprimary display
using matrix switching," Opt. Rev., 8, 3, 191-197 (2001)[16] H. Kanazawa, "Color conversion for multiprimary
displays using a spherical average method," Proc. 12th Color
Imag. Conf., 65-69 (2004)[17] S. Yu, Y. Murakami, T. Obi, M. Yamaguchi, and N.
Ohyama, "Multispectral image compression for high fidelity
colorimetric and spectral reproduction," J. Imag. Sci. andTech., 50, (1): 64-72 (2006).
[18] S. Takamura and N. Kobayashi, "Constructing a uniform
color space for visually lossless color representation andimage coding," IEEE International Conference on Image
Processing, 918-921 (2001)[19] T. Numahara, "From the standpoint of Dermatology," inDigital Color Imaging in Biomedicine, H. Tanaka, Y. Miyake,
M. Nishibori, and D. Mukhophadhyay, eds., Digital Biocolor
Society, 67-72 (2001)[20] M. Yamaguchi, et al., " Multispectral color imaging fordermatology: application in inflammatory and immunologic
diseases," Proc. 13th Color Imag. Conf., 52-58 (2005)[21] T. Abe, Y. Murakami, M. Yamaguchi, N. Ohyama, and Y.
Yagi, "Color correction of pathological images based on dye
amount quantification," Opt. Rev., 12, 4, 293-300 (2005)[22] H. Fukuda, T. Uchiyama, H. Haneishi, M. Yamaguchi,
and N. Ohyama, "Development of 16-bands multispectral
image archiving system," Proc. SPIE, 5667, 136-145 (2005)[23] A. Uchida, et al., "Product photography for catalogs withmultispectral camera," Proc. of Spring Conf. of the JapaneseSociety of Printing Science and Technology, 131-134 (2003)(in Japanese)[24] M. Tsuchida, et. al., "Development of BRDF and BTF
Measurement and Computer-aided Design Systems Based on
Multispectral Imaging," Proc. AIC Colour '05 129-132 (2005)[25] J. Kishimoto, M. Yamaguchi , H. Haneishi, and N.
Ohyama, "IRODORI - A Color-rich Palette Based on Natural
Vision Technology-," ACM SIGGRAPH 2004 Emerging
Technology (2004)

were presented, it might mislead one's decision. Exchanging
objective and accurate information will be more important inour society.
The author acknowledges the former research members of
Natural Vision project; Nagaaki Ohyama, leader, Hideaki
Haneishi, sub-leader, Toshio Uchiyama, Kenro Ohsawa,
Hiroyuki Fukuda, Junko Kishimoto, Hiroshi Kanazawa,
Tomohito Fujikawa, Hideto Motomura, Tatsuki Inuzuka andSatoshi Nambu, other fellow members from Olympus Co.,NTT DATA Co., NTT Co., Toppan Printing Co., Dai-Nippon
Printing Co., Matsushita Electric Industrial Co., Hitachi Co.,NHK and others, joint researchers of NV project, those
co-operating institutes and companies, National Institute ofInformation and Communications Technology, Ministry ofInternal Affairs and Communication, and Natural Vision
Promotion Council for the great support in NV project.

References[1] B. Hill, F. W. Vorhagen, "Multispectral image pick-up
system," US pat. 5319472 (1994)[2] P. D. Burns and R. S. Berns: "Analysis of multispectral
image capture," Proc. 4th Color Imag. Conf. 19-22 (1996)[3] M. Yamaguchi, R. Iwama, Y. Ohya, T. Obi, N. Ohyama
and Y. Komiya, and T. Wada, "Natural color reproduction in
the television system for telemedicine," Proc. SPIE, 3031,
482-489 (1997)[4] M. Rosen, F. Imai, X. Jiang, and N. Ohta, "Spectral
Reproduction from Scene to Hardcopy II: Image Processing,"
Proc. SPIE, 4300, 33-41 (2001)[5] Annual reports of Natural Vision Project, FY1999-2003(TAO), FY2004-2006 (NICT), (2000-2007)[6] M. Yamaguchi, T. Teraji, et. al., "Color image reproduction
based on the multispectral and multiprimary imaging:Experimental evaluation," Proc. SPIE, 4663, 15-26 (2002)[7] M. Yamaguchi, H. Haneishi, et al., "High-fidelity video
and still-image communication based on spectral information:
Natural Vision system and its applications," Proc. SPIE, 6062,
60620G (2006)[8] T. Ajito, T. Obi, M. Yamaguchi, and N. Ohyama,"Expanded color gamut reproduced by six-primary projection
display," Proc. SPIE, 3954, 130-137 (2000)[9] H. Motomura, N. Ohyama, M. Yamaguchi, H. Haneishi,
K. Kanamori, S. Sannohe, "Development of Six-Primary
HDTV display system," Proc. Int. Display Research
Conference, 563-566 (2002)[10] K. Ohsawa, T. Ajito, H. Fukuda, Y. Komiya, H. Haneishi,
M. Yamaguchi, and N. Ohyama, “Six-band HDTV camera
system for spectrum-based color reproduction,” J. Imag. Sci.
and Tech., 48, 2, 85-92 (2004)[11] R. Yajima, M. Kanazawa, and S. Sakaida, "Wide-colorgamut system," ITE '94 Annual convention, 355-356 (1994)

60

Proceedings of “Visual Interface Toward Future” (2006.9.11)

Electronic Paper: Display for Comfortable Reading- A Study on effects of media handling styles –Makoto Omodani
Department of Optical and Imaging Science & Technology, Tokai University
1117 Kitakaname, Hiratsuka, Kanagawa, 259-1292, JAPANfatigue and enhancing readability. Reading performanceswere evaluated for different medium handling styles using
current electronic books (LIBRIe, ȈBook) and paper. The
Analytic Hierarchy Processi (AHP) was used for thissubjective evaluation.10, 11

Abstract
The development of electronic paper, which has the
merits of both paper and electronic displays, is being
eagerly pursued. This study aims to clarify the factors of
readability which is naturally realized by paper; clarified
factors will yield good guidelines for realizing readable
electronic paper. Our expectation is that reading style isan important factor determining readability. We evaluate
the reading performance of three medium handling styles(vertical, horizontal, hand-held) using current electronic
books (LIBRIe, ȈBook) and paper documents. Test results
show, for both types of electronic books, that the "handheld" style is superior in terms of lower fatigue and better
readability. It is suggested that the readability ofelectronic displays can be enhanced by making themcompact and permitting the hand-held reading style.

2. Experiments
2.1 Experimental Method
Experiments were carried out in order to evaluate
fatigue levels (eyes or body) and readability under various
reading styles using current electronic books (LIBRIe,Ȉbook) and paper.
Three different reading styles were
examined: vertical (the medium was set on a desk (A)),
horizontal (the medium was laid on a desk (B)), and handheld (C).
Figure 2 shows the reading scenes as combinations of thethree reading styles and the three media.The subjects were
presented with material from a Japanese novel and thevolume of text read in a 30 minute period was measured.
Fatigue levels were assessed from the subjects' responses
collected just after each reading task. The tasks werespaced at 15 minute intervals.
Table 1 lists the specifications of the three media.
Table 2 shows experimental conditions. Table 3 gives the
instructions provided to the subjects.

1. Introduction
Electronic paper, which should provide theadvantages of both paper and electronic displays (Fig. 1),
is a prime research topic.1, 2 This study aims to clarify thefactors of readability with regard to paper; these factors
will yield good suggestions for realizing readable
electronic paper. Several reports presented on the factorsof readability have examined medium angle3, medium
form4, contrast5, lighting condition6 etc. However, it is
not clear how to assess these factors. We are attempting toclarify the causes of the difference in fatigue levels seen
for two kinds of media: paper and electronic displays(displays hereafter). One possible cause is the difference
in reading styles. “Freehand holding” is a popular way ofreading text on paper, and this reading style is totallydifferent from the fixed reading style provided by displays
on desks.
We have already reported a comparison of readability
and fatigue level of reading tasks on paper and on
displays. We have confirmed that freehand holding
increases readability and reduces fatigue levels especiallyThis study aims to reconfirm that thefor displays.7-9"hand-held" condition is advantageous for reducing

AnalyticHardcopy
Hierarchy Process (AHP)Display
has been developed࡮Readable࡮ Rewritableby T.L.Saaty (University of Pittsburgh)
for a method of࡮ Compact
࡮ Multi-functional
decisionmakingbestwaste
solution among࡮ Less
energyto choose 䯂theLessalternatives. AHP method is based on comparisons
between all possible pairs among all the alternatives.
Comparisons between several viewpoints for evaluations
are the first stepElectronicof the AHPPaper
method; ratings for the
viewpoints are gotten in this step. This step is followed bycomparisons between all the alternatives. The results of
1. intoConceptof Electronic
ratings Figure
are taken
account
for general Paper
evaluations at
the final step. The comparison performed between pair ofalternatives is considered to bring us accurate sensitivity.
i61㩿㪘㪀㩷㪭㪼㫉㫋㫀㪺㪸㫃㩿㪙㪀㩷㪟㫆㫉㫀㫑㫆㫅㫋㪸㫃All the evaluations were carried out using the methodof comparing pairs of reading styles. Evaluation result foreach reading style was calculated using the three sets ofcomparison results. Figure 3 shows the evaluation scale
used in all comparisons between each pair of evaluation
items and each pair of reading styles. Names of objects tobe evaluated are written at the ends of the scale: i.e. thepositions of “object 1” and “object 2” in Fig.3.

㩿䌃㪀㩷㪟㪸㫅㪻㪄㪿㪼㫃㪻㪣㪠㪙㪩㪠㪼

㰿㪙㫆㫆㫂㩿㪦㪹㫁㪼㪺㫋㩷㪈㪀㪐

Figure 2. Scenes of Evaluations
Table 1. Media used in evaluations㪮㪼㫀㪾㪿㫋㩷

㪧㫀㪺㫋㫌㫉㪼㩷㫊㫀㫑㪼㩷㪣㪠㪙㪩㪠㪼㩷
㰿㪙㫆㫆㫂㩷

䋳䋰䋰䌧㩷
䋵䋶䋰䌧㩷

㪧㪸㫇㪼㫉㩷䋳䋶䋰䌧㩷㪍 䌩䌮䌣䌨䋺䌓䌖䌇䌁㩷㪎㪅㪉㩷㫀㫅㪺㪿䋺㪯㪞㪘㩷㪙㪌㩷㫊㫀㫑㪼㩷
㩿㫃㪼㫅㪾㫋㪿㫎㪸㫐㫊㪀㩷㪏㪇㩷㫊㪿㪼㪼㫋㫊㩷㫆㫅㩷㪹㫆㪸㫉㪻

LIBRIe

㰿Book

Place
Illumination

800 lx (on the desk plane)

Subjects

Table 3. Instructions to subjects
Reading styles

㪊

㪈

㪊

㪌

㪎

㪐

Step 1 <Rating to evaluation items>
Table 4 shows a typical answer provided by a subject
and used for rating evaluation items. The subject was
required to answer only three values in the table, the othervalues were determined from the three entered values
automatically. Table 4 was then transformed into a 3 by 3
matrix, see Fig. 4. Eigenvectors were calculated from thematrix. These eigenvectors indicate the subject’s own
rating for three evaluation items.
The rating of this subject was thus calculated asfollows: [eye fatigue, body fatigue, readability] = [0.28,
0.33, 0.39]. This result shows that this subject gave mostimportance to readability.

Paper

6
6
6
Male and female students,
Age: 18-24
Sound-proof room㪌A typical example of evaluation sequence is asfollows.

Table 2. Evaluation conditions
Number of subjects㪎㪈㪑㩷㪜㫈㫌㪸㫃
㪊㪑㩷䌓䌬䌩䌧䌨䌴䌬䌹㩷㫊㫌㫇㪼㫉㫀㫆㫉㪌㪑㩷䌆䌡䌩䌲䌬䌹㩷㫊㫌㫇㪼㫉㫀㫆㫉㪎㪑㩷㪜㫏㫋㫉㪼㫄㪼㫃㫐㩷㫊㫌㫇㪼㫉㫀㫆㫉㪐㪑㩷㪭㪼㫉㫐㩷㪼㫏㫋㫉㪼㫄㪼㫃㫐㩷㫊㫌㫇㪼㫉㫀㫆㫉
Figure 3. Scale and indexes used in evaluations

㪧㪸㫇㪼㫉Mediums㩿㪦㪹㫁㪼㪺㫋㩷㪉㪀Direction for subjects
Put a medium against the stand on thedesk. You may initially adjust theangle as you like

Eye fatigue(1)

VerticalPlace a medium flat on the desk.

Body fatigue

(1/3)

(1)

3

Hand-held

Hold a medium in your hand

Readability

7

(1/3)

(1)

Horizontal

Table 4. A typical answer
Eye Fatigue Body fatigue Readability

2.2 Evaluations Method
Subjects were requested to answer three evaluation
items: 1) eye fatigue, 2) body fatigue (except eyes),3) readability.
First, subjects were asked to rate the importance ofthe three evaluation items: (1) to (3). Next, the subjects
were presented with one medium and used it in the three
reading styles: A to C. Last, subjects were requested toevaluate the effect of the three reading styles.
The score of each subject was calculated bymultiplying the following two elements: their initial
importance rating given to the three evaluation items (1)to (3), and evaluated scores of the three evaluation items.
Final score for each reading condition was gained byaveraging the scores of each subject.

㪈㪅㪇㪇
㪇㪅㪊㪊
㪎㪅㪇㪇

㪊㪅㪇㪇
㪈㪅㪇㪇
㪇㪅㪊㪊

3

㪇㪅㪈㪋
㪊㪅㪇㪇
㪈㪅㪇㪇

Rating matrix for evaluation items

(1/7)

㪇㪅㪉㪏
㪇㪅㪊㪊
㪇㪅㪊㪐
Eigenvector

Figure 4. Calculation of eigenvectors with a 3 by 3 rating
matrix for evaluation items
Step 2 < Evaluation of reading styles >Subjects were instructed to read the material in thethree reading styles, and then compare the three pairs of
three reading styles. The evaluation process of Fig. 2 wasapplied to their answer. Table 5 shows a typicalevaluation result for the three reading styles. Table 5 was62

3. Experimental resultstransformed into three matrixes; eigenvectors were
calculated as shown in Fig.5.

Results at the electronic books
Figures 7 and 8 show the evaluation results for theelectronic books, LIBRIe and Ȉbook, respectively.
Both yielded the same tendency, the hand-held” style
received the highest scores, while “vertical” and“horizontal” styles got quite low scores.

Horizontal Hand-held

Vertical

(1)

5

(1/9)

Horizontal

(1/5)

(1)

(1/9)

Hand-held

9

9

(1)

VerticalHorizontal Hand-held

Vertical

(1)

3

(1/5)

Horizontal

(1/3)

(1)

(1/9)

Hand-held

5

9

(1)

Vertical
(1)

5

(1/7)

Horizontal

(1/5)

(1)

(1/9)

Hand-held

7

9

(1)

㪌㪅㪇㪇
㪈㪅㪇㪇
㪐㪅㪇㪇
㪊㪅㪇㪇
㪈㪅㪇㪇
㪐㪅㪇㪇
㪌㪅㪇㪇
㪈㪅㪇㪇
㪐㪅㪇㪇

㪈㪅㪇

Horizontal Hand-held

Vertical

㪈㪅㪇㪇
㪇㪅㪉㪇
㪐㪅㪇㪇
㪈㪅㪇㪇
㪇㪅㪊㪊
㪌㪅㪇㪇
㪈㪅㪇㪇
㪇㪅㪉㪇
㪎㪅㪇㪇

Results for paper
Figure 9 shows the evaluation results for paper.
Differences between the three reading styles are not aslarge as is true with the electronic books. The hand-heldstyle and the vertical style are slightly superior to thehorizontal style.

㪇㪅㪈㪈
㪇㪅㪈㪈
㪈㪅㪇㪇
㪇㪅㪉㪇
㪇㪅㪈㪈
㪈㪅㪇㪇
㪇㪅㪈㪋
㪇㪅㪈㪈
㪈㪅㪇㪇

㪭㪼㫉㫋㫀㪺㪸㫃

㪟㪸㫅㪻㪄㪿㪼㫃㪻

㪇㪅㪍
㪇㪅㪋
㪇㪅㪉

㪇㪅㪈㪎
㪇㪅㪇㪍
㪇㪅㪎㪎
㪇㪅㪈㪏
㪇㪅㪇㪎
㪇㪅㪎㪌
㪇㪅㪈㪐
㪇㪅㪇㪍
㪇㪅㪎㪌

Evaluation matrix for reading styles

㪟㫆㫉㫀㫑㫆㫅㫋㪸㫃㪇㪅㪏㪜㫍㪸㫃㫌㪸㫋㫀㫆㫅㩷㫉㪸㫋㪼

Vertical㪇㪅㪇㪜㫐㪼㪽㪸㫋㫀㪾㫌㪼

㪙㫆㪻㫐㪽㪸㫋㫀㪾㫌㪼

㪩㪼㪸㪻㪸㪹㫀㫃㫀㫋㫐㪞㪼㫅㪼㫉㪸㫃
㪼㫍㪸㫃㫌㪸㫋㫀㫆㫅

Figure 7. Evaluation results for LIBRIe㪈㪅㪇㪭㪼㫉㫋㫀㪺㪸㫃

㪟㫆㫉㫀㫑㫆㫅㫋㪸㫃㪟㪸㫅㪻㪄㪿㪼㫃㪻㪇㪅㪏㪜㫍㪸㫃㫌㪸㫋㫀㫆㫅㩷㫉㪸㫋㪼

Readability

Body fatigue

Eye fatigue

Table 5. A typical evaluation result for the three
reading styles

Eigenvector

Figure 5. Calculation of eigenvectors from a 3 by 3
matrix for evaluation of reading style

㪇㪅㪍
㪇㪅㪋
㪇㪅㪉
㪇㪅㪇

Step 3 < General evaluation >The eigenvectors for each evaluation item in eachreading style were transformed into a 3 by 3matrix. The
general evaluation was obtained by multiplying thismatrix by the rating matrix as shown in Fig.6. For thissubject, the result of general evaluation is as follows:[vertical, horizontal, hand-held] = [0.18, 0.06, 0.75]. This
result indicates that the hand-held style is the mostfavored reading style for this subject. The same procedure
was used for all subjects.
㪇㪅㪈㪎 㪇㪅㪈㪏 㪇㪅㪈㪐㪇㪅㪇㪍 㪇㪅㪇㪎 㪇㪅㪇㪍㪇㪅㪎㪎 㪇㪅㪎㪌 㪇㪅㪎㪌

㪇㪅㪉㪏
㪇㪅㪊㪊
㪇㪅㪊㪐

㪇㪅㪈㪏
㪇㪅㪇㪍
㪇㪅㪎㪌

Matrix of reading styles

RatingGeneral evaluation㪜㫐㪼㩷㪽㪸㫋㫀㪾㫌㪼㪙㫆㪻㫐㪽㪸㫋㫀㪾㫌㪼

㪩㪼㪸㪻㪸㪹㫀㫃㫀㫋㫐㪞㪼㫅㪼㫉㪸㫃
㪼㫍㪸㫃㫌㪸㫋㫀㫆㫅

Figure 8. Evaluation results for Ǜbook
㪈㪅㪇㪭㪼㫉㫋㫀㪺㪸㫃

㪟㫆㫉㫀㫑㫆㫅㫋㪸㫃㪟㪸㫅㪻㪄㪿㪼㫃㪻㪜㫍㪸㫃㫌㪸㫋㫀㫆㫅㩷㫉㪸㫋㪼

㪇㪅㪏
㪇㪅㪍
㪇㪅㪋
㪇㪅㪉
㪇㪅㪇
㪜㫐㪼㩷㪽㪸㫋㫀㪾㫌㪼

Figure 6. Calculation of general evaluation㪙㫆㪻㫐㪽㪸㫋㫀㪾㫌㪼

㪩㪼㪸㪻㪸㪹㫀㫃㫀㫋㫐㪞㪼㫅㪼㫉㪸㫃
㪼㫍㪸㫃㫌㪸㫋㫀㫆㫅

Figure 9. Evaluation results for paper63

4. Discussion

and hardcopy”, Journal of the Imaging Society of Japan
Vol.129, pp.159-165 (1999).
4. M.Koshimizu, D.Tsuda, K.Baba: “Form Factors requiredfor Electronic Paper Media”, IEICE Technical Report
Vol.25, No.10, pp.19-24 (2001).
5. K.Sakuragi, M.Omodani: “Contour Mapping of Readabilityof Hardcopies on the Plane of Optical Density: Characters
and Background –A Guideline for Designing Electronic
Paper-”, IS&T’s NIP20, pp.369-372 (2004).
6. M.Kawakami, S.Matsumoto, O.Myodo: “Reasonable
Lighting Factors for Working Environment in VDT Work
Activities”, Journal of The Illuminating Engineering ofJapan Vol.83, No.5, pp.326-331 (1999).
7. S.Okano, M.Omodani: “Study of Readability as a Target ofElectronic Paper ~Effect of media handling styles onreading tasks~”, Japan Hardcopy 2004, pp.193-196 (2004).
8. S.Okano, M.Omodani, N.Nakata, M.Maeda: “Study ofReadability as a Target of Electronic Paper ~Effect ofmedia handling styles on reading tasks~”, IEICE Technical
Report, pp.13-16 (2005).
9. M.Omodani, S.Okano, E.Izawa, A.Sugiyama: “Studies on
Readability as a Target of Electronic Paper –Current results
and suppositions brought by reading experiments on
displays and papers-”, Journal of the Imaging Society ofJapan Vol.44, No.2, pp.121-129 (2005).
10. K.Kasahara: “The Analytic Hierarchy Process Decision
Methodology: Evaluation of Continuing Professional
Development Point”, Journal of IEICE Vol.88, No.3,
pp.200-207 (2005).
11. T.L.Saaty, L.G.Vargas: “Models, Methods, Concepts &Applications of the Analytic Hierarchy Process”, Kluwer
Academic Publishers, pp.1-26 (2001).
12. S.Okano, M.Omodani, “A Study on Readability as a Goalof Electronic Paper - Effects of media handling styles -”,IS&T’s NIP21 Conference, pp.65-68 (2005).
----------------------------------------------------------------------We have confirmed the advantage of hand-held stylein the two electronic books examined. This result isreasonable because the “hand-held” style is the mostpopular way for us to read books. It indicates why people
do not like to read on displays.
Paper, on the other hand, showed no significant
advantage for the hand-held style. The prominent
advantage of hand-held style shown at electronic books isreasonable when we consider the viewing angle
dependence of readability assumed at the electronic books.
Hand-held style is the easiest style for us to adjust theangle of a medium to the best condition to read. The
universal readability of papers independent to viewing
angle is not supposed to request fine adjustment of its
angle; the advantage of hand-held style may not be clear
at papers. Thus, we consider that the result at papers
indicates the next stage of goal for electronic books:
universal readability independent of reading styles.
We also consider that the light weight and compact
shape of the two electronic book terminals, which enabled
the subjects to use the hand-held reading style, greatly
improved their general attractiveness as a reading
medium; this is ascribed to their ability to freely alter theviewing angle12.

5. Conclusion
1) We confirmed that the "hand-held" reading styleincreases readability and reduces fatigue levels for theelectronic book units of LIBRIe and ȈBook.
2) It is suggested that compact medium design thatallows the hand-held reading style can improve thereadability of electronic displays.

. i Analytic Hierarchy Process (AHP) has been developed by
T.L.Saaty (University of Pittsburgh) for a method of decision
making to choose the best solution among alternatives. AHP
method is based on comparisons between all possible pairs
among all the alternatives. Comparisons between several
viewpoints for evaluations are the first step of the AHP method;ratings for the viewpoints are gotten in this step. This step isfollowed by comparisons between all the alternatives. The
results of ratings are taken into account for general evaluations
at the final step. The comparison performed between pair ofalternatives is considered to bring us accurate sensitivity.

References
1.

2.
3.

M.Omodani: “Concept of Digital Paper and its Technology
Trend”, Journal of the Imaging Society of Japan Vol.38,
No.2, pp.115-121 (1999).
M.Omodani: “What is Electronic Paper: The Expectations”,
SID 2004 Digest, pp.128-131 (2004).
K.Masuda, M.Omodani, Y.Takahashi: “Comparison ofhuman processing faculty for same tasks on visual display

64

Panel Discussion㪪㫆㫄㪼㩷㫋㫆㫇㫀㪺㫊㩷㫆㪽㩷㪠㫄㪸㪾㪼㩷㪺㪸㫇㫋㫌㫉㫀㫅㪾㩷㩽㩷㪧㫉㫆㪾㫉㪸㫄㩷㫇㫉㫆㪻㫌㪺㫋㫀㫆㫅㩿㪈㪀㪓㪠㫄㪸㪾㪼㩷㪺㪸㫇㫋㫌㫉㫀㫅㪾㪕䊶㪪㫌㫇㪼㫉㩷㪿㫀㪾㪿㩷㫊㪼㫅㫊㫀㫋㫀㫍㫀㫋㫐㪑㩷㪉㪇㪇㩷㫋㫀㫄㪼㫊㩷㫊㪼㫅㫊㫀㫋㫀㫍㫀㫋㫐 㫆㪽㩷㪚㪚㪛
HARP film

FEA㪟㫀㪄㪭㫀㫊㫀㫆㫅㩷㪟㪘㪩㪧㩷㪺㪸㫄㪼㫉㪸㩷㫋㫌㪹㪼

Drive Tr

Field emitter㪝㫀㪼㫃㪻㩷㪼㫄㫀㫋㫋㪼㫉㩷㪸㫉㫉㪸㫐㩷㪟㪘㪩㪧㩷㫀㫄㪸㪾㪼㩷㫊㪼㫅㫊㫆㫉㩷䊶㪪㫌㫇㪼㫉㩷㪿㫀㪾㪿㪄㫊㫇㪼㪼㪻㩷㪺㪸㫄㪼㫉㪸㪑㩷㪈㪃㪇㪇㪇㪃㪇㪇㪇㩷㪝㪆㪪㩷㫄㪸㫏㩷㩽㩷㪿㫀㪾㪿㩷㫊㪼㫅㫊㫀㫋㫀㫍㫀㫋㫐㪙㫌㫉㫊㫋 㫆㪽㩷㫎㪸㫋㪼㫉㩷㪹㪸㫃㫃㫆㫆㫅㩿㪝㫉㪸㫄㪼㩷㫉㪸㫋㪼㪑㩷㪈㪍㪃㪇㪇㪇㩷㪝㪆㪪㪀㪢㪼㫅㫂㫀㪺㪿㫀 㪫㪸㫅㫀㫆㫂㪸1㪪㫆㫄㪼㩷㫋㫆㫇㫀㪺㫊㩷㫆㪽㩷㪠㫄㪸㪾㪼㩷㪺㪸㫇㫋㫌㫉㫀㫅㪾㩷㩽㩷㪧㫉㫆㪾㫉㪸㫄㩷㫇㫉㫆㪻㫌㪺㫋㫀㫆㫅㩿㪉㪀㪓㪠㫄㪸㪾㪼㩷㪺㪸㫇㫋㫌㫉㫀㫅㪾㪕䊶㪪㫌㫇㪼㫉㩷㪟㫀㪄㫍㫀㫊㫀㫆㫅㪑㩷㪜㫏㫋㫉㪼㫄㪼㫃㫐㩷㪿㫀㪾㪿㩷㫉㪼㫊㫆㫃㫌㫋㫀㫆㫅㪄㪋㪊㪉㪇㬍㪎㪍㪏㪇㩷㫇㫀㫏㪼㫃㫊㪄㪍㪇䌦䌰䌳

㪘㫇㫇㫃㫀㪺㪸㫋㫀㫆㫅㫊㩷㪽㫆㫉㩷㪟㪛㪫㪭

䊶㪪㫌㫇㪼㫉㩷㫇㪸㫉㪸㫃㫃㪼㫃㩷㫊㫇㪸㫋㫀㪸㫃㩷㪊㪄㪛㩷㫀㫄㪸㪾㫀㫅㪾㪑㩷㪫㪿㫉㪼㪼㪄㪻㫀㫄㪼㫅㫊㫀㫆㫅㪸㫃㩷㪫㪭㪚㪸㫄㪼㫉㪸
㪦㪹㫁㪼㪺㫋

㪬㫇㫇㪼㫉㩷㫍㫀㪼㫎㫇㫆㫀㫅㫋㪛㫀㫊㫇㫃㪸㫐㩷
㫇㫃㪸㫋㪼㪣㫆㫎㪼㫉㩷㫍㫀㪼㫎㫇㫆㫀㫅㫋㪣㪼㪽㫋㩷㫍㫀㪼㫎㫇㫆㫀㫅㫋㪩㪼㫇㫉㫆㪻㫌㪺㪼㪻㩷㫀㫄㪸㪾㪼㪩㫀㪾㪿㫋㩷㫍㫀㪼㫎㫇㫆㫀㫅㫋
265㪪㫆㫄㪼㩷㫋㫆㫇㫀㪺㫊㩷㫆㪽㩷㪠㫄㪸㪾㪼㩷㪺㪸㫇㫋㫌㫉㫀㫅㪾㩷㩽㩷㪧㫉㫆㪾㫉㪸㫄㩷㫇㫉㫆㪻㫌㪺㫋㫀㫆㫅㩿㪊㪀㪓㪧㫉㫆㪾㫉㪸㫄㩷㫇㫉㫆㪻㫌㪺㫋㫀㫆㫅㪕㪤㪼㫋㪸㪻㪸㫋㪸㩷㫇㫉㫆㪻㫌㪺㫋㫀㫆㫅㩷㫋㫆㩷㫌㫊㪼㩷㪹㫉㫆㪸㪻㪺㪸㫊㫋㩷㪺㫆㫅㫋㪼㫅㫋㩷㪽㫆㫉㩷㪸㩷㫍㪸㫉㫀㪼㫋㫐㩷㫆㪽㩷㫊㪼㫉㫍㫀㪺㪼㫊㩷㪸㫅㪻㩷㫄㪸㫂㪼㩷㫇㫉㫆㪾㫉㪸㫄㩷㫇㫉㫆㪻㫌㪺㫋㫀㫆㫅㩷㪼㪽㪽㫀㪺㫀㪼㫅㫋㪚㫆㫅㫋㪼㫅㫋㪄㪹㪸㫊㪼㪻㩷㪭㫀㪻㪼㫆㩷㫋㫉㫀㫄㫄㫀㫅㪾

Visualization using Metadata㪤㪼㫋㪸㪻㪸㫋㪸㩷㫇㫉㫆㪻㫌㪺㫋㫀㫆㫅㩷㩽㩷㪬㫋㫀㫃㫀㫑㪸㫋㫀㫆㫅㩷㪪㫐㫊㫋㪼㫄㪛㫀㪾㪼㫊㫋㩷
㪭㫀㪼㫎㫀㫅㪾

䊶㪠㫄㪸㪾㪼㩷㫉㪼㪺㫆㪾㫅㫀㫋㫀㫆㫅䊶 㪪㫇㪼㪼㪺㪿㩷㫉㪼㪺㫆㪾㫅㫀㫋㫀㫆㫅䊶㪥㪸㫋㫌㫉㪸㫃㩷㫃㪸㫅㪾㫌㪸㪾㪼㩷㫇㫉㫆㪺㪼㫊㫊㫀㫅㪾
Metadata extraction by video, audio, and text analysis

㪤㪼㫋㪸㪻㪸㫋㪸㩷㪧㫉㫆㪻㫌㪺㫋㫀㫆㫅㩷㪝㫉㪸㫄㪼㫎㫆㫉㫂㩷㪹㪸㫊㪼㪻㩷㫆㫅㩷㪤㪧㪜㪞㪄㪎㪘㫉㪺㪿㫀㫍㪼㫊
Multimedia Encyclopedia3㪪㫆㫄㪼㩷㫋㫆㫇㫀㪺㫊㩷㫆㪽㩷㪠㫄㪸㪾㪼㩷㪺㪸㫇㫋㫌㫉㫀㫅㪾㩷㩽㩷㪧㫉㫆㪾㫉㪸㫄㩷㫇㫉㫆㪻㫌㪺㫋㫀㫆㫅㩿㪋㪀㪓㪧㫉㫆㪾㫉㪸㫄㩷㫇㫉㫆㪻㫌㪺㫋㫀㫆㫅㪕䊶㪚㫆㫅㫋㪼㫅㫋㩷㫈㫌㪸㫃㫀㫋㫐㩷㪼㫊㫋㫀㫄㪸㫋㫀㫆㫅㩷㫋㪼㪺㪿㫅㫆㫃㫆㪾㫐㪄㪫㪿㪼㩷㫉㪼㫃㪸㫋㫀㫆㫅㫊㪿㫀㫇㩷㪹㪼㫋㫎㪼㪼㫅㩷㫊㫆㫄㪼㫆㫅㪼㵭㫊㩷㪾㪸㫑㪼㩷㫎㪿㫀㫃㪼㩷㫍㫀㪼㫎㫀㫅㪾㩷㪸㩷㫇㫉㫆㪾㫉㪸㫄㩷㪸㫅㪻㩷㫋㪿㪼㫀㫉㩷㪺㫆㫄㫇㫉㪼㪿㪼㫅㫊㫀㫆㫅㩷㫆㪽㩷㫋㪿㪼㩷㪺㫆㫅㫋㪼㫅㫋㪅㩷䊶㪘㫅㪸㫃㫐㫑㪼㪻㩷㫋㪿㪼㩷㫄㫆㫍㪼㫄㪼㫅㫋㫊㩷㫆㪽㩷㫋㪿㪼㫀㫉㩷㪾㪸㫑㪼㫊㩷㪽㫆㫉㩷㪉㪏㩷㪼㫃㪼㫄㪼㫅㫋㪸㫉㫐㩷㫊㪺㪿㫆㫆㫃㩷㪺㪿㫀㫃㪻㫉㪼㫅㪅㩷䊶㪚㫆㫅㪺㪼㫅㫋㫉㪸㫋㪼㩷㫆㫅㩷㫋㪿㪼㩷㫋㪼㫏㫋㩷㫀㫅㩷㫋㪿㪼㩷㫀㫄㪸㪾㪼㫊㩷㩿㫋㪼㫃㫆㫇㪀㩷㪸㫅㪻㩷㫆㫅㩷㪸㩷㪚㪞㩷㪺㪿㪸㫉㪸㪺㫋㪼㫉㪅㩷㪞㪸㫑㪼㩷㪻㫀㫊㫋㫉㫀㪹㫌㫋㫀㫆㫅㩷㪧㫉㫆㪹㪸㪹㫀㫃㫀㫋㫐㪫㪭㩷㫀㫄㪸㪾㪼
㪚㪞㩷㪺㪿㪸㫉㪸㪺㫋㪼㫉㩷㪸㫇㫇㪼㪸㫉㫀㫅㪾㩷㫆㫅㩷㫋㪿㪼㩷㫊㪺㫉㪼㪼㫅㩷㫉㪼㪺㪼㫀㫍㪼㫊㩷㪺㫆㫅㪺㪼㫅㫋㫉㪸㫋㪼㪻㩷㪽㫆㪺㫌㫊㪅㪞㪸㫑㪼㩷㫃㫆㪺㪸㫋㫀㫆㫅㩷㪻㫀㫊㫋㫉㫀㪹㫌㫋㫀㫆㫅㩷㪼㫏㪸㫄㫇㫃㪼

66

4

New Research Department of NICT
Research Department 1

Research Department 2
Universal Communication

New Generation Network•New Generation Network•New Generation Wireless•Future-oriented ICT Network•Natural Communication•Universal Contents•Universal Platform•Common Reality

Network infrastructure that enables informationexchange anytime, anywhere, and anyhow

More friendlily, more really, and more freely.
We open up a world of new communicationand sensation with universal technology.

Research Department 3
ICT for Safety and Security•Information and Communications Security•Social Environment Safety with ICT•Safe and Secure Foundations forElectromagnetic Waves Applications

Yuichi Matsushima
Network

Establishment of safety and security for secure information
National Institute ofexchange and in preparation for disasters a dangers
Information and Communications Technology

Universal Media Research Center“Watching, Listening, Touching and Smelling” - Ultra reality at your side
The center plays pioneering role in the evolution of media technology, realizing ultrarealistic communication system by multisensory data transmission technology andcreating new communication infrastructure that provides natural and real information
to everybody.

¾ 3D spatial image and sound technology
Holographic 3D video system, 3D sound field
reproduction
System integration for realization of ultrarealistic
communication environments¾ Human perceptual and cognitive
mechanisms
Necessary requirements for ultrarealistic
systems based on underlying principles ofhuman information processing¾ Multisensory interfaces
sensing/transmission/display and integration ofvisual, auditory, tactile and smell information

67

3D video system and multisensory
ultrarealistic communication
system to be realized by 2020

Required Data Rate for Video and Sound Transmission

NTSC
D1

270M

HDTV

1G

Super Hi-Vision

24G
1T〜

Holographic 3D
Data rate（bps）

Fast Ethernet Gigabit Ethernet
100Mbps
1Gbps

10Gbps100Gbps1Tbps

Optical Network R&D Promoted by NICT
● From basic research to applied researchResearch on optical packet switching networks

We have developed the world’s first prototype of optical packet
routing switch
Research promote on ultra-fast communication systems
We promote research on highly-efficient optical communication
systems at more than 160 Gbit/s per wavelength. We have exploited
the JGN2 optical test bed and also have promoted inspection tests
The world’s first optical packet switch, developed by ain the fieldresearch team at NICT
Research on Lambda GRID networks
High performance wavelength (lambda) path networking for
distributed processing system (GRID) is being studied
Tera-bit class high-capacity optical network technology●Optical network technology applied and managed by GMPLS●Technology for the architecture/application of tera-bit class networks●Optical access/core network technology●Optical burst switching technology

Inspection testing scene at JGN2 optical test bed sections（at Otemachi access point）

Giga-bit network for R&D (JGN2)● 20/10 Gbit/s backbones performed by the OXC/WDM network● 63 access points nationwide●Connection with overseas●Optical test beds are provided in two sights for inspection research

Optical test bed（A）

Fukuoka

1G

Optical test bed（B）
Otemachi－Tsukuba 100km (SMF X
2Core)

SendaiKita-Kyushu

Keihanna－Osaka 63km
(NZ-DSF & SMF X 20Core)

Sapporo

NICT Kansai
Okayama

Kochi

Kanazawa

Osaka
1G

NICT Keihanna

Okinawa

NaganoTsukuba

NICT
Koganei
Nagoya

NICT Kashima

Tokyo
Otemachi

To USA

Optical cross connect (OXC)

68

20G
10G
1G
Dark Fiber

Prepared for Prospectus for Holding “Visual interface-toward future”

Limits and Opportunities of “Keitai”
Minoru Etoh
Research Laboratories, NTT DoCoMo
Prospects： Deployment of Broadband Mobile Networks (up toMbps) by 2010 -> Incubation Environment of Killer Applications
Limits in Visual Applications：
Battery, Power Consumption as Vital Issues
Display Devices, QVGA(320×240pels) , VGA and more?
Opportunities toward Future:
Wearable/Detached Interfaces
Low Power DSP, Graphic LSI, CODEC
Distribution of Personal Content, Anywhere and Anytime.
1

Moore’s Law vs. Battery Capacity &CODEC
Quoted from, Paradiso, J.A. and Starner, T.,“Energy Scavenging for Mobile & Wireless Electronics”,
IEEE Pervasive Computing,4(1), 00. 18-27, 2005

CPU Speed

Double per 18 months

Video Compression
Efficiency
Triple per 12 years: .261(1.0) -->MPEG-1(1.25) --> MPEG-4/MPEG-2(1.75) --> MPEG-4 ASP (2.2)--> H.264 (3.0)

MPEG-4AP
MPEG-2/4
2

H.264

Battery Energy Density
Double per 12 years.

H.261 MPEG-1

69

Keitai as Communication Platform and Connected Interface Devices
We may have functional separation of interfacing devices from Keitai.
● Wearable Device for
only Speech Communication ● E-mail Communicator ● Headphone
Of Black Berry Typefor Speech Communication
with Music Entertainment
A4 Size●Smartphone

8" display★ Paper Display■ HD Display■ Video Conference Station★ Wrist-watch style■ Pen Camera● E-mail communicator★ Eyeglasses style● head sets

3

Evolution of 3D Graphic LSI
12/2005

Polygon/sec

XBOX360145W

Production Date and Name
Performance (polygon/sec)

100M

7/2002 500M

ONYX350
140M
500W

WS
1/1996

10M

ONYX IR

2/2003

ATI Radeon9700

XBOX

300M

125M 10/2001
61W

ATI
3/2000PS2 Actually -5M?Radeon7500
45M(Ideally66M）
12/2004
6/2000

10M

GeFORCE256
15M

1998.11
12/1994

1M

PS
1.5M4W
9/19944Portable
Device

ＰＳＰ

33M
6.5Wh
4-6 hr.

2006

Qualcomm MSM
4M(dual CPU)

Dreamcast
3M 22
W

2006

GeFORCE5500

ONYX
100K

PC／Game Device

2004

210k

Z3D

2.67M
12/2004
NintendoＤＳ
120k 3.1Wh
6-10 hours durability

70k

1995

2000

70

2005

Trend (1): TV broadcasting & TV set
 New Services bring new TV devices into market.
Monochrome and Color

Digital and Flat Panel TV

120

Color TV

10,000
8,000

80

6,000

60

CRT

4,000
40

PDP
/LCD

Shipments[K Units]

2,000▲▲ Color TV
Monochrome TV service started
service started

95

01997 1998 1999 2000 2001 2002 2003 2004 2005▲
BS Digital start

97

91

89

1990
93

85

83

87

1980
81

79

73

77

75

67

1970
71

65

63

1960

61

57

1953
55

53

0

69

20

59

% of household

12,000

Monochrome TV

100▲ BS service
started80% penetration in10 years▲ Digital
Terrestrial start

Flat Panel TV : 20% penetration in 2005

since the services started

Flat panel TV : 5 million units
CRT TV : 3.4 million units

Monochrome TV: 1953 -> 1960(80%)
Color TV: 1960 -> 1974(86%)

Kazuo Okamura Matsushita/Panasonic - 1

Trend (2): Screen Size
 HD Broadcasting has made TV screens larger.

200
3
199
8

199
0

2006

50”, 58”, 65”, 103”
Full-HD PDP (PZ600)
103” Full-HD

1991
1965
19” Color
1960
1952
21” color TV set
mono-TV set(TC-96G)
chrome TV (K21-30)(17K531)

17”

1952

1960

1970

1985

21”

36”

Hi-Vision
Color TV TV set(TH-36HD)

50” Full-HD(TH21-H88GR)

1980

1990

2000

2006

Kazuo Okamura Matsushita/Panasonic - 271Trend (3): Storage Media Progress of storage media and video codec has leadthe changes of content format.
100Mbps

FTTH(100M)

H.264
Standard

HD resolution
10MbpsFTTH(10M)

MPEG-2Standard

SD resolution
CD

ISDN

MPEG-1
Standard

1Mbps

ADSL (8M)Video CDADSL (1.5M)

Below SD resolution

‘82

‘88

‘90

‘92

‘94

‘96

‘98

‘00

‘02

‘04

‘06

Kazuo Okamura Matsushita/Panasonic - 3

Discussion points
 How bigger? How finer?
 More than 50 inch ? 4Kx2K, 8Kx4K ?

 Will Internet change the way we watch the TV?
 IP network infrastructure is becoming ready for video.

 What will be the role of storage media ?
 HD package contents will be available pretty soon.

Kazuo Okamura Matsushita/Panasonic - 4

72

1

Environmental Issues in LCDs
3R (Recycle, Reduce, Reuse) Oriented R&D
and Manufacturing of Liquid Crystal Displays
Shunsuke Kobayashi･Liquid Crystal Institute and the Graduate School of
Science and Engineering
Tokyo University of Science, Yamaguchi･JSPS 130th Committee of Optoelectronics
LCI, TUS-Y

3R (Recycle, Reduce, Reuse) Oriented R&D
and Manufacturing in LCDs
1. Prime Minister of Japan Mr. Jun-ichiro Koizumi suggested“3R Initiative” in 2004 at Sea Island Summit.
2. Kyoto Protocol for Reducing CO2 Exhaustion came into effect onJanuary 15, 2006.
3. RoHS (Restriction of the Use of Certain Hazardous Substances inElectrical and Electronic Equipment) was enacted on July 1, 2006
on Hg, Cd, and others in EU.
4. Collecting and recycling systems for home appliances (TVs,
Refrigerators, Air Conditioners) and personal computers havestarted since July, 2005 in Japan.
We have to comply with the legislation of electrical andelectronic products issued by individual government.
LCI, TUS-Y

73

2

Reduction of Power Consumption in LCDs
Using Nanoparticle Technology

3

Almost all the physical properties of existing materials will be modified
by doping nanoparticles.
For example:
polymer film + nanoparticles → optical properties changes•

Improvement of the power consumption of back lights for LCDs that
dominate 70%～80% of total power consumption can be done by theenhancement of luminous efficiency of phosphors and LEDs by using
nanoparticles. 40inch LC-TV 120W(CCFL) and 360W(LED)

2. Reduction of power consumption of LCDsA) Field sequential fullcolor LCDs will realize the reduction of
their power consumption by 30%～50%.
B) Reduction of operating voltage of LCDs will be realized by
25%～40% by doping nanoparticles.
LCI, TUS-Y

Reduction of Power Consumption
in the Manufacturing of LCDs

4

The sizes of LCDs spreads from those for mobile phones, personal
computers and to TV having 45 inch to 60 inch diagonal.
Examples:
1) One drop filling (ODF) technology reduces the time for filling LCs into
LCD cells by 100 times.
2) Caret free glass cutting using lasers reduces the time for glass cutting
by 50 times.
These technologies are being utilized for 8G (2200×2500mm2 ) LCD
production.
3) Doping nanoparticles into individual material for LCD production will
be usefull for reducing power consumption.
Conclusions
1) We have to obey laws for environmental issues.
2) Nanoparticle and laser technologies are useful for reducing power
consumption in LCDs themselves and their manufacturing.
Thank you for listening

74

LCI, TUS-Y

Committee Member
General Chair
Vice-Chair

Yasuharu Suematsu, NII (National Institute of Informatics)
Shunsuke Kobayashi, Tokyo University of Science, Yamaguchi
Yuichi Matsushima, NICT (National Institute of Information andCommunication Technology)

Steering Committee Chair
Kazumasa Enami, NHK (Japan Broadcasting Corporation)
Organizer

Committee Member

Shunkichi Isobe, NICT
Tomoyuki Miyamoto, Tokyo Institute of Technology
Haruo Okuda, Shonan Institute of Technology
Hiroshi Saito, NTT (Nippon Telegraph and Telephone Corporation)
Katsunori Sakai, Takayanagi Foundation for Electronics Scienceand Technology
Masao Takahashi, Toshiba Corporation
Michiaki Katsumoto, NICT
Hiroshi Kikuchi, NHK
Toshiaki Maruoka, Toshiba Corporation
Eiju Matsumoto, The Society of Historical Metrology
Shuichi Matsumoto, KDDI Corporation
Takayuki Nagata, National Science Museum
Yasuo Omori, Takayanagi Foundation for Electronics Scienceand Technology
Yoshimitsu Ohtani, NTT
Yoshiaki Shishikui, NHK
Masayuki Sugawara, NHK
Masami Sukeda, Hitachi, Ltd
Nobuji Tetsutani, Tokyo Denki University
Akio Toda, Mitsubishi Electric Corporation
Yoshiyuki Yashima, NTT

C 2006
The Institute of Image Information and Television Engineers
The Institute of Electrical Engineers of Japan