ITE Trans. on MTA Vol. 2, No. 1, pp. 74-81 (2014)

Copyright © 2014 by ITE Transactions on Media Technology and Applications (MTA)

T2V:
New Technology of Converting Text to CG Animation
Key words:
1. Introduction
Masaki Hayashi † (member), Seiki Inoue †† (member), Mamoru Douke †† (member),Narichika Hamaguchi †† (member), Hiroyuki Kaneko †† (member),
Steven Bachelder ††, Masayuki Nakajima † (member)
Abstract

We have developed a technology that enables the creation of TV-program-like CG animation

generated automatically from text-based script. We call this technology "T2V (Text-To-Vision)" and havedeveloped a PC application that we call the "T2V Player." The application allows users to create animations
instantly by simply typing in text in a customary fashion as when writing on a word processor. The T2V Player
employs a unique framework with a capability to accommodate any text processing module to realize any type oftext-to-animation conversion. The T2V Player is not only an animation tool for writers creating animations fromtheir texts, but also an application of various text-to-vision services such as automatic animation generation
form Web sites and other text sources. This paper describes the concept of T2V and the technical specifications of
the T2V Player and its benefits.
Keywords: Text visualization, Computer animation, Authoring tool, Text-to-animation, TVML.

Hello, welcome to theprogram.㻌
(upshot)㻌
This is the news from CML.㻌
(wait)㻌
First, top news.㻌
(nod)㻌
subimage: news01.jpg㻌
The CML is a puzzle withbillions of possible䈈䈈䈈㻌

1. Introduction
It is common today to see digital content made by nonprofessionals. This is particularly true on the Internet
where numerous armature movies can be found in blogs
and Social Network Systems such as Facebook, Twitter,etc. and in movie sharing services such as YouTube.

Text-to-vision㻌

T2V㻌

ANIMATION㻌

TEXT㻌

However, TV-program-like film clips with highFig.1 Concept of T2V.

production value are found less frequently. There is stilla considerable barrier to the creation of this type ofcontent for non-professionals.

available to non-industry individuals. Our aims are to

Based on this background, we have been developing a

open the rich TV program production methodology in thetechnology of producing TV-program-like CG animations

world of UGC (User-Generated-Content).

(CGA) automatically from text-based script. We call thisTo facilitate this, we have developed a PC application

technology "T2V: Text-To-Vision" (Fig. 1). One of thethat we call the "T2V Player". This application allows

significant characteristics of TV content is its large

users to create animations instantly by simply typing invariety. The genre of TV programs includes news,

text in a customary fashion as when writing on a word

drama, reality shows, sports, variety, music, education

processor. This is expected to lower the barrier to theetc. The TV program production industry has

creation of CGA dramatically.

accumulated production methodologies for many years.

The T2V technology is built with the TVML (TV program

But these methodologies have not been made readily

Making Language) which was proposed by NHK (Japan
Broadcasting Corporation)1). TVML is designed as a special

Received April 5, 2013; Revised July 29, 2013; Accepted September 27, 2013
† Uppsala University

computer language that describes the content of TV(Visby, Sweden)

program. A TVML Engine creates the complete CG-†† NHK Science and Technology Research Laboratories

animated sequence by integrating real-time CG character(Tokyo, Japan)

74

Paper » T2V: New Technology of Converting Text to CG Animation

animation, synthesized voices, superimposing, CG camera

understanding technology to achieve higher level

work, font display, movie file play back, still image displays,

animation generation, the last one is the automatically2)- 5)

sound file play back etc

converting annotated fiction text into 3D animations9).

.

In the T2V Player, users begin by writing their

These are only relatively successful as there are many

content in a common but descriptive language. The text-

difficulties to overcome; Such as: managing theprocessing module then converts this to TVML, which is

ambiguity of natural language, how to give appropriatethen converted into CGA by the TVML Engine.

attributes to CG objects, how to design autonomous

In order to realize the text processing, the T2V Player

virtual agents and so on. In fact, the trials of rebuilding

provides a special framework to accommodate text-

appropriate animation analytically from any given

processing plug-in modules. We call this module thearbitrary instruction written in natural language, often"Flexible Interpretation Loader" or "FIL". A number oftend to result in incomplete visual representation

FILs can be cascaded. These FILs process the user input

confined to a very narrow application domain.

text in various ways ultimately converting it to TVML

On the contrary of this straightforward AI method,

script.

there is another approach for the problem. This

This framework is both simple and flexible allowing

approach focuses on TV-program-like animation

developers to plug in their text processing modules toproviding users with an "authoring tool" instead of thethe T2V Player easily. This opens up for various ways ofuser providing arbitrary text to make animations. Thereusing the T2V Player not only for the easy-to-use CGA

are a variety of services currently available especially on

creation tool but also for automatic conversion of Web

the Web. These are, for example, "Xtranormal",sites to animation as explained later in this paper.

"GoAnimate", "SitePal" etc. Among these services, theThis paper is organized as follows. Chapter 2 describes"Xtranormal" is particularly good. Users can create high-the related works. Chapter 3 is a brief introduction ofquality CG animations and publish results by pressing aour previous work on TVML. Chapter 4 describes the

few buttons followed by typing in the speech text andoverall technology of T2V. Section 4.1 describes the new

specifying character actions and camera work etc. byarchitecture of cascaded text processers. Section 4.2

using a special GUI (Graphical User Interface) provided

describes the text-processing framework. Section 4.3

on the Web browser. This service is popular and many

describes the newly developed T2V Player. Section 4.4

movies have been uploaded on the site. However, theintroduces a field test conducted with the T2V Player asservice has been completed on the Web and is limited by

distributed freeware to individuals and discusses thethe user creating animations through the GUI on the

results. Section 4.5 describes the automatic movie

Web page. Therefore, it is not a simple structure to

generation from Web sites, which became possible by thecreate animation from text which we intend to realize

T2V Player architecture. Finally, Chapter 5 contains thewith the T2V. Other services such as "GoAnimate" andconclusion for this paper.

"SitePal" are similar in this regard.
2.2 Studies based on TVML

2. Related work

As mentioned in the introduction, T2V is based on
TVML. TVML is unique as a language to describe andIn this chapter, we will survey related work of text

generate TV programs at a script level. NHK, which

visualization, which is the basic concept of the T2V.
2.1 Text visualization

proposed TVML, has performed several studies based on

Technology to generate animation from text is called

TVML10)-12). The studies are primarily for TV content"Text-to-animation" and has been studied for many

production in the present TV industry. While the aim ofyears. In particular, the research on analyzing text

the T2V is to encourage UGC through user communities

written in natural language to visualize the written text

on the Internet.

by CG has been mainly carried out in the area ofAs an animation production tool using TVML, Digital

Artificial Intelligence (AI.)6) In this text-to-animation

Movie Director (DMD) has been studied for years 13).

research area, many attempts have been made. Three

DMD is an authoring application with a dedicated GUI.

such examples are "WordsEye"

7)

which generates a

A user can create CG animation on a PC with the DMD

static CG scene from given text written in natural

installed, and then publish it on the Web. The structure

language, another is "Confucius" 8) which uses 3Dof the GUI in DMD is similar to that of "Xtranormal"

modeling/animation techniques with natural language-

mentioned in the previous section. DMD uses the TVML75

ITE Trans. on MTA Vol. 2, No. 1 (2014)

internally, however, the TVML is hidden from the user.

to the basic elements used in the real TV production

In addition, since the script is not a plain text like as isshown in Table 1. As seen in this table, the TVML

the case in the "Xtranormal", it cannot be applied tocovers almost all the functions needed to produce

non-production purposes such as automatic movie

information types of TV programs. Each event provides

generation, which is explained later in this paper.

number of commands, and each command has a number

Next, we will introduce TVML, which is the baseof parameters that specify the behavior of the command.

technology of T2V, followed by the description of the

In the example script shown in Fig. 2, a scene istechnical details of the T2V.

described where "The camera creates a two-shot ofcharacter A and character B, then character A looks at

3. TVMLcharacter B, then character A says: 'How's it going?',then character B says: 'Awful', then the sound file

As mentioned above, T2V technology is built on thenamed 'boo.mp3' is played back ."TVML. In this chapter, we will explain the TVML tohelp the comprehension of the T2V technology described

3.2 TVML Engine

in the next chapter.

Fig. 4 shows the configuration of the TVML Engine. It
consists of three parts: The "Sequencer", which reads the

3.1 The language

3. TVMLTVML
is a special computer language designed to3. TVML

given text, line by line, and the "Parser", which parses

produce TV-program-like CG animation by a computer.

the one TVML line and generates corresponding

Unlike programming languages, the nature of the TVML

instructions. The "Animation engine" then generates

is rather a "scripting language" which is similar to areal-time audio and video according to the instructions

script structure used in actual TV production. Fig. 2

coming from the parser. The animation engine executes

shows an example script of the TVML. Each line

this immediately upon receiving the trigger of action

represents an event to be performed. The software called

The language

the TVMLTheEngineproduces CG animation in real-time
language

Table 1 TV production versus TVML.

by reading the TVML from the beginning to end, line byCategory
Studio shotline.
Fig. 3 shows the format of an "event" as one line. The
event consists of three parts: "Event type", "Command",and "Parameters". The event types of TVML correspondFootage
Telop
Superimpose
Art workcamera: twoshot (target1=A , target2=B )㻌
character: look ( name=A , target=B ) 㻌character: talk ( name=A, text=“How's going?”)㻌
character:talk ( name=B, text=“Awful” )㻌
3. TVMLsound: play( filename="boo.mp3" )㻌
............㻌
TVML script 㻌SoundTV production
Studio set
Prop
Actor
Lighting
CameraVideo playback
Text, image display
Text, image display
Making figureSound effect
Audio mixing

TVML
set
prop
characterlightcamera
movie
title
super
drawing
sound
sound

3.2 TVML Engine
3.2 TVML Engine

TVML Engine㻌The language
Animation㻌
Fig.2 Example of TVML script.

character: talk ( name = Bob, text = "Hello" )(YHQWW\SH&RPPDQG

3DUDPHWHUV
Fig.4 Configuration of TVML Engine.

Fig.3 Format of "Event".

76

4. T2V
4.1 Mechanism of T2V

Paper » T2V: New Technology of Converting Text to CG Animationfrom the parser. Therefore the video and audio output

and retrieves corresponding FIL and input text to thecan be obtained immediately after receiving one or more

FIL which then performs the process. Finally when a

TVML lines given from the external application. ThisFIL outputs the text whose first line is as follows, thenenables the engine to work as a perfect interpreter of

the text is passed to the TVML Engine.
//! TVML

given TVML script.

Finally, the TVML Engine visualizes the processed

4. T2V

text resulting in the CG animation. Any number ofcascade FILs can be connected.

4.1 Mechanism of T2V
The basic concept of T2V is "to visualize any arbitrary

4.2 Mechanism of FIL

text". However, our approach is not to make a complete

The details of the FIL can be described as follows: A

text-to-animation AI machine with the capability of

FIL is represented by a text file in XML format named

interpreting any arbitrary input text given by users in"FIL description". The FIL description is an XML filenatural language as explained in section 2-1. Instead ofwith the extension "fil." Fig. 6 is an example of a FIL

developing an "Ultimate Machine", we provide a

description. As in the figure, <name> tag gives FIL

framework enabling many types of text-to-animation

name; <engine> tag specifies an entity of the text-

engines made by developers. As a method to achieve

processing module. In this example, the <stage> tag is

this, we provide a plug-in socket for modules to process

specially defined for the T2V Player as described in thethe 4.
given
text string as an input to generate a TVML
T2V

next section. The content of <stage> tag here is TVML

script which is then fed into the TVML Engine andinitialization script which is referred by the text

4. T2V
playedin order to ofobtain4.1back
Mechanism
T2Vthe final CG animation.

processing module.

Fig. 5 shows the basic configuration of T2V. Input text

The text processing module specified by the <engine>is fed into the text processing module called the FIL

tag is a dll (Direct Link Library) in the Windows OS(Flexible Interpretation Loader). The processed result is

implementation. In this dll, only the following simple

output in an arbitrary text format. As shown in theC++ function is defined.

4.1 Mechanism of T2V

char* filproc (char* intext) ;figure, the text conversions are performed one after
another as the FILs cascade, and then ultimately the

The function "filproc" processes the given string data

last FIL outputs TVML script that is then visualized by"intext" then returns the output string data.

4.2 Mechanism of FIL

the TVML Engine resulting in the final animation. Each

4.2 Mechanismof FILSpecificationsof input andoutput text processed by the

FIL has its unique name called the "FIL name". The FIL"filproc" are left to the developer, except the first line ofidentification format is as follows: The first FIL informs

the text as explained in the previous section. The T2V

the system which FIL will follow and become the second

Player provides only the framework allowing the

FIL. The second FIL name is written in the first line (asindicated below) of the text output from the first FIL.

"[POYHUVLRQ   HQFRGLQJ 87)"!
ILO'HVFULSWLRQ!
  QDPH!7HVWQDPH!
  FDSWLRQ!7HVW6KRZ)RUPDWFDSWLRQ!
  HQJLQH!%HH),/GOOHQJLQH!
  LFRQ!WHVWMSJLFRQ!
  KHOS!WHVWKWPOKHOS!
  VWDJLQJ!
    QXPRIFKDU!QXPRIFKDU!
    PRGHOBIRUBFKDU  FKDUDFWHU $ !      FKDUDFWHU  RSHQPRGHO PRGHO %8&&+8                     ILOHQDPH EXFFKX?EXFFKXEP       FKDUDFWHU  FDVWLQJ QDPH $PRGHO %8&&+8       FKDUDFWHU  VHWYRLFH QDPH $YRLFHW\SH PDQ       FKDUDFWHU  SRVLWLRQ QDPH $]      PRGHOBIRUBFKDU!
    VHWWLQJ!
      VNLSVFULSW   VZLWFK RQ       VHW  RSHQPRGHO QDPH +$//                     ILOHQDPH KDOO?KDOOREM       VHW  FKDQJH QDPH +$//       OLJKW  IODW QDPH )/$7Y[        OLJKW  IODW QDPH )/$7Y[        FDPHUDSRVLWLRQ QDPH $FDP]   YDQJOH        VNLSVFULSW   VZLWFK RII     VHWWLQJ!
  VWDJLQJ!
ILO'HVFULSWLRQ!

//! textfilter="FIL name"
The system checks the "FIL name" in the line above,....................㻌....................㻌....................㻌

7H[WLQSXW㻌),/

),/

//! filter = "FIL#2"㻌
....................㻌...................㻌....................㻌....㻌),/Q

790/
(QJLQH//! TVML㻌....................㻌....................㻌....................㻌2XWSXW㻌

Fig.6 Example of FIL description.

Fig.5 Configuration of T2V Player.

77

ITE Trans. on MTA Vol. 2, No. 1 (2014)

developer a high degree of freedom.
A FIL is deployed in a folder on a file system. The
single folder is treated as the entire module of the FIL.
The folder contains a FIL description file and a text
processing module (a dll in Windows OS) and other data
files. The other data files include: a configuration filereferred by the text-processing module, an instruction
manual of the FIL and all the material data such as CG
character modeling data, image data, sound data etc.

Fig.7 T2V Player.

referred by the TVML Engine when it is played back.
4.3 T2V Player

Table 2 Available commands and tags (excerpt from approx. 60
commands, 8 tags).

T2V Player is developed as one of the applications
using the FIL framework as described in the previous&RPPDQGV

section. The T2V Player is developed to allow users to

KDSS\ create animations instantly by typing in text as done

ERZ 
QRG 
QR 

when writing words on an ordinary word processor. The
specification of the text is designed intuitively so that

ORRNDWIHOORZ 

non professionals and without previous knowledge can

ORRNDWFDPHUD 

easily create animations with it.

VKDNH VWDQG VLW The application runs on a Windows PC. Fig. 7 shows
the screen-shot of the application. The left side of thescreen is the animation playback area and the right side

ZDONQVWHS 

is the text editing area. Users can type in script in the
text editing area, and press the play button in the lower

WXUQQGHJUHH 

left to generate a CG animation with the CG charactersXSVKRWRIQ 

appearing and acting according to the script written.

WZRVKRW 

Table 2 shows an excerpt from the specification of the

WDJVV

command and the tag used in the T2V script.

PDLQLPDJH

Approximate sixty commands and eight tags areprovided in the current version. Fig. 8 shows an

VXELPDJH

example of the script written using the commands andthe tags. Fig. 9 shows several snapshots from the%*0resulting animation. The main part of the script is

7KHFKDUDFWHUFKDQJHVLWVIDFLDO
H[SUHVVLRQ KDSS\ 
7KHFKDUDFWHUERZV
7KHFKDUDFWHUQRGVRQFH
7KHFKDUDFWHUVKDNHVLWVKHDGRQFH
7KHFKDUDFWHUORRNVDWWKHRWKHUFKDUDFWHU
7KHFKDUDFWHUORRNVDWDFDPHUDZKLFKLVVKRRWLQJWKHFKDUDFWHU
7KHFKDUDFWHUVKDNHV
7KHFKDUDFWHUVWDQGVXS
7KHFKDUDFWHUVLWVGRZQ
7KHFKDUDFWHUZDONVQVWHSVDKHDG([ZDONVWHS  
7KHFKDUDFWHUWXUQVQGHJUHHV([ WXUQGHJUHH  'LVSOD\DFORVHXSVKRWRIFKDUDFWHUQ([ XSVKRWRI$ 'LVSOD\DWZRVKRWRIWKHWZR
FKDUDFWHUV$QLPDJHGHVLJQDWHGE\DILOHQDPHZULWWHQDIWHUWKLVWDJZLOOEHGLVSOD\HGWR
ILOOWKHGLVSOD\DUHD$QLPDJHGHVLJQDWHGE\DILOHQDPHZULWWHQDIWHUWKLVWDJZLOOEHGLVSOD\HGRQDYLUWXDOPRQLWRUGHILQHGLQD'&*VHW$VRXQGILOHGHVLJQDWHGE\DILOHQDPHZULWWHQDIWHUWKLVWDJZLOOEHSOD\HG

speech text. "Command" is represented by a command//! textfilter="HanaTalk"(A)
title:FOOD EXPLORE
subtitle:Gukbap will cure your hangover<br>in Seoul(B)(happy, little)
In the afternoon of the second day when I stayed in Seoul,
my Korean friends took me to a Korean style cheap
restaurant.
(nod)
(C)
subimage:seoul1.jpg
This is it!
The restaurant's speciality is'Gukbap', a famous Korean soup with rice.
(D)mainimage:seoul1.jpg
A raw egg and dried seaweed are served in a small bowl on
the side.
You're supposed to put them in the soup, mix well and eat.
(back to studio)This tastes very good!

name in parentheses, which controls character's
movement, camera, etc. "Tag" is also available to display

4.3 T2V Player

image-files and playback sound-files, movie-files
playback etc. As shown in the fig. 9, the scene (A) is thetitle and the sub-title display, the scene (B) is thecharacter speech with automatically added lip-sync withsuperimposed subtitles, the scene (C) is the sub-image
display with automatic camera panning, and the scene(D) is a full screen display of the main-image.
There is no limitation in the number of CG characters.
Each character can be distinguished by giving "A:", "B:","C:" and so forth at the head of the speech text or thecommand. All the commands and tags are described in
the form of template in XML format in a FIL

Fig.8 Example of a T2V script.

description. This allows developers to easily define new78

Paper » T2V: New Technology of Converting Text to CG Animation

recorded number of downloads on the download log wasapproximately 20000 one year after the first download
started. The distribution and resulting downloads havecontinued since that time. The current version bundles
more than twenty show formats with numerous example(A)

scripts.

(B)

The number of the active users is difficult to count.
However, based on the number of downloads, weapproximate that several thousand people have used the

4.4 Field
test of T2V Player
software.
We investigated
the number of movies madewith the T2V Player on the movie sharing service"YouTube" by giving a keyword "T2V" in the search(C)function. Approximately 350 movies were found on the(D)site, (accessed July 2013). And several particularly

Fig.9 Snapshots from the playback of the script of Fig. 8.

interesting types of movies were uploaded. For example,
an elderly woman 70-79 years of age has uploaded aseries of movies. These movies are mostly inspired byevents in her ordinary daily life, such as visits nearby,
taking pictures, etc. and then used to create stories on
T2V. The T2V Player does not have a cumbersome userinterface requiring users only to type in text similar towriting E-mail. This dramatically lowers the barrier interms of software operation. The T2V movies made by
the elderly could be a good example to prove theinclusive advantage of the T2V methodology.
As we conducted several workshops using the T2V
Player, one workshop will be introduced below. This
workshop occurred in 2010. The participants were in
their 20s and did not have any experience in creating
CG animation. They received instructions for a period of
one hour on how to use the T2V Player. They were thenFig.10 List of show format.

asked to create their own T2V scripts within one week.
The results were positive and each of the participants
commands and tags by adding the corresponding

successfully created their scripts. We can say that thistemplate to the FIL description.

proves its usability that enables non-professionals to

The T2V Player provides a list of show formats fromcreate their own CG animations with the T2V Player.

which users can choose and apply to the text input. Fig. 10

The evaluation of the quality of those twenty animations

shows the GUI for choosing the show format from theis inevitably subjective. However, one script out of thelist. The show format contains a number of instructions

twenty was deemed particularly well made resulting inthat direct which characters appear in which studio set,a high-level production. This suggests that the T2V

camera angle, camera work, BGM playback etc. The

player is not only an easy-to-use animation tool but also

basic concept is to combine a "Text" and a "Show format"

an effective content creation tool, which could produce

4.4 Field
test of T2V Player
to generatean "Animation".
Internally, one show format

high-level results by creative users. Finally, listed below

corresponds to one FIL. The T2V Player loads theare excerpts from participants' hearings.
* This is a quite easy way to make animation unlike

prepared FILs at the start-up and creates a list of showlearning computer programming.

formats.

* It might be useful for pros as well, if the quality of

4.4 Field test of T2V Player

graphics and synthesized voice becomes better.

The Japanese version of the T2V Player has been
distributed as freeware14)* It is amazing that I have made animation by myself,

since March 2009 and the79ITE Trans. on MTA Vol. 2, No. 1 (2014)

Web page㻌

CGA㻌

4.5 Automatic movie generation

HTML㻌<!DOCTYPE html
PUBLIC"-//W3C//DTD
XHTML 1.0
Transitional//EN"㻌......㻌
HTML㻌

Fig.11 Automatic generation of news show.

just only after a few hours practice.
* I wanted to insert commands by icons or from pull
down menus instead of writing the command words.

FIL#1

FIL#2

//! textfilter="FIL#2"㻌
(opening)㻌
(upshot)㻌
(bow)㻌
Hi I'm Cathy Rollen
this is news fromreuters.㻌.......㻌
T2V script㻌

TVML
Engine//! TVML
change(name=newsset)㻌
character:casting(name=A, model=Hana)㻌
camera: position(x=0,
y=1.5, z=3.5)㻌
character:talk(name=A,
text="Hi I’m Cathy…")㻌
......㻌
TVML script㻌

According to the comments, we can say that the T2V

Fig.12 Configuration of the automatic generation of news show.

Player clears the basic functional requirement needed
for making animations. However, there is still a room for

it is possible to visualize other types of Web sites andimprovement to enhance the usability and functionality.
4.5 Automatic movie generation

contents. We have prototyped a number of FILs

In this section we will introduce uses for the T2V

converting Web sites such as weather forecasts, BBS

Player beyond that of an animation production tool. The(Bulletin Board Systems), E-commerce etc. Apparently,

T2V Player can be used for automatic visualization of

the converter would need to be re-configured if theexisting textual information. As an example, we will

original site's HTML configuration has changed.

describe an automatic news-show generation from

This application is still in the trial phase however, it isHTML used for News sites. This is also an example ofpossible to perform various enhancements in addition to

how to use cascaded two or more FILs in the T2V Player.

the features described above. The enhancements could

Fig. 11 shows the scene of automatic generation offor example be; to provide specially compiled news shows

news-show produced from the News site "Reuters". The

based on the search result of RSS of the news source

system generates the animation when the HTML fromaccording to the user's keyword input. Additional

the site is copy and pasted into the T2V player. Fig. 12

features could provide native language news shows

shows the configuration of the automatic generation oftranslated from other languages by inserting anothernews-show. The two FILs are used to convert HTML to

FIL of machine translation to the chain of the FILs, andTVML. The operation of each FIL is as follows.

more.

(1) FIL #1 for analyzing the HTML5. Conclusion

The FIL analyses the tag structure of the HTML anddivides it into each news article then extracts a news

In this paper, we described the T2V technology, which

title, a main body of the article, JPEG images and theenables users to create CG animation automaticallydate and time. The FIL processes these elements andfrom text-based scripts. We have developed the PC

generates the T2V script as shown in Fig. 12 together

application named "T2V Player" and have distributed itwith the first line specifying the next FIL name to which

as freeware 14) encouraging UGC. The T2V Player isthe script is going to pass.

based on the TV program Making Language (TVML)(2) FIL #2 for producing news show

which is designed as a computer language to describe

This FIL is one of the standard FILs prepared by the

TV-program-like animation. In order to facilitate a user-

T2V Player. It has some special tags which for example

friendly and more inclusive format of text input, the T2V

superimpose the news title to be scrolled in with soundPlayer employs a special framework to accommodate

effect. The T2V Player plays back the script coming fromtext-processing modules on top of the TVML. The text

the first FIL, and the automatic generated animation isprocessing module is called FIL. The FIL processes user

obtained.

input text and converts it to a TVML script for

By applying the same framework as described above,generating animation. Based on this framework, we
80Paper » T2V: New Technology of Converting Text to CG Animation

Masaki Hayashi received Dr. Eng. fromhave also developed an automatic news show generation

Tokyo Institute of Technology in 1999. He worked at
NHK Science and Technology Research Laboratories to
conduct research of content creation from 1986 to 2006.
After the career as a director and a research fellow atseveral profit corporations, he began working at the
Department of Game Design, Uppsala University,
Campus Gotland as Associate Professor.

function in the T2V Player. This is done by converting
HTML from the Internet news site to a TVML script byusing two cascaded FILs.
We consider that the T2V Player is not only a useful
tool for UGC but also a valuable tool for various text-

Seiki Inoue is a senior research engineer in
NHK Science & Tech. Research Labs. He got B.S.
degree in electrical engineering, M.S. and Ph.D.
degrees in electronics engineering all from Univ. ofTokyo in 1978, 1980 and 1992, respectively. He joined
NHK in 1980 and moved to Science & Tech. ResearchLabs. in 1983. From 1995 to 1998, he worked for ATR
Media Integration & Communications Research Labs.
From 2002 to 2005, he worked at NHK Broadcast
Engineering Department. His research interests
include image and video processing, CG and visualexpression, and text-to-video technology (TVML: TV
program Making Language).

visualization applications using text-processing
techniques. We now aim to develop the system further tocreate a more interactive and intelligent televisionsystem based on the T2V technology.

[References]1) M. Hayashi: "Automatic Production of TV Program from Script: A
proposal of TVML", Proceedings of the 1996 ITE Annual
Convention, S4-3, pp.589-592 (1996)
2) M. Hayashi: "TVML (TV program Making Language) -Automatic
TV Program Generation from Text-based Script-", Proceedings ofImagina'99, pp.119-133 (1999)
3) M. Hayashi, M. Douke, K. Ariyasu, N. Hamaguchi: "TV Program
Making Language -Benefits of Object-based Content-creation
Method", IBC2001 (2001)4) M. Douke, N. Hamaguchi, M. Hayashi, K. Ariyasu: "Production of
TV Programs on a Single Desktop PC -Special scripting language,
TVML, generates low-cost TV programs-", Broadcast Asia 2002
Conference (2002)
5) M. Hayashi: "TVML Technology and Applications", 2002
International Workshop on Broadcasting, Seoul (2002)
6) N. Badler, B. Webber, J. Kalita and J. Esakov: "Animation from
Instructions. In Making them Move", N. Badler, B. Barsky and D.
Zeltzer (Eds.), 51-93, Cambridge, MA: MIT Press (1991)
7) B. Coyne and R. Sproat: "WordsEye: An Automatic Text-to-Scene
Conversion System", SIGGRAPH 2001 Conference Proceedings,
pp.487-496 (2001)8) M. Ma: "Automatic conversion of natural language to 3D
animation", PhD thesis, University of Ulster, Ulster, Ireland (2006)
9) K. Glass, S. Bangay: "A Method for Automatically Creating 3D
Animated Scenes from Annotated Fiction Text", IADISInternational Journal on Computer Science and Information
Systems, 4, 2, pp.103-119 (2012)10) N. Hamaguchi, M. Douke, M. Hayashi: "Internet-Based Private TV
Station Enabling Individuals to Produce, Distribute and View TV
Programs", IADIS2005 (2005)11) H. Kaneko, N. Hamaguchi, M. Doke: "Sign language animation
using TVML", VRCAI '10 Proceedings of the 9th ACM SIGGRAPH
Conference on Virtual-Reality Continuum and its Applications in
Industry, pp.289-29 (2010)12) M. Douke, H. Kaneko, N. Hamaguchi, S. Inoue: "Engaging Viewers
Through the Connected Studio: Virtual Participation in TV
Programs", Consumer Electronics Magazine, IEEE, 1, 4, pp.30-39(2012)
13) H. Yasuda: "Digital Content Creation/Distribution in a
Broadband-Ubiquitous Environment", IEICE Transactions on
Information and Systems, E90-D, 1, pp.76-80 (2007)
14) T2V Home Page: http://t2vlab.jp/en/ (last visited: 2013)

Mamoru Doke received his Masters degreefrom electronics engineering from Nagoya University
in 1992. He joined NHK the same year. Since 1998, he
had been with NHK Science and Technology ResearchLaboratories. He had joined NHK Engineering Services
from 2007 to 2009. He received Ph.D. from Electrocommunication University in 2008. He is engaged inresearch on CG based video content generation andhundling, human interface.
Narichika Hamaguchi had conducted
research and development about content production,
description, distribution, and user behavior analysis ofsocial TV at NHK Science and Technology ResearchLaboratories (STRL) since 2000. He had Professional
Master degree of MOT (Management of Technology)
and Ph.D. in Engineering degree at Tokyo Institute ofTechnology. Currently, he is in charge of R&D planning
at STRL.

Hiroyuki Kaneko joined NHK in 2002. He
has been engaged in research on computer graphics
based content production technology, especially on sign
language animation in NHK Science and Technology
Research Laboratories since 2006. Currently, he is incharge of domestic and international public relations at
STRL.
Steven Bachelder holds a professorship in
artistic expression in convergent media at Uppsala
University, Campus Gotland where he cofounded thegame design education in 2001. Today he is subject
responsible for Expression in Convergent Media andhead of the Convergent Media Lab. Bachelder has beenteaching in the interactive arts and convergent media
since 1995. He is currently conducting research on
engagement in digital games.
Masayuki Nakajima received Dr. Eng.
degree from the Tokyo Institute of Technology, Tokyo,
Japan in 1975 and has been Professor at the
Department of Computer Science, Graduate School ofInformation Science & Engineering, Tokyo Institute ofTechnology during 1997-2012 March. He began
working at the Department of Game Design, Uppsala
University, Campus Gotland, (Sweden) in April 2012.

81