Open Framework
Facilitating Automatic Generation of CG Animationfrom Web Site
Masaki Hayashi† Steven Bachelder† Masayuki Nakajima†‡†Department of Game Design
Uppsala University, Campus Gotland
Visby, Sweden{masaki.hayashi, steven.bachelder}@speldesign.uu.se
Abstract—We have been studying and developing the system
which enables to generate Computer Graphics Animation (CGA)
automatically by processing HTML data of Web site. In thispaper, we propose an open framework to facilitate this. Theframework is functioning all at a server side, obtaining theHTML, converting it to a script describing the CGA story andupdating the script. And at a client side, a user accesses the scripton the server to visualize it by using real-time CG character withsynthesized voice, camera work, superimposing, sound file
playback etc. We have constructed the framework on the serverand deployed the substantial engines to convert Web sites toCGAs. This paper describes the detail of the framework and also
shows some example projects providing automatically generated
News show, Talk show and personal Blog visualization.
Keywords—animation, text visualization, media conversion,
scripting langauge

I. INTRODUCTION
We have been studying the technology called Text-ToVision (T2V) which is able to convert text to Computer
Graphics Animation (CGA) [1]. T2V is developed based on
TV program Making Language (TVML) which has been usedin many ways [2]. A TVML script is interpreted by special
software called TVML engine to generate CGA by using realtime CG character with synthesized voice, CG camera work,
superimposing, displaying image data, playback of movie andsound data, etc. Since the text written in TVML can beconverted to CGA, any type of text source is viable to beturned to CGA by analyzing the original text to obtain TVML
script. Using this methodology, many attempts have been madeto visualize information sources available on the Web [3-4].
However, those attempts are more or less research based anddeveloped as standalone applications or services. We havebeen in lack of shared space on the Internet where any
individual people enable to develop their own automatic CGA
generation experiments and services.
In this time, we have designed and developed an open
framework to facilitate this automatic animation creation frominformation sources on the Web. Our aim is to provide thedevelopment environment to individual people to make theirown services of broadcasting the CGAs. In order to do this, wehave made a service site on a server where the developers canmake their own conversion engines and upload them to the‡Kanagawa Institute of Technology
Kanagawa, Japan
masayuki.nakajima@gmail.com

server instantly so that people can watch the generated CGAs
(i.e. News show, Talk show, Weather report, etc.). We can also
say that this automatic animation generation system on theInternet could become "Auto broadcast station". Our ultimate
goal is to establish the technology of constructing Internet
broadcast station providing CGAs generated automaticallyfrom the Web sources [5].
This paper illustrates the architecture of the framework anddescribes the technical details. Also some actually
implemented auto CGA generation services are introduced
followed by the demonstrations.
II. PRINCIPLE OF AUTOMATIC CGA GENERATION
Figure 1 shows the basic principle of the automatic CGA
generation from Web site. HTML data of a specific Web site is
first obtained, then it is converted to an arbitrary XML script (itis called "ARC script", later explained), then it is converted to a
TVML script and it is ultimately played back by a TVMLengine to produce CGA. There are two conversion engines inthis figure which we named SEC engine (taken from"SEmantic Conversion") and ARC engine (taken from"ARbitrary Conversion") respectively. Both engines work

Figure 1: Automatic CGA generation from Web page

..................
<div class="container"><div class="promo module2 clearfix"> <div class="hero2"><img src="http://ichef.bbci.co.uk/.../reu.jpg"/><span class="hero_title"> Cyclone-hit Vanuatu appeals for help </span><p class="hero_summary"> Vanuatu needs help right now after powerful
Cyclone Pam destroyed all economic progress of recent years, President
Baldwin Lonsdale says. </p> </div>..................

HTML..................
<text>First, top news.</text><pause>2.0</pause><article>Cyclone-hit Vanuatu appeals for help</article><text>Cyclone-hit Vanuatu appeals for help</text><subimage>http://ichef.bbci.co.uk/.../reu.jpg</subimage><text>Vanuatu needs help right now after powerful Cyclone Pam destroyed
all economic progress of recent years,</text><text>President Baldwin Lonsdale says.</text><pause>2.0</pause><endarticle /><endsubimage /><pause>1.0</pause>..................

ARC script (XML)character:openmodel(model=MIA, filename="Characters/mia/mia")
set:openmodel(namee=STUDIO, filename="Sets/studio").................
character: talk(name=A, text="First, top news.")
sound: playfile(filename="Sound\opening.wav")
drawing: settext(name=Lcap, text="<font size='55%'>Cyclone-hit Vanuatu
appeals for help</font>", x=-73%, y=42%, height=10%, width=50%)
drawing:move(name=Lcap, x=-23%, y=42%, speed=0.5s)
drawing:display(name=Lcap, switch=on)character: mocap(name=A, motion=talk01, wait=no)character: talk(name=A, text="Cyclone-hit Vanuatu appeals for help")
prop: openimageplate( name=IMAGEPLATE1,
filename="http://ichef.bbci.co.uk/.../reu.jpg", hsize=1.2, vsize=0.9 )prop: position( name=IMAGEPLATE1, x=0.7, y=1.2, z=0 rx=8, ry=-11)
camera: move(name=Cam, x=0.1, y=0.8, z=1, rx=-9.6, ry=-6.8, rz=0,vangle=56.4)
character: mocap(name=A, motion=sasu01, wait=no)character: talk(name=A, text="Vanuatu needs help right now after powerful
Cyclone Pam destroyed all economic progress of recent years,")
character: talk(name=A, text="President Baldwin Lonsdale says.")prop: visible( name=IMAGEPLATE1, switch=off )camera: move(name=Cam, x=0.1, y=0.8, z=0.8, rx=-9.6, ry=9.8, rz=0.00)
wait( time=1.0 )..................

TVML script

Figure 2: Conversion from HTML to TVML via ACR
script (simplified)
simply processing input text to produce output text. In ourcontext, the SEC converts HTML to XML (ARC script) andthe ARC converts XML (ARC script) to TVML.
Fugure 2 shows an example of this process when it isapplied to automatic news show generation. The SEC engine isdesigned to extract "Title of an article", "URL of an image filein the article" and "Main text of the article" from a given
HTML to output XML data (ARC script) which describes astory made by those three items using XML tags. The ARC
script does not contain visual elements used in the output CGA
such as what character is used, how the camera angles aregoing to be and so on. Those visual elements are maintained by
ARC engine. The ARC engine interprets a story expressed inan ARC script to generate TVML script which describes all thevisual elements used in the CGA.
As explained above, we define SEC and ARC having
different roles. The former one makes a story from the source
Web page and the latter one gives visual expression to the story.
The benefits of this method would be:To enable to design two different engines separately.

Figure 3: Different show formats applied to the same text
To enable a user to choose her or his favorite visualpresentation manner for the same story. (i.e. favorite
announcer, favorite studio set, etc. see Figure 3)

To process the story relatively easy (i.e. to translate toother languages using machine translation, to make adigest version of the original story, etc.)

On the other hand, drawback of the method would be thatthe quality of the output CGA would be affected by the fact ofcutting a way of interaction between the story and the visual.
III. ARCHITECTURE OF THE FRAMEWORKThis chapter describes how we deploy the method actuallyin our framework.
Figure 4 shows the architecture. As explained in theprevious chapter, SEC and ARC are text processing engines.
We deploy those engines on the server as PHP programs(obviously, it does not necessarily have to be PHP, it could bePerl, Python, etc.) where a client side can utilize thosefunctions via network. The SEC engine works as CRON on theserver accessing the target source Web pages periodically toupdate the ARC script so that the ARC script is maintaining thelatest story. And the ARC module is also on the server as PHP
program file which is ready to receive a request from the client
for the conversion from the ARC script to TVML script. Then
ultimately the TVML script is played back by a TVML engine
at the client side and the user can watch the CGA. The PHP
codes of SEC and ARC can be uploaded and registered to theserver by any individual by using a service page on the Web.
IV. THE ENGINESThis chapter describes the detail of three engines: SECengine, ARC engine and TVML engine.
A. SEC engine
As explained before, the SEC engine converts HTML toARC script. Apparently, the design of the engine depends on astructure of a target Web page. If the target HTML is changed,
the engine needs to be updated to be able to analyze it,otherwise it cannot produce an ARC script correctly. But if theHTML is standardized following a rule of, for example,
Semantic Web, the engine should work despite the revision ofthe target page. And if we use RSS data for the analysis, itwould work right owing to the RSS standardization.
Furthermore, if the target page is a personal Blog using
Content Management System (i.e. Wordpress), it is feasible toTable 1: Standard ARC specification (excerpt)

Unity version of TVML SDK (System Development Kit).
TVML engine works combining the input script with various
necessary data such as character and studio CG modeling data,
image, sound and movie data, etc. The engine can refer thosedata both on a local machine and on a server by specifying
URL.

Figure 4: Architecture of the framework

And owing to the Unity multiplatform ability, the engine
can work on various types of platform such as PC, smart phone,game machine and even on a Web browser using "Unity WebPlayer".
V. DEMONSTRATIONS

give the engine consistency. And apparently, if the same
person or organization maintain the target page and the ARC
engine, it would be easy to keep up the simultaneous update forthe both. Our solution for this matter is simple. We will leave itup to the developers by providing them the open framework
where they can maintain their services in a consistent way.
B. ARC engine
The ARC engine converts ARC script to TVML script.
Designing matter of this engine is first to make a specificationof ARC script written in XML. We do not fix the strict
specification for this. Instead, we open it to the developers.
However, we have a basic standard specification of ARC script
and also provide an ARC engine capable of the conversion.
Table 1 shows some excerpt from the standard ARCspecification. The standard ARC engine is made using PHP
code and is on the server and ready to use. The usage of theARC is simple. A client program passes the ARC script text
data by POST request together with a variable indicating thename of the ARC engine, then the data is processed in theserver executing the PHP program and return the result text tothe client.
C. TVML engine
TVML is a special language to describe TV-program-likeCGA. Table 2 shows visual and audio effects available inTVML. It mostly covers the functions necessary for producing
TV-program-like animation. The TVML engine to interpret theTVML has been developed for years and a Windows version
and a Unity version are now available. This time, we use theWe have developed the open framework explained in theprevious chapters and produced substantial contents working
on the framework. In this chapter, we introduce some attempts
that we have done with it.
A) Automatic News Show
We have tried several different News sites such as Reuters,
Google News, BBC, etc. Among those, here we introduce theautomatic News Show using RSS data of "BBC World News".
SEC engine is designed to extract RSS tags and to make an
ARC script. Also we have designed ARC engine to simulate an
existing News show format. Figure 5 shows the output. TheTable 2: TV production versus TVML event type

generation from a personal Blog in our framework. If the Blog
uses a general Content Management System such as Wordpress,
the HTML structure does not change unlike, for example,
News sites driven by the establishments. So it would be easy todesign the SEC engine. Here, we also set up a "hiddencommand" which activates the ARC engine to function as acertain control commands (i.e. to make a character move) but
not appears on the Blog page. The Blog user inserts thecommand in the Blog text as a HTML comment (i.e. "<!-(jump) -->" makes a character jump).
VI. CONCLUSION

Figure 5: Automatic conversion from RSS of BBC NewsSEC updates the ARC script once an hour, and a user canwatch the latest news by accessing it. We have developed aclient viewing software as PC application by using Unity.
B) Automatic Talk Show
We have made a SEC engine converting Japanese popularbulletin board (Ni channeru) to Talk Show. The SEC engine
extracts speech text of each attendant and assigns the text to sixcharacters randomly, then generates the ARC script. Figure 6
shows the output screen shots. The ARC engine here is quite
special. It gives appropriate camera switching and where eachcharacter is looking at and their gesture (nodding)automatically based on an algorithm that we developed. The
algorithm is rule-based one with statistic data obtained byanalyzing a real TV talk show. Owing to the algorithm, theoutput animation is something that is fun to watch like a real
talk show on a TV.

We have proposed and developed an open framework forautomatic generation of CG animation from Web site. The
HTML of a Web site is obtained by a server side program
called "SEC" to produce ARC script in XML format. The ARC
script is then processed by a visualization engine called "ARC"to produce TVML script. The TVML script is ultimately
played back by a TVML engine at a client side. Using thisframework, individual people enable to develop their own
SECs and ARCs by PHP programming and to deploy them on
the server so that people can access to watch the CGAs by
special viewing software for PC. The viewer is developed byUnity game engine with TVML SDK. We have also developed
several automatic CGA generation services on the framework
such as automatic News Show, automatic Talk Show andpersonal Blog visualization. You can actually see the servicesite at http://niz237gt.sakura.ne.jp/budatv/
Our ultimate goal is to establish the automatic TV
broadcast station on the Internet. We have successfully showed
the potential of our concept by providing automatic CGA
through our framework. In this paper, we introduced theoutline of the entire framework, however, there are many
details left in the development of the whole system which
include: improvement of the service quality, security problem,
content rights problem, implementation of a client side
interactive service, and more. We are now challenging those
aspects towards the future.
REFERENCES[1][2]C) Personal Blog
We use Wordpress for constructing an automatic CGA

[3]

[4]

[5]

Figure 6: Automatic conversion from bulletin board

M. Hayashi, et al.: "T2V: New Technology of Converting Text to CG
Animation", ITE Transactions on Media Technology and Applications,Vol.2, No.1, pp.74-82 (2014)
M. Hayashi: "TVML (TV program Making Language) - Automatic TV
Program Generation from Text-based Script -", In Proceedings ofImagina'99, pp.119-133 (1999)
A. Nadamoto: "Complementing your TV-viewing by web content
automatically-transformed into TV-program-type content", In
Proceedings of the 13th annual ACM international conference onMultimedia, ACM. (2005)
K. Tanaka: "Research on Fusion of the Web and TV Broadcasting. In
Informatics Research for Development of Knowledge Society
Infrastructure, 2007. ICKS 2007. Second International Conference on(pp. 129-136). IEEE. (2007)
M. Hayashi, et al.: "Text Generated TV – A New Television System
Delivering Visual Content Created Directly by Text –", In proceedingsof IWAIT2014 (2014)