TextAlive: Integrated Design Environment for Kinetic Typography
Jun Kato
Tomoyasu Nakano
Masataka Goto
National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan{jun.kato, t.nakano. m.goto}@aist.go.jp
ABSTRACTThis paper presents TextAlive, a graphical tool that allows
interactive editing of kinetic typography videos in which
lyrics or transcripts are animated in synchrony with thecorresponding music or speech. While existing systems haveallowed the designer and casual user to create animations,most of them do not take into account synchronization withaudio signals. They allow predefined motions to be applied
to objects and parameters to be tweaked, but it is usually
impossible to extend the predefined set of motion algorithms
within these systems. We therefore propose an integrated
design environment featuring (1) GUIs that designers can use
to create and edit animations synchronized with audio signals,(2) integrated tools that programmers can use to implement
animation algorithms, and (3) a framework for bridging theinterfaces for designers and programmers. A preliminary
user study with designers, programmers, and casual users
demonstrated its capability in authoring various kinetic
typography videos.
Author Keywords

Kinetic typography; animation; creativity support tool;
integrated design environment; live programming.
ACM Classification Keywords

H.5.2. Information interfaces and presentation (e.g., HCI):
User interfaces – GUI; D.2.6. Software Engineering:
Programing Environments – Integrated environments.
INTRODUCTION

Kinetic typography, a technique for moving and reshaping
text in a video, is widely used in music, narrative, andcommercial videos to deliver text information in a dynamic
way. It can deliver the emotional feeling associated with atext in addition to the text’s literal meaning [17]. When it isused in music and narrative videos, text is animated insynchrony with vocal music or speech. The animated text
augments the listening experience with visual effects,
helping listeners better comprehend what is being vocalized.
Kinetic typography videos have been produced with generalpurpose tools for creating animations (e.g., Adobe Flash or

After Effects [1]). While such tools provide a fine granularity
of control over the animations, producing the final result
takes long time even for the experienced user because
manual synchronization of the text animation and audio istedious. In addition, general tools do not provide any
guidelines to help novice users produce convincing effects.
Tools specifically designed for making kinetic typography
videos (e.g., ActiveText [18] or Kinedit [7]) provide
predefined templates of effective animations, from which theuser can choose. Such domain-specific tools, though, have
had a certain limitation in their flexibility. That is, theprogramming needed to create new animation templates has
to be done in separate development environments.
With the goal of pushing forward the state of the art in kinetic
typography design, we present TextAlive, a graphical toolthat allows interactive editing of kinetic typography videos(Figure 1). While tools for designers and programmers haveusually been designed separately, TextAlive is designed tobenefit both. We therefore call it an integrated design
environment (an integrated development environment is one
designed only for the programmer). Thanks to its duality forthe designer and programmer, its research contribution isthree-fold. First, from an input pair consisting of audio andtext information, TextAlive can automatically compose aplayable kinetic typography video that serves as a concrete
starting point for the designer. It also provides interactive
user interfaces for editing the video in synchrony with theaudio signals. Second, TextAlive allows live programmingof animation templates with its built-in development
environment that provides continuous graphical feedback to
the programmer. Third, its unique framework allows the
programmer to easily extend its GUI so that the programmer
or designer can intuitively debug or customize theparameters of animation templates.
The remainder of this paper is organized as follows. Related
work is introduced to highlight our research contribution.
Next, the design and implementation of TextAlive’s

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights forcomponents of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, topost on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from Permissions@acm.org.
CHI 2015, April 18 - 23 2015, Seoul, Republic of Korea
Copyright is held by the owner/author(s). Publication rights licensed to
ACM.
ACM 978-1-4503-3145-6/15/04…$15.00
http://dx.doi.org/10.1145/2702123.2702140

Figure 1. Overview of the TextAlive system.

interaction for the designer and programmer are introduced.
Then, the results of a preliminary user study are discussed.
Finally, the conclusion and future work are presented. For
better understanding, readers are encouraged to see thedemonstration video and/or visit http://textalive.jp to try out
the system before further reading.

created animations. Instead of using visual programming,
TextAlive integrates text-based programming into agraphical tool. It provides more precise and flexible control
over the text animation and an abstraction of an animation
template that is reusable not only by the programmer but also
by designers and casual users.

RELATED WORK

Tools for Creating Animations

Tools for Creating Kinetic Typography

Various tools for creating kinetic typography videos havebeen proposed. However, they focus on graphical aspects ofcreating videos. To our knowledge, none of them deals withaudio signals synchronized with the videos. This is because
their primary use of kinetic typography is for augmenting
text-based communication. TextAlive, in contrast, aims toaugment audio listening experience with dynamic graphicalrepresentation of text information. Prior work on editing
time-series multimedia includes synchronized editing oftranscription and narration [21]. This is a complementary
technique and can be integrated into TextAlive.
The existing tools for kinetic typography videos target
programmers, designers, or casual users, not all three.
TextAlive is built on top of these. For instance, a framework
to help programmers create kinetic typography videos by
writing Java source code has been proposed [16]. TextAlive
shares some concepts with this framework, such as thehierarchical structures of phrases, words, and characters forrepresenting text information. The ActiveText [18] andKinedit [7] systems allow designers and casual users to input
text, choose animations from several templates, and edit theirparameters by using sliders and other standard GUIs.
TextAlive also provides similar features. A combination ofnatural language processing (NLP) and heuristics enables theextraction of emotional information from text and automatic
generation of the associated videos [20]. While the existing
techniques benefit from NLP, they do not allow userinteraction, which is essential if the user is to express his/hercreativity.
The existing tools have limitations because of their targetingcertain users. While tools for designers and casual users
provide interactive GUIs, they typically force the user to use
a predefined set of motion algorithms and do not allow any
extensions to be implemented. The tools for programmers,on the other hand, have extendibility based on their APIs, but
their development environments are usually equipped with
text-based user interfaces and do not provide interactive
feedback on the source code changes.
There is a visual programming environment that overlays
code elements on top of the animation and allows binding a
set of elements to text units [3]. Each element represents one
typography property (font size, color, style, or scaling), andthe set of elements affects the properties of nearby text in a
way that depends on the distance between a set element anda text unit. It does not allow precise control of properties, nordoes it provide any abstraction, preventing reuse of the

While this work focuses on text animation, it is closely
related to prior work on tools for general animation. Adobe
Flash and After Effects [1] are professional tools used tocreate various kinds of animation. The user specifies key
frames for objects, and the system smoothly interpolates the
motion between the frames. These tools have good support
for direct manipulation, such as visualizing object
trajectories and using drag-and-drop to change the motion or
seek time periods. Compared with these professional tools,
TextAlive has a limited, or rather focused, GUI for editing
text animations synchronized with audio signals, including atimeline interface. The user does not specify key frames, butrather timings of each text unit being vocalized. The timing
information is then passed to animation templates that output
fluid motions. TypeMonkey [2] is a plug-in for After Effects
that has additional user interfaces for choosing text
animations from templates and editing their parameters forthe purpose of making kinetic typography videos. While it
reduces the pain of key-framing, it still requires manualsynchronization of the animation and audio. In addition, it
has the same issue as other tools for kinetic typography: thelimited number of templates restricts the user’s expressivity.
Tools that let casual users make animations, such as
Microsoft PowerPoint and Movie Maker, do not require keyframing. However, the resulting motions are usually limited
to those following simple trajectories. In particular, various
sketching interfaces allowing the casual user to create
animations have been proposed. K-Sketch [4] allows users tospecify the motion of objects by drawing trajectories, which
is implemented as our “Trace Path” animation. Draco [15] isa sketch-based tool to animate multiple objects. We also
target casual users who make animations of multiple objects(text units) but with more finely grained control for eachobject. In addition, TextAlive integrates a programming
environment that allows authoring of reusable animation
templates.
Tools for Live Programming

Live programming [19] is a technique to provide theprogrammer with continuous feedback about the program
being developed. It aims to get rid of the gulf of execution
by providing a seamless programming experience withregard to writing code, executing the program, anddebugging it.
Live programming in TextAlive is inspired by Victor’s demo
[23] that allows the programmer to dynamically rewrite
parameters of graphical applications during their execution.
In our system, the programmer can simply click the “update”
button, and there is no need to re-launch the entire programand reconfigure the parameters. It allows the programmer torewrite the logic of animation templates without losing theirparameter information. VisionSketch [13] uses a similar
technique, where the logic of image processing algorithms
can be dynamically replaced, but it does not preserve thestate information when the logic is updated. Subtext [6]continuously revalidates the program without explicit
compilation. It updates the program output upon the typingof each character in the source code editor. We intentionally
avoided such implicit revalidation and kept the “update”
button because the implicit revalidation flushes the screen
very often and is distracting.
Professional tools for animation are typically equipped withscripting engines. Adobe Flash and Adobe After Effects, forexample, have a built-in scripting engine to control theproperties of objects. Some tools meant for casual users, suchas Microsoft PowerPoint, also allow such scripting. Whilemost scripting engines allow conversational programming,these tools do not have decent support for bridging theactivities of writing code and running the program. On theother hand, Juxtapose [12] is capable of generating a slider

bound to a variable value, and Unity allows the programmer
to extend its GUI for runtime parameter tuning. Within theseenvironments, GUI widgets bridge that gap between thestatic definition and the runtime behavior of the program.
TextAlive takes the same approach, balancing the variety ofwidgets (more than Juxtapose) and the lines of code (lessthan Unity). Moreover, TextAlive is specialized foranimation authoring. For instance, a set of APIs is provided
for easily animating text units. The Stage interface canvisualize object trajectories calculated from the user code.
Whereas the prior work intended for game development hassuch visualizations [23, 9], TextAlive uses the trajectories
not only for visualization but also for seeking time periods.
TEXTALIVE: AN INTEGRATED DESIGN ENVIRONMENTThis section introduces the interaction design of TextAlive.
The first subsection describes interactive editing of kinetic
typography videos by designers or casual users. This sort ofediting does not require prior knowledge of kinetic
typography or programming. The second subsection
describes live programming of animation templates by
novice programmers. These days, it is not uncommon for thedesigner to have programming skills. For instance,
programming skills in scripting are required to precisely
control many objects when using existing tools such as
Adobe After Effects [1]. In TextAlive, programming abilities
enable designers and casual users as well as programmers tocontrol not only the currently present objects on the screen
but also their later reuse and customization.
Interactive Editing of Kinetic Typography Videos

Figure 2. Timeline interface.

Figure 3. Editor interface and integrated source code editor; theuser can switch between them by clicking the buttons (f).

TextAlive can automatically create a kinetic typography
video when given an audio file and its transcription. Whereas
the existing tools require the user to create a video fromscratch and to spend much time on manual synchronizationof audio and text, TextAlive significantly reduces the effort
required. The composed video can be interactively edited sothat it matches the user’s style.
Automatic Video Composition and Timing Correction

The user first inputs a pair consisting of a song and its lyrics
or a narration and its transcription. Then, the systemestimates the timings of each character in the text being
vocalized [8]. Through this process, not only characters but
also words that consist of characters and phrases that consistof words and correspond to one line in the original text cometo have timing information about the starting and ending of
the vocalization. The system then assigns a default animation
template to each text unit (a character, word, or phrase) andsynthesizes a playable video. If a song is provided, its chorus
parts are detected and assigned a more gorgeous animation
template (explained in the implementation section). Beat
information is also analyzed, and the result is visualized on
the Timeline (Figure 2). After this automatic synthesis, thevideo is shown on the Stage interface (Figure 1), and it will
always be playable by clicking the “play” button in theTimeline interface. Any operation described hereafter can be
carried out interactively so that the video can be instantly
updated.

TextAlive cannot always estimate correct parameters. If
there is an error in the estimation, the timing information canbe interactively corrected in many ways before being saved
for use the next time the same pair of audio and text is loaded.
For instance, text units in the Timeline can be drag-anddropped to instantly fix their timing information. Drawing a
rectangle in the Timeline selects multiple text units thatoverlap the rectangle. Chorus parts are highlighted in thebottom row of the Timeline, and clicking them selects the
text units vocalized during the chorus. Once the text units are
selected, the “align left,” “center,” “right,” and “justify”
buttons in the Editor interface (Figure 3b) can be used to fix
their timings collectively. Timing information can also be
copied and pasted (Figure 3c), and this is quite useful forgroups of text units with the same structure, such as repeating
chorus parts.
The seek bar in the Timeline visualizes the sequentialchanges in vocal volume information extracted from theoriginal audio signals, thereby helping the user to estimate
correct timings visually. The seek bar allows one to select a
specific time region. While a region is selected, the “play”
button plays the corresponding part repeatedly, allowing one
to concentrate on correcting the timing and editing theanimation of the specific part. Automatic synchronization
estimating the timings of the text units contained in theselected region can be applied again by clicking “auto-sync”
in the Editor interface (Figure 3a).
Font and Animation Template Assignment

An automatically created video does not at all reflect theuser’s intention. To make it his/her own, the user selects text
units in the Timeline and changes their properties, including
fonts, and animation templates with the Editor interface(Figure 3d, e). As in timing correction, property changes canbe collectively made on multiple text units. For instance,
when a phrase is selected, changes in the font style and size
are applied to all of the characters in the phrase.
Predefined animation templates allow novice users to create
aesthetic kinetic typography. To avoid implausible visual
effects, they are not always applicable to every type of text
unit: phrases, words, and characters have different sets ofassignable animation templates. In addition, multiple
templates can be assigned to a single text unit at the sametime. For example, “Sliding Animation” can be assigned to aphrase, and “Hopping Animation” can be assigned to thecharacters in that phrase. “Karaoke Animation” that changes

Figure 4. A simple example of kinetic typography effects
generated from different sets of preset animation templates.

the text color before/after vocalization can also be assigned
to the characters (Figure 4). Combinations of different
templates that can be assigned to suitable types of text units
allow flexible yet effective kinetic typography design.
Animation Customization

Each animation template has parameters for customizing its
appearance, and these parameters can be interactively
manipulated with the GUI widgets shown on the Editor(Figure 3g) or the Stage. These widgets are dynamicallygenerated by the template implementation, which is
described in detail in the next subsection. They appear when
the corresponding template is selected in the Editor interface.
While the Stage can usually show only the current frame ofinterest in the video, it would be desirable that the user can
instantly see how an object’s motion is affected by parametertuning. To make this possible, the trajectories of the selected
text units can be visualized on the Stage (Figure 1). The
trajectories can be used to seek time periods in the way
shown in prior work [5].
Live Programming of Animation Templates

While each animation template can be customized to meetthe user’s needs, it still limits the resulting animation to acertain extent. General tools for creating animations are often
equipped with scripting engines with which the animation oflots of objects can be controlled in a mathematically precise
way. Such scripting, though, typically does not allow laterreuse because it lacks an abstraction mechanism. There is notmuch graphical feedback while the user is writing code either.
TextAlive aims to address this issue by enabling “liveprogramming” of animation templates: the user canseamlessly go back and forth between editing animation andimplementing templates (Figure 3f). The implementation canbe updated by just one click and there is no notion ofcompilation or execution. The program that creates theanimations continues running virtually. The update process
updates not only the behavior of the text units but also the
GUI widgets specific to the template (Figure 3g). Thesewidgets help the programmer debug the template and help
the user interactively customize it.
Implementation of Animation Templates

To read the implementation of an existing animation
template, the user selects the template and clicks the “edit”
button in the Editor interface. TextAlive then shows the code
editor, with which the source code written in Java can beedited (Figure 3h). Each template is defined as a Java classthat implements a specific Java interface. The template
implementation can be updated by clicking the “update”
button, which instantly updates animations of the text units
that use the template. This update can be done at any moment,such as during the playing of a song or when the user isobserving the motion of text units. To create a new template,the user can either click the “new” button or change the classname in the source code of any existing template and click
the “update” button.

Updating running software is generally called a hot swap. It
replaces existing objects using old code with dynamicallygenerated new objects using new code. Since such a processsimply discards the state information of the old objects, thestate information (in our case, customized parameters of ananimation template) usually needs special care, such as by
the programmer writing some code to store and restore it.
TextAlive handles this transparently, which frees the
programmer from having to write boilerplate code andallows him/her to concentrate on the algorithm.
Implementation of GUI Widgets for Each Template

As discussed in Animation Customization, each template has
dedicated GUI widgets to customize its appearance. “Sliding
Animation,” for instance, has a box of radio buttons to selectits horizontal alignment and a slider to control its sliding
angle. In order to implement a GUI widget, the programmer
should declare a Java public field and write a comment blockright before the field declaration in the source code. The
comment should be written in a simple format: “@uiWidgetName([options]).” The generated GUI widget isbound to the public field; GUI operations on the widgets
change the field values.
The widgets are not only meant to help designers and casual
users manipulate the parameters intuitively. They can also
help the programmer debug and test the template during thedevelopment process. For instance, the programmer cantemporarily create a box of radio buttons for switching
between multiple implementations having a similar effectand check which one works best. Once he/she knows which
one is best, he/she removes the radio buttons and publishes
it for other users.
Below is the list of preset GUI widgets available in thecurrent implementation of TextAlive (Figure 6). Other
widgets can be easily added by writing Java source code.

@ui Check() shows a check box to switch between true andfalse values of the corresponding boolean field.
@ui Radio(“label1”, v1, “label2”, v2, …) shows a box ofradio buttons used to select a value.
@ui Color() shows a button with a colored rectangle so thatthe user can choose a color with the color picker dialog.
@ui File() shows a button used to choose a file whose pathis stored in the corresponding String field.
@ui Track() shows a check box used to enable/disabledrawing a path by dragging the mouse on the Stage. When apath is drawn, its data is represented as a list of (x, y)
coordinates and stored in the corresponding List field. Thisis used to implement the “Follow Path” animation templatethat makes text move along the specified path.
IMPLEMENTATIONThis section introduces the current implementation of the
TextAlive system. It first gives an overview of the systemalong with the techniques used in the implementation, andthen it explains the system in more detail. The first
subsection explains how the creation of plausible kinetic
typography videos is supported, and the second explains how
a live programming experience is provided.
Overview

The architecture of TextAlive is shown in Figure 7. It iscurrently implemented with Java 7 (64-bit) and tested on a
personal computer with a Windows 8.1 (64-bit) OS. Input
sound files should use the MPEG-2 Audio Layer III format
for playback in the system and are converted to the monaural
MS WAV format using FFmpeg for further audio signal
processing. Input text is written in a natural language (either
Japanese or English) and is converted into a series ofphoneme information by using MeCab, open-source
software for morphological analysis and part-of-speechtagging.

@ui Slider(min, max) shows a slider to change the value ofthe corresponding integer field between min to max.

Figure 5. Comments and GUI widgets generated from them.

Figure 6. Software architecture of the TextAlive system.

The audio signal is synchronized with the corresponding
phoneme information [8] by attaching timing information to
the phrases, words, and characters of the original text. If the
input sound file is a song, the beat information and the chorus
part are detected [10]. This information is referenced forautomatic video composition. The Java source code ofanimation templates is dynamically compiled with the
Eclipse Compiler for Java and loaded into the Java VM with
the Java Reflection API. In order to store and restore the state
information of the animation templates, a text-based JSON
format is used as a temporary backup. The animation data isalso saved as a JSON file (Figure 10).
Implementation for Kinetic Typography Design
Automatic Video Composition

function – in other words, it should return a value without
modifying the state of the animation template object.
While changing the rendering parameters can reshape theappearance of the text unit as a vector graphic, there are
kinetic typography effects that cannot be supported by such
reshaping and instead demand pixel-based operations. They
include adding blur or shadows to the text graphics. To
support such effects, an animation template can implement
background(graphic context, time) and foreground(graphic
context, time) methods that render graphics behind and in
front of the text units. Since these methods are called after
rendering all text units in the off-screen buffer by using theanimate(time) method, they have access to the pixel-based
representation of the text units.

Given the automatically estimated parameters, TextAlive
automatically assigns animation templates to text units andgenerates a playable kinetic typography video. By default, it
simply assigns the “Sliding Animation” to all phrases and the“Karaoke Animation” to all characters. If the input sound isa song, phrases in the chorus part are assigned the “Sliding
and Rotating Animation” and characters in the chorus partare additionally assigned “Hopping Animation” so that thechorus part looks more prominent than the other parts.
The strategy that defines how to assign animation templates
is currently written as Java source code and can be easily
customized. We plan to implement a GUI for choosing andimplementing new strategies.
Text Animation

In a lot of kinetic typography videos, phrases, words, andcharacters have their own motion algorithms independent
from the others. In order to animate text information in ahuman-readable way, characters often keep their positions
relative to each other and are aligned horizontally or
vertically. Multiple words are often organized into a phrasethat is prevented from overlapping with other phrases. Such
motion algorithms can be described in a simple and clean
way by using the coordinate system relative to the parent text
units (e.g., characters to words and words to phrases).
Given the characteristics of text animation, each text unit
holds its timing information (when it starts and ends being
vocalized), a list of assigned animation templates, a font
name, and one or more characters contained in the unit. Eachanimation template is required to implement a method named
animate(time) whose argument is the current time. An
example implementation is shown in Figure 8. The template
has access to its assigned text unit, and the animate method
modifies the rendering parameters of the text unit. The
parameters include font color, an alpha compositing rule
(how to blend the text graphic with the existing pixel values),
and an affine transform matrix (a three-by-three matrix thatdefines how to transform the text graphic and includes
translation, rotation, and scaling) relative to the parent text
unit. Since the animate method can be called with any timevalue at any time, the method is usually expected to be a pure

Figure 7. Example implementation of an animation template.

Figure 8. Pseudocode for rendering each frame.

TextAlive plays and renders the video in a 30-frames-persecond format. The pseudocode for rendering each frame isshown in Figure 9. Graphic units are optional units that holdthe start and end times as well as a list of animation templates
mainly used for rendering the background or foreground tocomplement or emphasize text information.
Trajectory Rendering

Trajectory rendering allows the user to see the past and future
motion at a glance. The same procedure as used for rendering
frames is repeatedly called without actually using renderinggraphics, and the absolute time-series positions of theselected text units are visualized as their trajectories.
The trajectory can also be used as an interface to seek time
periods. Since the system knows the correspondence
between a specific point in the trajectory and timing, thecurrent mouse position can easily be converted into timing
information by finding the closest point in the trajectory.
When the trajectory is not straight-forward, though,following the closest point sometimes results in a big jumpin time, which is not desired. To avoid this problem, thedistances between the mouse position and the next orprevious point in the trajectory are calculated.
Implementation for Live Programming ExperienceHot-swap for Updating Animation Templates

Animation templates are represented by Java classes that arecompiled and dynamically loaded into the Java VM when
TextAlive launches. After the user edits their source code,
the corresponding classes are recompiled and the resulting
Java bytecode is reloaded into the VM.
Such a process, i.e., replacing classes without restarting theVM, is called a hot-swap. A general hot-swap simply
replaces an old class definition with a new one and is not
sufficient for updating animation templates. If the updatingprocess only updated the class definition, existing objects
with old definitions would not be updated. To see theupdated motion, the user would need to remove the objects,
re-assign the animation template, and reconfigure parameters.
To avoid such hassle, TextAlive automatically extracts state
information from old objects, creates new objects with theupdated implementation, and makes its best effort to restore
the state information. Before a hot-swap, a backup of thestate information of the existing objects is created by calling
the Java Reflection API to serialize all public fields of eachobject. It is used after the hot-swap to construct objects withthe updated class definition, in effect replacing the old
objects. The process of restoring state information is
identical to that of loading videos, which is described later.

Each comment that begins with @ui and is located rightbefore a public field declaration is parsed and divided into
the widget class name and its options. The option values areevaluated using a subset of the Java programming language,
thereby providing dynamic parameters to widgets. The
language supports simple mathematical calculations andread-only access to useful parameters such as the resolutionof the video (the size of the Stage). For instance, a slider
with the maximum value of the height of the Stage can begenerated by writing “@ui Slider(0, stageHeight)” as shownon line 17 of Figure 8.
Saving and Loading Videos

TextAlive saves a kinetic typography video in plain text
JSON format. It consists of a hierarchical architecture ofphrases, words, and characters. Each text unit has the
vocalization start and end times and a list of information
about the assigned animation templates. Unlike phrases andwords, characters are also saved with their font name and size.
Each animation template is represented by its name and thecustomized parameters. Figure 10 shows an example of a
JSON object of a video.
Loading videos is exactly the reverse of saving them, except
for the chance of animation templates being updated. If an
animation template has an updated definition, one or more
public fields might be added or removed. To construct ananimation template object with an updated definition,
TextAlive makes its best effort to maintain the state
information; newly added fields have their default values,
and old values of removed fields are simply discarded.
PRELIMINARY USER STUDY

We conducted a preliminary study to gain user feedback on
the TextAlive system and to investigate the capabilities,
limitations, and potential of our interaction design. Seven
users with varying levels of expertise in authoring videos andprogramming graphical applications participated in the study.
They were asked to use the system and to create a kinetic
typography video with one favorite song or narration. They
were asked to compare their use of the system against theirprior experience on tools for creating videos and graphicalapplications.
Participants

Seven participants aged 23 to 30 years old (mean 27) took
part in the study. Each was paid $30 for their participation.
P1-4 are designers and casual users (two females and two
males). P1 is an amateur singer who has used Adobe After

Static Code Analysis for Generating GUI Widgets

The GUI widgets for customizing animation templates are
generated by static code analysis upon compilation of theclass definition. The widgets can collectively change thevalues of multiple variables (i.e., the same parameters ofmultiple animation template objects) when multiple text
units are selected in the Timeline.

Figure 9. JSON-formatted text for saving the animation.

Effects [1] to create a simple slide show for her music video;
P2 is an amateur illustrator who has no prior experience increating videos; P3 is an amateur disk jockey who has
extensive experience in mixing music and recording
narrations for Rakugo (a traditional form of Japanesestorytelling) but no video authoring experience; P4 is aninteraction designer who has used Motion, Final Cut Pro,
openFrameworks, and Unity [22] for creating videos.
P5-7 are programmers and are all male; P5 has a small
amount of experience creating demonstration videos for his
software; P6 is an experienced user of Adobe After Effects(> 2 years) and has created music videos; P7 has a hobby ofwriting songs and has created music videos with AviUtl,
which is freeware for casual video authoring.
Experimental setup

All participants used a standard set of laptop computers witha full HD (1920 x 1080 pixels) display and a mouse (Figure11). Some of the participants brought their headphones or
earphones; others used earphones we provided. The
TextAlive system occupied the entire screen during the study.
The experiment followed the steps described below and took2-3 hours for each participant.
Pre-experiment questionnaire: Each participant was asked tofill out a form asking his/her experience in authoring songs,
narrations, videos, and programming.

Introduction and demonstration: Each participant was given
a 5-15 minute introduction to the TextAlive system by the
instructor. The instructor then demonstrated the timing
correction, font and animation template assignment, andanimation customization using the GUI widgets. For the
programmers (P5-7), the instructor also demonstrated the useof the integrated source code editor to modify and create an
animation template.
Kinetic typography design: Each participant was given 2hours maximum to create a kinetic typography video of
his/her favorite subject with the system. The participant wasallowed to bring data of a song, narration, or image to be usedin the video or could choose from a list of the songs we
prepared. Since it is usually impossible to create a video foran entire song or narration in 2 hours, the participant was
asked to work on his/her favorite part of the song or narration.
During this open-ended task, the participant was free to ask
any questions about the usage of the system, including how

to use the preset animation templates and APIs for
implementing animation templates. The participant wasallowed to end the study when he/she was satisfied playing
with the system.
Post-experiment questionnaire: Each participant was asked
to fill out a form asking five questions about the usability ofthe system (selected answers on a 7-point Likert scale), andthree questions about the good and bad points of the userinterface.
Results and Lessons Learned

All participants could successfully create kinetic typography
videos for their favorite part of a song or narration, some ofwhose screenshots are shown in Figure 12. Two participants
used a song we prepared, while the others used their favorite
song or narration. The results of the post-experimentquestionnaire consisting of the mean, standard deviation, andpercentage of positive responses (>4 on a 7-point Likert
scale) are shown in Table 1.
Everyone who participated in the study was positive about
how suitable TextAlive is for authoring kinetic typography
videos (Q5). There were no answers representing negative
impressions of the TextAlive system on Q1-3, but there wereseveral neutral answers (= 4 points). The neutral answers
were found to come from non-critical reasons such as theparticipant not being enthusiastic enough in creating kinetic
typography videos (P1) and practical reasons that can beaddressed by making minor revisions to the system such asadding keyboard shortcuts to frequently used commands(P7). Those who used the live programming feature (P5 and
P7) indicated they felt the need for technical support in order
to use the system. Their negative answers stem from theprogramming experience described later.
Automatic video composition was useful for both novice andexperienced users: All participants liked the automatic video

composition feature a lot. P7 commented “automaticallyestimating timing information and synthesizing a playable
video is extremely useful, reducing the effort always neededfor existing tools I have used.” P3 commented “it is nice that
we no longer need to create but just edit the video.”

The abstraction of phrases, words, and characters was
welcomed: Participants with prior experience of creating

videos with text information such as subtitles (P1, P4, P6)
especially favored the hierarchical abstraction, enabling
assignment of different animation templates to each phrase,
word, and character. P1 noted that “the capability ofmodifying the properties of each character is very useful.”This feature is often missing from existing video authoring
tools.

The abstraction of animation templates suggested potential
applications: While animating text in a specific time period

Figure 10. Setup of the user study. (a) A participant using oursystem. (b) A screenshot taken during the study.

requires manual parameter tuning in existing authoring tools,
animation templates in TextAlive adapt the animation to fit
in the period. This allows the user to almost instantly animate
text in synchrony with sound. P3 and P4 expressed theirdesire to play “text jockey” during their live disc jockey
performances. P3 also foresees using it for Rakugo
storytelling.
No need to stop the video, enabling a fluid experience: Many
users appreciated that almost every operation on the systeminteractively updates the resulting video. P5 commented that“there was no need to pause or stop the video. With TextAlive,
I could select the time region of interest and play the video
continuously while editing it. The only exception was when I
got too tired to listen to the same part of the song.” P6
appreciated that “animation templates can be edited in-place(without launching external editors.)”
Having a variety of animation templates counts: P6
appreciated that the system allows novice users to create
good looking videos simply by choosing animations from alist of preset templates. P1 and P7 requested more variety of
templates. P4 wrote that “more variety of animation
templates would make TextAlive more appealing, but too
many templates might overwhelm the user.” Actually, P3 andP7 complained that the current combo box for choosing an
animation template is not informative enough. They
commented that more detailed explanations or graphicalpreviews would be desirable. In addition, P6 pointed out thedifficulty of predicting the result of combining multiple
animation templates, suggesting that we provide preset
combinations of animation templates.
The Timeline interface is at the heart of synchronizing audio
and video: All users appreciated the usability of the Timeline

interface dedicated to timing correction. P1, P2 and P5
especially appreciated the wave-form visualization of vocal
sound and the visualization of beat information. In addition,
P1 and P5 requested snapping to the beat information whilemoving or resizing text units in the Timeline.
# Question
1 I would like to use it frequently.
2 I found it unnecessarily complex.
3 I thought it was easy to use.
4 I needed technical support to use it.
5 I thought it is suitable for authoring kinetic
typography videos.

The TextAlive system
Mean
SD
%
5.57
1.20
5/7
2.71
1.02
0/7
4.86
0.75
4/7
3.29
1.74
2/7
5.57

0.75

7/7

Table 1. Results of the post-experiment questionnaire.

Figure 11. Screenshots of kinetic typography videos in various
styles created by the participants (selected and trimmed.)

The Stage interface should allow (more) direct manipulation:

P3 appreciated that most of the interfaces for customizing
appearance of the selected units are gathered in the Editor
interface. On the other hand, P2 commented that most of thepreset animation templates only use the widgets in the Editor
interface and the animation templates should make more useof the Stage interface for their customization. Currently, the“Follow Path” animation is the only preset template thatallows the user to interact with the Stage to interactively
update the tracing path. P1, P3 and P4 requested that simple
animations should be customized by direct manipulation on
the Stage, e.g., allowing drag-and-drop of text units to fix
their positions at a certain timing. However, it is not straightforward to support direct manipulation of objects whose
motion is defined by a combination of multiple algorithms.
Since each animation template transforms the coordinatesand does not know how the other templates are implemented,
the parameters for the template to move the assigned unit to
the mouse cursor position cannot be easily retrieved. The
process is identical to finding a set of parameters for an
unknown function that returns a specific value. Similar
problems in physical simulations have been addressed byintroducing novel interaction techniques that might beapplicable to our case [11]. More improvements to thesketching interaction might also be helpful, as has been
indicated in prior work [15].
Programming ensures extendibility but needs training: Users
who tried the programming interface (P5-7) all appreciatedthe extendibility of TextAlive, but at the same time, found it
difficult to learn in 2-3 hours. P5 hesitated to use the feature.
P7 commented that “the live programming feature allows
fine adjustments of animation but requires prior knowledge,
making me procrastinate in its use for some time.” However,
he was the most active person to create animation templates
by the end of the experiment and also commented that “thechanges in code could be instantly applied to the video,
which is nice.” P6 appreciated how easy it is to provide GUI
widgets for tweaking parameters. Given that the live
programming feature itself is not considered harmful, more
software engineering effort to support more “learnable
programming” [24] is desired.
CONCLUSION

We have described TextAlive, a tool for creating kinetic
typography videos. It can automatically estimate the timingof each character being vocalized. Instead of providing a
blank canvas and forcing the user to create videos fromscratch, TextAlive analyzes a given pair of audio and text
information and creates a playable and editable video. The
video serves as a foundation on which the user can create
his/her own video. The user interface for interactive editingof the video synchronized with audio signals was highly
appreciated by the participants of the user study who hadvarious professional backgrounds. The capability of editing
animation templates by writing programs ensures theextendibility of the TextAlive system as a tool that cancontinue to evolve through use. While the user study

confirmed its usefulness, its learning curve should be gentler,
and more software engineering help provided.
Although the number of websites for sharing videos andaudio files is increasing and more people are getting involvedin creating new media content, it is still not easy to create
videos synchronized with audio. Music videos on suchwebsites typically feature the artists and illustrations. The
lyrics are often placed in the safe area near the edges of thevideo or are not rendered on the video at all. Kinetic
typography is effective at expressing the message of a songor narration visually. The tool described in this paper
enhances the expressiveness of casual users by enablingthem to create kinetic typography videos. While its current
implementation is limited to placing objects on a twodimensional plane, we are working on an extension to threedimensional space.
TextAlive is available online at http://textalive.jp. Currently,
many songs and narrations and their lyrics and transcriptions
are available on the Internet. Preprocessing such information
allows the user to create kinetic typography videos without
first wading through the synchronization process. Corrected
timing information can be shared with others, enabling
crowdsourcing of such timing information. Animation
templates can also be easily shared with others, meaning thatthe integrated design environment can be extended
cooperatively. While this study focused on the experience ofusers with different backgrounds, the proposed interaction
design enables various forms of collaboration (e.g.
programmer with designer, designer with end user, etc.).
Investigating such interactions would make an interesting
follow up.
ACKNOWLEDGMENTS

This work was supported in part by JST, CREST.
REFERENCES

1. Adobe After Effects.
http://www.adobe.com/products/aftereffects.html.
2. aescripts + aeplugins. TypeMonkey.
http://aescripts.com/typemonkey/.
3. Chao, C. M. and Maeda, J. Concrete programming
paradigm for kinetic typography. In Proc. IEEE VL
1997, 446-447.
4. Davis, R. C., Colwell, B. and Landay, J. A. K-Sketch: a“kinetic” sketch pad for novice animators. In Proc. CHI
2008, 413-422.
5. Dragicevic, P., Ramos, G., Bibliowitcz, J.,
Nowrouzezahrai, D., Balakrishnan, R. and Singh, K.
Video browsing by direct manipulation. In Proc. CHI
2008, p.237-246.
6. Edwards, J. Subtext: uncovering the simplicity of
programming. In Proc. OOPSLA 2006, p.505-518.
7. Forlizzi, J., Lee, J. and Hudson, S. The Kinedit system:
affective messages using dynamic texts. In Proc. CHI
2003, 377-384.

8. Fujihara, H., Goto, M., Ogata, J., and Okuno, H. Lyric
Synchronizer: automatic synchronization system
between musical audio signals and lyrics. IEEE Journalof Selected Topics in Signal Processing, 5(6), 2011.
9. Fukahori, K., Sakamoto, D., Kato, J. and Igarashi, T.
CapStudio: an interactive screencast for visualapplication development. Ext. Abstracts CHI 2014,1453-1458.
10. Goto, M, Yoshii, K., Fujihara, H., Mauch, M. andNakano, T. Songle: A web service for active music
listening improved by user contributions, In Proc.
ISMIR 2011, 311-316.
11. Ha, S., McCann, J., Liu, C. K. and Popovic, J. Physics
storyboards. Computer Graphics Forum 32, 2 (Proc. EG
2013), 133-142.
12. Hartmann, B., Yu, L., Allison, A., Yang, Y. andKlemmer, S. Design as exploration: creating interface
alternatives through parallel authoring and runtime
tuning. In Proc. UIST 2008, 91-100.
13. Kato, J., Sakamoto, D. and Igarashi, T. VisionSketch:
Integrated support for example-centric programming ofimage processing applications. In Proc. GI 2014, 115122.
14. Kazi, R., Chevalier, F., Grossman, T. and Fitzmaurice,
G. Kitty: Sketching dynamic and interactive
illustrations. In Proc. UIST 2014, 395-405.
15. Kazi, R., Chevalier, F., Grossman, T., Zhao, S. and
Fitzmaurice, G. Draco: bringing life to illustrations withkinetic textures. In Proc. CHI 2014, 351-360.
16. Lee, J., Forlizzi, J. and Hudson, S. The kinetic
typography engine: an extensible system for animating
expressive text. In Proc. UIST 2002, 81-90.
17. Lee, J., Jun, S., Forlizzi, J. and Hudson, S. Using kinetic
typography to convey emotion in text-basesd
interpersonal communication. In Proc. DIS 2006, 41-49.
18. Lewis, J. and Weyers, A. ActiveText: a method for
creating dynamic and interactive texts. In Proc. UIST
1999, 131-140.
19. McDirmid, S. Living it up with a live programming
language. In Proc. OOPSLA 2007, 623-638.
20. Minakuchi, M. and Tanaka, K. Automatic kinetic
typography composer. In Proc. ACE 2005, 221-224.
21. Rubin, S., Berthouzoz, F., Mysore, G. J., Li, W. andAgrawala, M. Content-based tools for editing audio
stories. In Proc UIST 2013, 113-122.
22. Unity. http://unity3d.com/.
23. Victor, B. Inventing on Principle (2012).
http://vimeo.com/36579366.
24. Victor, B. Learnable Programming (2012).
http://worrydream.com/LearnablePrgramming/.