Past, Present, and Future of“Toolsmiths”
in Japanese Animation
Jun Kato (Arch Inc.)in collaboration with Yuki Koyama (Graphinica Inc.), Tatsuo Yotsukura(OLM Digital, Inc.), and Koya Imamura (Toei Animation Co., Ltd.)SAS 2022 (Animation Unlocked),Hybrid (Middlesbrough, UK & Online), June 29

https://research.archinc.jp/enjunkatohttps://junkato.jp

Introduction
Past, Present, and Future of “Toolsmiths” in Japanese Animation

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

2

Who are we?
Jun Kato,Arch Inc. (Technical Advisor) / AIST (Senior Researcher)
Yuki Koyama,Graphinica Inc. (Research Scientist) / AIST (Researcher)
Tatsuo Yotsukura,
OLM Digital, Inc. (Head of R&D Division)

Koya Imamura,TOEI ANIMATION Co., Ltd. (General Manager of R&D Section)

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

3

SAS 2021 20-minute conference paper (slides available online)

Past, Present, and Future of
Storyboarding in JapaneseAnimation
Jun Kato, Ryotaro Mihara, Nao Hirasawa (Arch Inc.)SAS 2021 (Animated Energies), Online, June 15

https://research.archinc.jp/enOur research question
Many studies of animation begin with a question about theobject—what is anime?—but I suggest a different entry point:
Who makes anime?
Ian Condry, “The Soul of Anime,” p.3

We pose yet another entry point—our research began with
understanding our role as “toolsmiths” for creators and its
potential for helping produce good anime, which can besummarized as: how to make anime?

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)5

Toolsmiths?

• Fred Brooks. ‘96. “The Computer Scientist as Toolsmith II.” CACM Vol.39 (3), 61-68.
• “The art challenges the technology, and the technology inspires the art.” [JohnLasseter, https://twitter.com/Pixar/status/93002111918415872]• “Making animation films is like a fusion of art and technology.” [Kenichi Anjo, from
our interview, 2022]
9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

6

Dawn of computer-aided
animation
Past, Present, and Future of “Toolsmiths” in Japanese Animation

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)7

Three case studiesin the dawn of computer-aided animation

New York Institute of Technology
Computer Graphics Laboratory

Japan Computer Graphics
Laboratory

Toei Douga Co., Ltd.
(Toei Animation Co., Ltd.)

NYIT example image from https://blog.siggraph.org/2020/08/pioneering-pixels-the-nyit-computer-graphics-lab-then-and-now.html
JCGL logo from https://note.com/nahnah/n/ne7e30432c6fe
Toei Animation logo from https://corp.toei-anim.co.jp/ja/company/about_pero.html

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

8

New York Institute of Technology
Computer Graphics Lab (NYIT CGL)
• Founded in 1974, initially led by Edwin Catmull (later hired byLucasfilm and led a team that became Pixar in 1986)• Initial focus: 2D CG animation• Example outcome: Tween (Ed Catmull), Paint (Alvy Ray Smith), andSoftCel (Garland Stern)• Continuous film and paper publications at ACM SIGGRAPH(top-tier academic conference on computer graphics)
My belief was, and still is, that we show all our work and that we publish. It is an illusion to thinkthat we get ahead by holding on to secrets. The real issue is: How do you attract the bestpeople? You do this by encouraging publishing and open communication in a community.
I have lifelong friends in the SIGGRAPH community. – Ed Catmull, 2020
9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

9Japan Computer Graphics Lab, Inc.
(JCGL)• The 1st commercial CG studio in Japan, founded in 1981
and led by Mitsuru Kaneko, faced financial issues and got
acquired by Namco Inc. in 1988
• Initial focus: 2D CG animation• Imported and adapted NYIT CGL software (e.g., Tween)• Collaborated with Agui/Nakajima Lab, Tokyo Institute of Technology• Gradually shifted focus to 3D and 2D/3D hybrid animation• Significant contributions to Japanese CG research/industry• Employees/collaborators continued their work
• Founding member of CG-ARTS (Computer Graphic Arts Society)9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)10

Toei Douga Co., Ltd.
(Toei Animation Co., Ltd.)• The commercial animation studio with the longest history
in Japan, founded as Toei Douga in 1956 (with its former
history as Nihon Douga)• Primary focus: digital transformation of analog workflow• Discussed use of NYIT CGL software in late 1970s and gave up• Collaborated with Mitsubishi Electric in 1980s and failed• Collaborated with Toshiba to design Computer-Aided Toei
Animation System (CATAS) in 1985 and implemented it by 1991

• Continuous effort in digitizing/improving production workflow

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)11

A very brief summary (1970s—1980s)

NYIT CGL (Lucasfilm, Pixar)

JCGL

Toei Animation Co., Ltd.

• shifted focus to 3D andmarked a major success ina 3D CG film 20 years later• innovated the field in Japanbut suffered from rapidly
deprecating hardware• iterated cost-benefit
analysis and investment on
digital production workflow• continuously collaborated
with academic researchers• collaborated with domesticresearchers• collaborated with domestic
engineering companies

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)12

Three case studies, three perspectives• Technical issues in aiding 2D animation production• It is ultimately a process of drawing thousands of frames• 3D CG has the mathematical representation whose animation canbe described as computer programs in a relatively easier manner• Overhead costs of altering existing production workflow• Hundreds of people with diverse roles are involved• Cost-benefit estimation is always necessary• Needs for sustainable R&D• It takes time to recoup the technical investment• Toolsmiths respect the community (regardless of their affiliations)
9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

13

TicTacToon, a tool in France in 1990s(later bought by Toon Boom Technologies)
Why are computer graphics tools so difficult toapply to 2D animation? It is not enough tosimply solve technical problems. The studiomust also be convinced. Today’s creation
process is essentially a production line, in whicha studio of 50 to 300 people work together toproduce tens of thousands of drawings for asingle feature film or television episode.
Everyone has a specified role and follows
detailed procedures to move from one stage tothe next. Any attempt to computerize thetraditional set of tasks must take into accountoverhead costs in both time and quality.
Jean-Daniel Fekete et al., “TicTacToon: A Paperless
System for Professional 2D Animation,” In Proc. ACM
SIGGRAPH ’95, 79-90
9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

14

Digitization and beyond
Past, Present, and Future of “Toolsmiths” in Japanese Animation

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)15

Toolsmiths in tool vendors• Anime production has typically been carried out by a group ofsmall companies, making it difficult to invest in R&D• Tool vendors have played key roles in R&D and community
management such as CELSYS, Inc. and Wacom Co., Ltd.
• Foreign companies also provided tools for Japanese anime
production such as TVPaint, Toon Boom, and CACANi

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

16

Current anime production workflow
Most part of the pipeline can alreadybe handled by digital tools:• Word processors (1980s): scenario• RETAS STUDIO (1990s-2008):
scanning, painting, composition• CLIP STUDIO, Photoshop, Procreate,etc. (2000s-): scanning, painting• Adobe AfterEffects, etc. (2010s-):
composition• Stylos (RETAS STUDIO), CLIP
STUDIO EX, Toon Boom Harmony,
TVPaint Animation, OpenToonz,CACANi, etc. (2010s-): key andinbetween animation

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

17More recent examples of toolsmiths
in anime production companies• OLM Digital Inc.
• Founded in 1995 as a child company of OLM, hired Kenichi Anjo (former researcher atHitachi, Ltd.) and started building tools for creators and contributing to the international
research community• Graphinica Inc.
• Founded in 2009 as a successor of Gonzo digital division, having strengths in 2D/3D hybrid
anime production, recently started investment for use of real-time rendering techniques forcel shading anime production• Project Studio Q, Inc.
• Founded in 2017 by a joint investment by Khara, Inc., Dwango Co., Ltd., and Aso College
Group, investing on IT and education, recently open-sourcing Blender-related tools• Arch Inc.
• Founded in 2017 and started its R&D division Arch Research in 2018, building tools for preproduction process such as a digital storyboarding tool
9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

18Other production studios
with strengths in digital anime production• Polygon Pictures Inc.
• Founded in 1983 (the commercial CG studio with the longest history in Japan) and started
making TV anime series in 2014 (Knights of Sidonia), known for its thorough production
workflow management• Digital Frontier Inc.
• Founded in 2000 with strengths in similar domains to Polygon Pictures Inc. (CG/VFX),
collaborating with other anime productions• Orange Co., Ltd.
• Founded in 2004 with strengths in cel shading animation production, recently starting toadvertise their research and development outcome

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

19

Local events for artists (and toolsmiths)• Ani-tsuku (あにつく) since 2015
• Animation Creative Technology Forum since 2015
• CGWORLD Creative Conference since 2011

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

20

Recap of the “past” section – what now?
• Technical issues
are tackled bydata-intensive
approaches (e.g.,deep learning)• Overhead cost is
not the major issue
any more• Sustainable R&D
is always important
9:00-10:30am, Jul. 29 (Wed), 2022"Toolsmiths" in Japanese Animation (SAS 2022)

21

A very brief summary (1990s—2010s)• Benefits start to win against costs of digitizing the workflow• Cel (celluloid) sheets stopped being produced• High-resolution videos had become mandatory for streaming• Young creators are used to digital tools and welcome new ones

• Creative use of digital tools had enabled production of highquality anime films, but creative development of novel
digital tools had still been quite rare
• Academic research had faced difficulties contributing tothe anime industry because of the technical issues andcomplex industrial structure
9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

22

State-of-the-art inR&D for anime production (2020s–)• Experimental animations created by small teams set the bar of the newstandards [Kenichi Anjyo and Nao Hirasawa]• Past examples: Pixar short films [Computer Animation Festival at SIGGRAPH]• Recent domestic examples: music videos• “Agile” anime production has become more and more important• Collaborations beyond each production studio – between academia andindustry – are starting to take place• Academic conferences have invited technical artists and toolsmiths from theanime production companies [CGVI workshops]• Anime production companies have started providing public datasets for academic
research [TRIGGER Inc., Arch Inc.]
Related: SIGGRAPH Asia 2021 panel https://research.archinc.jp/en/events/siggraph-asia-2021/
9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

23

Future outlook• Most part of the pipeline can be theoretically handled bydigital tools, but it is just the beginning (from Moon to Mars)• Novel tools have been developed for use in anime production,such as Unity and Blender, enabling an entirely new
production pipeline (to Saturn!?)

In either way, toolsmiths are
exploring the answer to “whatmakes a good anime?” withcollaborations with artists...
The figure was originally created by Koya Imamura (TOEI ANIMATION Co., Ltd.) andadapted for use in this talk

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

24

Past, Present, and Future of“Toolsmiths”
in Japanese Animation
Jun Kato (Arch Inc.)in collaboration with Yuki Koyama (Graphinica Inc.), Tatsuo Yotsukura(OLM Digital, Inc.), and Koya Imamura (Toei Animation Co., Ltd.)SAS 2022 (Animation Unlocked),Hybrid (Middlesbrough, UK & Online), June 29

https://research.archinc.jp/en

Appendix
Past, Present, and Future of “Toolsmiths” in Japanese Animation

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

26

Jun Kato 加藤 淳

junkatohttps://junkato.jp

The University of Tokyo Igarashi Lab ’09 BSc, ’11 MSc, ’14 PhD
Microsoft Research Asia ’12/1-4 Research Intern / Microsoft Research ’12/6-9 Research Intern
Adobe Creative Technologies Lab, Seattle ’13/8-11 Research InternNational Institute of Advanced Industrial Science and Technology (AIST) ’14/4- Researcher, ’18/10- Senior Researcher
Arch Inc. ’18/7- Technical Advisor, Principal Investigator of Arch Research

TextAlive[ACM CHI '15]

Songle Sync
[ACM Multimedia '18]

TextAlive for lyric video
Hatsune Miku “Magical Mirai”

Griffith Sketch[2020]

Griffith[2020-2022]

Conference talk on Storyboarding[SAS '21]

Human-Computer Interaction researcher studying creative activities e.g., programming,
video authoring, storyboarding and building and deploying creativity support tools
Hatsune Miku “Magical Mirai” credits: © Crypton Future Media, INC. www.piapro.net / © SEGA, Graphics by SEGA / MARZA ANIMATION PLANET INC., Organized by TOKYO MX / Crypton Future Media, INC.

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

27

Our approach• Literature review• Expert interviews and self reflections• Kenichi Anjyo

OLM Digital, Inc. (Technical Advisor)• Nao Hirasawa

Graphinica Inc. (CEO), Arch Inc. (CEO)• Hiroshi Moriguchi

Graphinica Inc. (Animation Producer)• Hiroyasu Kobayashi

Project Studio Q, Inc. (CEO)• (Anonymous)

A person working at a famous tool vendor• Yusuke Matsui

The University of Tokyo (Lecturer)• Four of us working as researchers/managers of R&D teams in the industry

9:00-10:30am, Jul. 29 (Wed), 2022

"Toolsmiths" in Japanese Animation (SAS 2022)

28