4-Frame Manga Drawing Support System
Yuto Nagao
Kwansei Gakuin UniversitySanda, Japan
gdu84444@kwansei.ac.jp
ABSTRACTThis project proposes a 4-frame manga drawing support
system that assists users in creating drawings. The proposedsystem recognizes each frame of an unfinished manga
drawn by a user and proposes successive frames,
considering the content recognized till then, e.g., thestoryline, frame composition, and punchline. The system
updates the proposal as the manga drawing proceeds. The
proposed system comprises four modules for 1) recognizing
the user’s drawings, 2) generating four sentences that
describe the storyline, 3) generating images, each of whichcorresponds to a manga frame from the above sentences,and 4) choosing the user-preferred manga candidate from anumber of potential choices. The proposed system does not
require AI to generate the 4-frame manga; instead, the userdraws the 4-frame manga with the help of the system. In
other words, the user decides whether to accept or reject theAI’s proposal.
Author Keywords

Soichiro Fukuda
Kwansei Gakuin UniversitySanda, Japan
gsw81451@kwansei.ac.jp
The system recognizes each frame of the unfinished manga
drawn by the user, and proposes a succession of frames,
taking into account the content recognized so far, e.g., thestoryline, frame composition, and punchlines. “Silentmanga”, prevalent in Japan, is a manga without dialogue,
and the 4-frame manga handled by the proposed system islimited to “silent manga”. While the story of the manga isgenerated by text-generation AI, the frames of the mangaare generated by image-generation AI. With the proposedsystem, the user first starts drawing a 4-frame manga, asshown in Figure 1. The AI recognizes the user’s 4-frame
manga drawing and proposes candidates for successiveframes. In Figure 1, the user draws two frames of a 4-frame
manga, with the ideas “I found a wallet → I took it to thepolice station”. In response, AI proposes two successiveframe strips: “go shopping at the supermarket → take out
the wallet from the pocket only to realize that it is the one I
found earlier” (i.e., the wallet left in the police station is theuser’s).

Image-generation AI; Image-recognition AI; Claude largelanguage model
ACM Classification Keywords

H.5.m. Information interfaces and presentation
INTRODUCTION

This project proposes a 4-frame manga drawing support
system, in which AI is employed to suggest 4-frame manga
ideas to the users, as they create their own 4-frame mangadrawings. 4-frame manga is a form of Japanese manga, inwhich four frames are used to draw a short story. Recently,
a project called TEZUKA2023 employed two generativeAIs for drawing manga in cooperation with humans. A
rudimentary story is first generated by GPT-4 [1], while thecharacter faces and frames are generated by Stable
Diffusion [2]. In this proposed system, AI first presents
ideas for the storyline and characters; however, the actual
frame composition and dialogue of the manga need to becreated by the user, referring to the ideas suggested by AI.
Paste the appropriate copyright/license statement here. ACM now supports
three different publication options:• ACM copyright: ACM holds the copyright on the work. This is thehistorical approach.
• License: The author(s) retain copyright, but ACM receives anexclusive publication license.
• Open Access: The author(s) wish to pay for the work to be open
access. The additional fee must be paid to ACM.
This text field is large enough to hold the appropriate release statement
assuming it is single-spaced in Times New Roman 8-point font. Please do
not change or modify the size of this text box.
Each submission will be assigned a DOI string to be included here.

Figure 1. System overview
METHODIn the proposed system, as the user starts drawing a 4-frame
manga, the AI continuously generates the storyline, framecomposition, and punchline, and proposes them to the user.
The proposed system requires four modules for 1)
recognizing the user’s drawings, 2) generating foursentences that describe the storyline, 3) generating images,
each of which corresponds to a manga frame from theabove sentences, and 4) choosing the user-preferred manga
candidate from potential candidates.

employed to recognize the user’s facial expressions in realtime and ranks the candidates.

The first module will be developed by employing both
image-generation AI and image-recognition AI. As it isdifficult to train AI to directly recognize drawings by users,
the system generates a photographic image of the object
drawn in each frame, employing image-generation AI. The
generated image is then converted into an object description
text, by employing image-recognition AI. In Figure 2, theuser draws two frames that imply “I found a wallet”, and “I
took it to the police station”. The first frame is converted toa photographic image of a wallet and the second to that of apolice station by the image-generation AI. The object
descriptions “Wallet” and “police station” were obtained
from the photographic images of the first and secondframes, respectively, by the image-recognition AI.

Figure 3. Module for generating four sentences that describe
the storyline

Figure 2. Module for recognizing user drawings

The second module will be developed by employing Claude
large language model (LLM). The Claude LLM generates asentence for each manga frame. Thus, it generates foursentences that depict the storyline, given the two object
descriptions, which are “I found a wallet”, “I took it to thepolice station”, “I went shopping at the supermarket”, and“I took out the wallet from my pocket and realized that it
was the one I found earlier”. Each of these sentences isconverted again into an object description of thecorresponding frame by Claude LLM as “Wallet”, “PoliceStation”, “Shopping Cart”, and “Wallet”; this is depicted inFigure 3.
The third module proposes 4-frame manga candidates
generated from the object descriptions obtained above using
the image-generation AI, as depicted in Figure 4. Such AI
proposals will be made interactively even as the user
continues the drawing. Finally, the fourth module obtains
the “best” candidate interactively by considering the user'sactions and facial expressions. The image-recognition AI isFigure 4. Module for generating images, each of whichcorresponds to a manga frame from the sentences generated in
the third module
CONCLUSION

We propose a 4-frame manga drawing support system that
employs AI, and recognizes a user’s drawings, generates
four sentences that describe the storyline, generates images,
each of which corresponds to a manga frame from theabove sentences, and finally chooses a user-preferredmanga candidate from a number of potential choices. With
the help of the proposed system, users can generate theirown manga drawings.

REFERENCES

1.

Open AI. 2023. GPT-4. Retrieved August 12, 2023
from https://openai.com/research/gpt-4

2.

Stability.ai. 2023. Stable Diffusion XL. Retrieved
August 12, 2023 from https://ja.stability.ai/stablediffusion