ReciPic: A Tool for Generating Infographic from Recipe
Procedure Text
Advanced Sciences Computer Science 2040664
Zhang Lechang (Supervisor：Itoh Takayuki)

1

上白糖を入れて、手羽元を入れて強火にかける. (Putwater, hondashi, cooking sake, sweet sake, dark soy
sauce, caster sugar and the chicken wings in the pot,and set the strong ﬁre.)[Output]: 鍋 (pot) 水 (water) 本だし (hondashi)
料理酒 (cooking sake) みりん (sweet sake) 濃口醤油
(dark soy sauce) 上白糖 (caster sugar) 入れ (put) 手羽元 (chicken wings) 入れ (put) 強火 (strong ﬁre) かける (set)
2.1.2 Word Classiﬁcation
Word classiﬁcation is the process of classifying thesegmented units into appropriate categories and labeling them with corresponding tags. Three categories will be used in generating infographics, namely
food materials (Tag: F), cooking tools (Tag: T), andcooking actions (Tag: Ac). ReciPic implements thisfunction with a tool called PWNER [3]. A classiﬁcation dictionary has been prepared to supplement
words that cannot be recognized by this tool.
A word classiﬁcation result is shown below:[Output]: 鍋 (pot)/T 水 (water)/F 本だし (hondashi)/F 料理酒 (cooking sake)/F みりん (sweetsake)/F 濃口醤油 (dark soy sauce)/F 上白糖 (castersugar)/F 入れ (put)/Ac 手羽元 (chicken wings)/F 入
れ (put)/Ac 強火 (strong ﬁre)/T かける (set)/Ac
2.1.3 Dependency Parsing
Dependency parsing is analyzing the grammaticalstructure in a sentence and ﬁnding the relationshipbetween semantic units. ReciPic implements thisfunction with a tool called cabocha [4].
A dependency parsing result is shown below:[Output]: 鍋 (pot)/1-8 水 (water)/2-3 本だ し(hondashi)/3-4 料理酒 (cooking sake)/4-5 みりん(sweet sake)/5-6 濃口醤油 (dark soy sauce)/6-7 上白糖 (caster sugar)/7-8 入れ (put)/8-12 手羽元 (chickenwings)/9-10 入れ (put)/10-12 強火 (strong ﬁre)/1112 かける (set)/12-0
Each semantic unit is marked with two numbersconnected with a hyphen. The number before the hyphen indicates the order of this unit, and the number
after the hyphen indicates its grammatically relatedunit. Take ”鍋 (pot)/1-8” as an example, ”1” shows”鍋 (pot)” is the ﬁrst semantic unit, and ”8” shows”入れ (put)” is its grammatically related unit. It is
worth mentioning that the ”0” in ”かける (set)/12-0”
means ”かける (set)” is the last semantic unit.

Introduction

In recent years, more and more people have chosen to use recipe websites to look for recipes, cooking
advice, or share their cooking ideas. The most popular recipe website in Japan is Cookpad [1], which
has over 50 million monthly users in Japan, and an
additional 42 million monthly users across 74 countries, with 3 million recipes. To support such a large
user community and improve user satisfaction, it isnecessary to analyze the recipes in recipe websites.
A recipe in recipe websites contains the recipe
name, food image, knack&points, ingredients, andcooking procedure. Among these components, thecooking procedure is the crucial part that users will
follow. However, they are often displayed by puretext. It is diﬃcult for users to grasp valuable information and have a big picture of the food materials,
cooking tools, and cooking methods at a glance. Even
though some recipes contain food images, those images usually show only the ﬁnished dishes to attract
users rather than indicate the cooking process.
This thesis proposes ReciPic, an interactive visualization tool to make the cooking procedure text more
understandable. Our tool allows users to enter recipe
procedure text and transform it into the corresponding infographic. With the help of infographics, users
can understand the procedure text information eﬀectively.

2

Proposed Tool

2.1 Text Analysis
To understand the meaning of the input, ReciPic
will perform text analysis on it. Text analysis willﬁrst segment the input into appropriate units and extract the valuable parts, then classify diﬀerent parts
into three categories (namely food materials, cookingtools, and cooking actions), and ﬁnally analyze thedependency relation between the segmented parts. Inaddition, ReciPic will also complement the omitted
cooking tools before generating the infographic.
2.1.1 Word Segmentation
Word segmentation is the process of splitting text
into meaningful units. ReciPic implements this function with a tool called mecab [2].
A word segmentation result is shown below:[Input]: 鍋に水、本だし、料理酒、みりん、濃口醤油、1

Finally, ReciPic will combine the word classiﬁcation result with the dependency parsing result, checkif there is any omitted part in the input, complement
it with a prepared dictionary, then generate the text
analysis result.
A text analysis result is shown below：[Output]: 鍋 (pot)/T/1-8 水 (water)/F/2-3 本だし (hondashi)/F/3-4 料理酒 (cooking sake)/F/45 みりん (sweet sake)/F/5-6 濃口醤油 (dark soysauce)/F/6-7 上 白 糖 (caster sugar)/F/7-8 入 れ(put)/Ac/8-12 手羽元 (chicken wings)/F/9-10 入れ(put)/Ac/10-12 強火 (strong ﬁre)/T/11-12 かける(set)/Ac/12-02.2 Infographic Generating
After getting the text analysis result, ReciPic will
look for the illustrations of all the food and tools ina prepared database and combine them according to
the infographic-generating rules shown in Figure 1.
Which rule to pick is determined by the cooking
action relevant to the food and tool. For example,
cooking action 入れ (put) means putting the foodinto the tool, so rule (d) will be picked. Action 切る(cut) means cutting the food on the chopping board,so rule (b) will be selected. Action 洗う (wash) means
washing the food under the water tap, so rule (e) will
be picked.

Figure 2: Generated infographic

4Conclusion

This thesis introduced ReciPic, an interactive toolthat accepts a recipe procedure text as input, transforms it into an infographic, and then represents it tousers. We explained the key components of ReciPicin detail and experimented with an input example.
In the future, we plan to test ReciPic with abroader input range and complement the text analysis dictionaries and the illustration database to makeit applicable to more recipe texts. In addition, wewill extract other words, such as adjectives and thequantity of ingredients, and discuss how to present
them in the infographic.

References[1] Cookpad company, https://cookpad.com/.
Figure 1: Rules for generating infographics[2] Mecab, Part-of-Speech and Morphological
Analyzer, https://taku910.github.io/mecab/.

3[3] PWNER, PointWise-based Named Entity
Recognizer, www.lsta.media.kyoto-u.ac.jp/resource/tool/PWNER/home.html/.

Experimental Result

An experiment result of ReciPic is shown below:[Input]: 鍋に水、本だし、料理酒、みりん、濃口醤油、上白糖を入れて、手羽元を入れて強火にかける. (Putwater, hondashi, cooking sake, sweet sake, dark soysauce, caster sugar and the chicken wings in the pot,and set the strong ﬁre.)[Output]: 鍋 (pot)/T/1-8 水 (water)/F/2-3 本だし (hondashi)/F/3-4 料理酒 (cooking sake)/F/45 みりん (sweet sake)/F/5-6 濃口醤油 (dark soysauce)/F/6-7 上 白 糖 (caster sugar)/F/7-8 入 れ(put)/Ac/8-12 手羽元 (chicken wings)/F/9-10 入れ(put)/Ac/10-12 強火 (strong ﬁre)/T/11-12 かける(set)/Ac/12-0[Infographic]: The result is shown in Figure 2.

[4] Cabocha, Japanese Dependency Structure
Analyzer, http://taku910.github.io/cabocha/.

2