Hiroshima City University
at NTCIR-11 Cooking Recipe Search Task
Hidetsugu Nanba
Hiroshima City University
3-4-1 Ozukahigashi
Hiroshima 731-3194 JAPANABSTRACTOur group participated in the subtask involving an ad hoc
Japanese recipe search. Our goal was to evaluate the effectivenessof our Japanese cooking ontology for the recipe search. To
investigate the effectiveness of our ontology-based approach, we
conducted experiments and found that our method can improveupon traditional document retrieval systems.

each entry word in Sections 2.2 and 2.3, respectively. We also
show the statistics of our ontology in Section 2.4.

Keywords
ontology, synonym, query expansion

1. INTRODUCTION
There has recently been an increase in research work focusing on
cooking recipes, including recommendation [7], summarization[8], and predicate-argument structure analysis [5]. However,different terms appear in different recipes, particularly in usergenerated recipe-sharing sites, even though these terms often referto the same thing. We have therefore constructed a cooking
ontology that can be used in a variety of language processing
tasks as a linguistic resource [6]. Our goal in NTCIR-11 [9] wasto evaluate the effectiveness of our ontology for the recipe search.
This paper is organized as follows. Section 2 explains the method
for constructing a cooking ontology. Section 3 contains a systemdescription. To investigate the effectiveness of our method, we
conducted experiments, as reported in Section 4. We present ourconclusions in Section 5.

2. A JAPANESE COOKING ONTOLOGY
2.1 Overview of Our Cooking Ontology
The structure of our cooking ontology is shown in Figure 1. Our
ontology employs a two-level hierarchy. The top level comprises
the following seven categories.

Figure 1: Structure of Our Cooking Ontology.

2.2 Determination of Entry Words
We determined entry words via the following two substeps.
(Step 1-1) Collecting candidate words from patents(Step 1-2) Selecting entry words manuallyl

Ingredient - seafoodl

Ingredient - meatl

Ingredient - vegetable

In Step 1-1, we applied Hearst’s method [2] to patents1, andcollected candidates for entry words. For the following fivecategories, we prepared seed words that were synonyms of eachcategory name, and then collected hyponyms using the pattern“NP0 ((、|や)NPn)* (等|など)の[seed word]” ([seed word] suchas NP0 ((,|and|or) NPn)*).

lIngredient - otherll

Condimentl

Kitchen toollMovement

Among these categories, “Movement” is a category involving
verbs, while the others involve nouns. Each category contains
several entry words. For example, in Figure 1, the ingredientseafood category comprises several entry words such as “squid”
and “shrimp.” For each entry, several related terms are classified
into three categories: “synonymy,” “meronymy,” and “attribute.”
Among these categories, we used synonymy relation for the recipe
search. In the following section, we will introduce (1)determination of entry words and (2) collection of synonyms forl1

Ingredient – seafood
魚類(fish)，魚介類(fish)，海産物(sea product)，水産物(fishery product)
Ingredient – meat
肉類(meat)，食肉(edible meat)，食肉類(edible meat)，
原料肉(ingredient meat)

We used those unexamined Japanese patent applications over 19
years (1993-2011) to which any of the International PatentClassification codes A23L (foods, foodstuffs, or non-alcoholic
beverages), A47J (kitchen equipment), or H05B (electricheating, electric lighting) were assigned.

l

l

l

Ingredient – vegetable
野 菜 (vegetable) ， 果 菜 類 (fruit vegetable) ， 野 菜 類(vegetable)，果菜物(fruit vegetable)，農産物(agricultural
products)
Condiment
調味料(condiment)，香辛料(spice)，薬味(condiment)，
スパイス類(spice)
Kitchen tool
調理器具(kitchen tool)，調理容器(cooking container)，
調理器(cooking device)，調理具(cooking tool)，調理道
具(cooking utensil)

For example, when we collected candidate words for the
ingredient-seafood category, we found sentences that contained
patterns such as “(NP0 ((、|や)NPn)*)などの魚類” (fish such as((,|and|or) NPn)*)) or “(NP0 ((、|や)NPn)*)等の水産物” (fisheryproduct such as ((,|and|or) NPn)*)). We then extracted nounphrases (NP0 and NPn), such as “イカ” (squid) or “エビ” (shrimp),
as candidates for entry words in the ingredient-seafood category.
Although Hearst’s pattern-based method is useful for collecting
hyponyms from texts, there are several cases where inappropriate
words are mistakenly extracted. In the following sentence, “食用”
(edible use) and “鑑賞用” (ornamental purpose) are mistakenly
extracted as candidates for the ingredient-seafood category.
食用や観賞用等の魚介類をいう。
(This indicates fish for edible use and for ornamental purposes)
We therefore delete such inappropriate words manually from thecandidate list in Step 1-2. From among the remaining candidates, we
statistically determined one representative word for each group ofsynonyms. As an example, for the three candidates “サケ” (salmon),“鮭” (salmon), and “さけ” (salmon), we manually selected “サケ” as
the representative word, because the frequency of the phrase “サケ(など|等)の魚介類” (fish such as a salmon) is greater than those of“鮭(など|等)の魚介類” (fish such as a salmon) and “さけ(など|等)の魚介類” (fish such as a salmon).
We selected ingredient words that do not belong to any of theingredient-seafood, -meat, and -vegetable categories as entry words inthe ingredient-other category. Most of the words in this category
are processed foods, such as cheese and pasta.
For entry words in the movement category, we manually selectedverbs appear frequently in the Rakuten Data provided by Rakuten, Inc.

2.3 Collection of Synonyms for Each Entry
Words
The procedure of collecting synonyms for each entry word
comprises the following two substeps.
(Step 2-1) Collecting candidates for synonyms.
(Step 2-2) Identifying synonyms manually.

main ingredient is usually written first in the ingredient list of arecipe, he assumed that this first ingredient is strongly related tothe category to which the recipe belongs. They confirmed
experimentally that his method for calculating relation scores
between ingredients and category names using the ingredientposition was effective for collecting synonyms from a recipe
database. We use this method to extract synonyms.
As we explained in the section on related work, the basic idea of
distributional similarity is to calculate the similarity between two
words in terms of their context words. Our algorithm is as follows.
1.

2.
3.
4.
5.
6.

Analyze the dependency structures of all sentences in the
Rakuten Data, which contains about 440,000 recipes, using theJapanese parser CaboCha2
Extract <noun phrase><verb> pairs that have dependency
relations from the dependency trees obtained in Step 1
Count the frequencies of each <noun phrase><verb> pair
Collect verbs and their tf*idf scores for each noun phrase,
creating indices for each noun phrase
Calculate the similarities between two indices for noun phrases
using the cosine distanceObtain a list of synonymous noun phrases

In addition to collecting verbs for each noun phrase in Step 4 of thealgorithm, we collected noun phrases for each verb similarly,
obtaining a list of synonymous verbs.
In Step 2-2, we selected synonyms from the candidates obtained using
the three methods. The characteristics of these methods are
summarized in Table 1. We checked all candidates collected by
Methods 1 and 2, because the numbers of candidates were small. The
candidates collected by Method 3 were checked in order of similarity
to each other as much as possible.

Table 1. Characteristics of the Three Methods Used forCollecting Synonyms.
Reliability
Method 1

Fully reliable

Number ofcandidates
Very small

Targetcategory
All

Highly reliable

Small

Moderately
reliable

Very large

Exceptfor“Movement”
All(Deleted words in
Step 1-2)

Method 2(Chung)
Method 3(distributional
similarity)

2.4 Statistics of Entry Words and Synonyms
In Table 2, we show the numbers of entry words that werecollected using the method mentioned in the previous section.

In Step 2-1, we used the following three methods.
(Method 1) Using words that were deleted in the process ofdetermining representative words in Step 1-2(Method 2) Chung’s method [1](Method 3) Distributional similarity method [3, 4]
We have already described Method 1. Here, we explain Methods 2
and 3. Chung [1] proposed a method for extracting synonyms
based on a recipe data structure. From the observation that the2

https://code.google.com/p/cabocha/

Table 2. The Number of Entry Words for Each
Category.

(Step 2) Ontology-based query expansion

Category

4. EVALUATION

Number of entry words

Ingredient - seafood

61

Ingredient - meat

6

Ingredient - vegetable

122

Ingredient - other55

Condiment

51

Kitchen tool48

Movement

131

Total

474

We expand queries using the synonyms obtained in Section 2.
We used 500 Japanese queries for the ad hoc subtask to evaluate
our method. All the systems were evaluated in terms of mean
average precision (MAP).

4.1 Submitted Systems
We submitted the results produced by two systems: “HCU-JA1BASE-01” and “HCU-JA1-TEST-01.” Each system used a title, alist of ingredients, and procedures for creating an index for each
recipe. The difference between these systems is that “HCU-JA1TEST-01” used the ontology-based query expansion. We show
the official results in Table 4.

Table 4. Official Results for Our Systems.

In Table 3, we show the numbers of synonyms for each category
together with the number of synonyms for each entry word.

Table 3. The Number of Synonyms for Each Category.
Category

The number of synonyms(per each entry word)

Ingredient - seafood453 ( 7.4)

Ingredient - meat

383 (63.8)

Ingredient - vegetable

947 ( 7.8)

Ingredient - other732 (13.3)

Condiment

909 (17.8)

Kitchen tool

643 (13.4)

Movement

956 ( 7.3)

Total

5,023 (10.6)

Systems

MAP

MRR

nDCG@10HCU-JA1BASE-01

0.0706

0.07630.1441HCU-JA1TEST-01

0.06670.07000.1441After we submitted the results, we found some parameter setting
errors when combining multiple items. We therefore conducted
some additional experiments as reported in the following
subsections.

4.2 Identification of the Best Combination of
Items
We constructed the following 11 systems to enable identificationof the best combination of items, using items listed in Step 1 ofSection 3. In this examination, we did not use the ontology-based
query expansion.

3. SYSTEM DESCRIPTIONl

TITLE: Create an index using a title.

Our system comprises the following two steps:l

ING: Create an index using a list of ingredients.

(Step 1) Morphological analysisl

PROC: Create an index using a procedure.

We introduce a vector space model as the retrieval model and use
Solr3 as the retrieval engine. Here, each recipe in the Rakuten
Data comprises the following items.

l

TAG1: Create an index using tag 1.

l

TAG2: Create an index using tag 2.

l

A titlel

TAG3: Create an index using tag 3.

lA list of ingredientsl

CATEGORY1: Create an index using the 1st category level.

lProceduresl

CATEGORY2: Create an index using the 2nd category level.

lThree levels of category (1st, 2nd, and 3rd levels)l

CATEGORY3: Create an index using the 3rd category level.

lThree tags (tags 1, 2, and 3)l

EXP: Create an index using the explanation of the recipe.

l

An explanation of the recipell

Know-how information

KNOW-HOW:
information.

We use one or more of these items to create an index of each
recipe, which we will describe in more detail in Section 4.2.
3

http://lucene.apache.org/solr/

Create

an

index

using

know-how

We examined each of these systems in terms of the dataset of thead hoc subtask. The experimental results are shown in Table 5.

Table 5. Evaluation Results for 11 Systems Using aSingle Item.
MethodMAP

Method

MAP

ING

0.5734TAG3

0.0056PROC

0.2110

CATEGORY3

0.0049TITLE

0.1950KNOW-HOW

0.0013EXP

0.0423

CATEGORY2

0.0004

TAG1

0.0148

CATEGORY1

0.0000TAG2

0.0074In the next step, we combined items that obtained top-n MAP
scores. In combining these multiple items, we gave the samewords appeared in different items the same weight. The results areshown in Table 6.

Table 6. Evaluation Results for Combinations of Some
Items.
Methodl

SEAFOOD+MEAT+VEGE: Query expansion usingsynonyms in the ingredient-seafood, -meat, and -vegetable
categoriesl

SEAFOOD+MEAT+VEGE+CONDI: Query expansion
using synonyms in the ingredient-seafood, -meat, vegetable, and condiment categories

In this examination of these five methods, we employed thenumber of retrieved relevant documents within the top-100 results
as another evaluation measure for investigating the effects of ourontology-based query expansion approach.
The results are shown in Table 7. Here, BASELINE indicates the
ING+PROC+TITLE method of Table 6. As can be seen from
Table 7, we can confirm that VEGE is useful for improving the
MAP score of the BASELINE system. Although the MAP scoreof the SEAFOOD+MEAT+VEGE method was lower than the
BASELINE method, the number of retrieved relevant documents
was slightly improved.

Table 7. Evaluation Results of Ontology-based Query
Expansion.
Method

MAP

ING

0.5734

ING+PROC

0.5736

ING+PROC+TITLE

0.7518

ING+PROC+TITLE+EXP

0.7277

ING+PROC+TITLE+EXP+TAG1

0.5271MAP

The number of
retrieved relevant
documents within
the top-100 results

SEAFOOD

0.7508601

MEAT

0.7306594

VEGE

0.7635596

SEAFOOD+MEAT

0.7406603

0.6858587

0.7518599+VEGE
As can be seen from Table 6, the combination of
ING+PROC+TITLE obtained the best MAP score. For thiscombination, we also examined a variety of weights. We
considered the MAP scores in Table 5 to be the reliability scoresof each item, and employed these scores as weights of each field;that is, we employed weight values of 0.5734, 0.2110, and 0.1950
for ING, PROC, and TITLE, respectively. We examined the
combination of ING, PROC, and TITLE using these weights, andobtained a MAP score of 0.7156, which is lower than the result of
ING+PROC+TITLE in Table 6. We therefore used the
combination of ING, PROC, and TITLE items with the same
weighting for each item in the next step; namely, the ontologybased query expansion.

SEAFOOD+MEAT+VEGE+CONDI
BASELINE

There is a reason that the VEGE method was an improvement on
the BASELINE method, whereas both the MEAT and SEAFOOD
methods were not. It is mainly a result of the policy used inconstructing the ontology. Consider these two examples ofsynonyms for a carrot and a salmon.
l

Synonyms of “carrot”: ニンジン (carrot), 金時ニンジン(Japanese red Kintoki carrot), にんじん (carrot), 人参(carrot), キャロット (carrot)l

Synonyms of “salmon”: サケ (salmon), さけ (salmon), 鮭(salmon), シ ャ ケ (salmon), し ゃ け (salmon), 秋 鮭(autumn salmon), 紅 鮭 (sockeye salmon), サ ー モ ン(salmon), キングサーモン (king salmon), サケフレーク(salmon flakes), さけフレーク (salmon flakes), 鮭フレー
ク (salmon flakes), シャケフレーク (salmon flakes), しゃけフレーク (salmon flakes), 塩鮭 (salted salmon), 生鮭
(raw salmon)

4.3 Ontology-based Query Expansion
We constructed the following five methods using the
ING+PROC+TITLE method, which had obtained the best
performance, as described in Section 4.2.
l

SEAFOOD: Query expansion using synonyms in theingredient-seafood categoryl

MEAT: Query expansion
ingredient-meat categoryl

VEGE: Query expansion using synonyms in the ingredientvegetable categoryusingsynonyms

inthe

We can identify that some intermediate products, such as しゃけ
フレーク (salmon flakes) and 塩鮭 (salted salmon), are also akind of synonym. The ratios of such intermediate products in theingredient-meat and ingredient-seafood categories are larger thanfor the ingredient-vegetable category, and this can often cause adecrease in the MAP scores.

5. CONCLUSION
In this paper, we proposed an ontology-based query expansion forthe subtask of ad hoc Japanese recipe search. One of our methods,
namelyVEGE,
outperformedabaselinemethodING+PROC+TITLE. From our results, we confirmed theeffectiveness of our ontology for recipe search.

ACKNOWLEDGMENT
We would like to express our gratitude to Rakuten, Inc., which
provided us with the Rakuten Data.

REFERENCES[1] Chung, Y. Finding food entity relationships using usergenerated data in recipe service. Proc. 21st ACM
International Conference on Information and Knowledge
Management (CIKM2012) (2012) 2611-2614.
[2] Hearst, M.A. Automatic acquisition of hyponyms from large
text corpora. Proc. 14th International Conference on
Computational Linguistics (1992) 539-545.
[3] Lee, L. Measures of distributional similarity. Proc. 37th
Annual Meeting of the Association for Computational
Linguistics on Computational Linguistics (1999) 25-32.

[4] Lin, D. Automatic retrieval and clustering of similar words.
Proc. COLING/ACL 1998 (1998) 768-774.
[5] Mori, S., Maeta, H., Yamakata, Y., and Sasada, T. Flowgraph corpus from recipe texts. Proc. 9th Language
Resources and Evaluation Conference (LREC 2014) (2014).
[6] Nanba, H., Doi, Y., Tsujita, M., Takezawa, T., and Sumiya,
K. Construction of a cooking ontology from cooking recipes
and patents. Proc. Workshop on Smart Technology for
Cooking and Eating Activities (CEA2014) (2014) 507-516.
[7] Ueda, M., Takahara, M. and Nakajima, S. User’s foodpreference extraction for cooking recipe recommendation.
Proc. 2nd Workshop on Semantic Personalized Information
Management: Retrieval and Recommendation (2011) 98-105.
[8] Yamakata, Y., Imahori, S., Sugiyama, Y., Mori, S., andTanaka, K. Feature extraction and summarization of recipes
using flow graph. Proc. 5th International Conference inSocial Informatics, LNCS 8238 (2013) 241-254.
[9] Yasukawa, M., Diaz, F., Druck, G., and Tsukada, N.
Overview of the NTCIR-11 Cooking Recipe Search Task.
Proc. 11th NTCIR Workshop Meeting (2014).