Acquisition of Know-How Information from Web
Shunsuke Kozawa1 , Kiyotaka Uchimoto2 , and Shigeki Matsubara1
1

Nagoya University, Furo-cho, Chikusa-ku, Nagoya, 464-8601, Japan
National Institute of Information and Communications Technology
3-5 Hikari-dai, Seika-cho, Soraku-gun, Kyoto, 619-0289, Japankozawa@el.itc.nagoya-u.ac.jp, uchimoto@nict.go.jp, matubara@nagoya-u.jp
2

Abstract. A variety of know-how such as recipes and solutions for troubles have been stored on the Web. However, it is not so easy to appropriately ﬁnd certain know-how information. If know-how could be appropriately detected, it would be much easier for us to know how to tackle
unforeseen situations such as accidents and disasters. This paper proposes a promising method for acquiring know-how information from theWeb. First, we extract passages containing at least one target object andthen extract candidates for know-how from them. Then, passages containing the know-how are discriminated from non-know-how informationconsidering each object and its typical usage.
Keywords: know-how, how-to type question answering, object, usage
information, procedural question

1

Introduction

A variety of know-how such as recipes and solutions for troubles have been
stored on the Web and they are often referred to by using web search. It has
been reported that about 40% of all non-factoid questions on Q&A sites are
how-to questions [8], and the know-how information has potential to give us theanswers to the questions. However, it is not so easy to appropriately ﬁnd knowhow information related to a particular how-to question. If know-how could beappropriately detected, it would become easy to ﬁnd answers to how-to questions. Also, it would become possible to let us know how to tackle unforeseen
situations if know-how information could be stored beforehand and classiﬁed
into several classes according to their topics.
In this research, we assume that know-how is a procedure or an advice. Figure
1 shows two examples of know-how. In previous researches, know-how information was acquired by using typical words such as “how to” and clue words suchas “only if” and “ﬁrst” [10]. However, it is diﬃcult to eﬃciently acquire knowhow information based on the conventional methods because a variety of clue
expressions are required to appropriately detect know-how information which isoften described without using clue words or the typical words “procedure”, “howto” and “ﬁrst” as you can see in Figure 1.
Well then, what is a clue to eﬃciently obtaining know-how information? Our
observation for this question is as follows. Firstly, know-how information often

Fig. 1. Examples of know-how

includes at least one object that plays an important role; for instance, “hairdryer” in the ﬁrst example and “thermometer” in the second one in Figure
1. Secondly, the typical usage of the object is often described in each knowhow; for instance, “to heat something” in the ﬁrst example, and “to monitor
a temperature” in the second one. As seen in the examples, know-how is often
characterized by an object name and the description of its usage. We examined
100 lists of know-how information randomly sampled from a web site1 and foundthat 75 out of the 100 lists included at least one object and the description of
its usage. This supports our intuition.
In this paper, we propose a method for acquiring know-how information byfocusing on each object and how it is used. First, we extract passages containing
at least one target object and then extract candidates for know-how from them.
Then, lists of know-how information are acquired based on the description of
the object and its typical usage.

2

Related Works

In previous works, a few studies have addressed the acquisition of proceduraltexts. Takechi et al. proposed a method to categorize HTML texts tagged with⟨OL⟩ or ⟨UL⟩ as either procedural or non-procedural by using word N-grams [7].
Aouladomar proposed a method to estimate questionability of web texts by using
tag information (title, advice, warning, etc.) annotated by rule-base methods [2,1

http://know-how.fc2.com/ (in Japanese)Fig. 2. Flow for acquiring know-how information

3] and clue expressions [1]. Yin et al. proposed a method for measuring the degreeof procedurality of texts containing the phrase “how to” by using syntactic tags,
morphological tags and cue phrases [10].
In this study, we acquire procedures and advices as know-how information.
Our proposed method uses not only clue expressions which were also used inprevious works but also an object and how it is used.

3

Acquisition of Know-How

Aouladomar classiﬁed procedural texts into three categories [1]; procedures (e.g.
recipes, maintenance and construction manuals, etc.), injunctions (e.g. orders,regulations, etc.), and advices (e.g. beauty advices, health management methods, etc.). In this research, we acquire procedures and advices as know-how information since they are frequently asked on Q&A sites, namely there are many
demands to obtain information on procedures and advices.
We acquire know-how information by focusing on an object and how it isused. In this research, we assume that objects are noun phrases which are hyponym of “physical entity” in Japanese WordNet 1.12 . We use expressions representing utilization of an object (utilization roles) to capture how the object
is used. For example, the utilization role of “hair dryer” is to “heat” and thatof “thermometer” is to “monitor.” Figure 2 illustrates the ﬂow for acquiringknow-how information. We assume that a unit of know-how is a passage sinceknow-how is very often composed of multiple sentences, but it is not always
composed of all sentences in a document. We also assume that lists of knowhow information are acquired for each object. First, given a target object, thecandidate passages for know-how are extracted by using several methods sincemost passages would not contain know-how information. Then, the candidates
2

http://nlpwww.nict.go.jp/wn-ja

are classiﬁed as either they contain know-how information or not by using clue
patterns, an object and its utilization roles.
In the following sections, we assume that the target language is Japanese
because know-how information is not well organized yet, although the method
can be expanded to any languages. We used the Web corpus consisting of 500M
Japanese parsed sentences extracted automatically from the Web [5]. The sentences in the corpus have been automatically annotated with morphological andsyntactic information. The syntactic information in a sentence is represented asa dependency structure between Japanese phrasal units, bunsetsu.
3.1

Extraction of Know-How Candidates

First, passages are extracted in the following way. Every segment that contains
at least one target object and is enclosed by a pair of the following HTML tags
is extracted:
body, div, table, span, p, blockquote, h1, h2, h3, h4, h5, h6
If the number of sentences contained in a segment is less than or equal to a
threshold α, the segment is extracted as a passage. Otherwise, we split thesegment into one or more passages with the TextTiling algorithm [4] using α as
the window size. This is because there exist Web sites where HTML tags areused incorrectly.
Next, the candidate passages that might contain know-how information are
extracted. They are extracted by using the following four conventional methods(A,B,C,D) and our proposed method (E). Multiple methods are used since theacquired know-how information might be limited when only one method is used.
We take the union of the candidate passages as the candidates for know-how.
(A) Extraction of the passages containing the term houhou (how to) [10] 3 .
(B) Extraction of the passages tagged with ⟨OL⟩ or ⟨UL⟩ tags [7].
(C) Extraction of the passages containing any of 47 expressions (e.g. “in order
not to”, “prefer”, “as long as”, etc.) which have been manually generated byreferring to the policy of generating patterns in the previous works [3, 10].
(D) Extraction of the passages containing any of 638 expressions such as “getwell” and “feel good” found in the semantic lexicon constructed by Kobayashi
et al. and tagged with keiken (experience) tags in the lexicon [6]. This isbecause we assume that know-how information contains expressions representing experience.
(E) Extraction of the passages containing any of 3-tuples which are composedof an object and its utilization roles. Utilization roles of a given object oare deﬁned as paraphrases of such expressions as “using o” or “enjoying o”
and expressed by a pair of a postposition p and a verb v. For example, theutilization role of a hair dryer is ⟨de(by), atatameru(heat)⟩. The 3-tuples
3

The term was originally proposed in English. We manually translated them into
Japanese and used them.

⟨o, p, v⟩ (ex. ⟨ doraiya(hair dryer), de(by), atatameru(heat) ⟩) are assumed
to appear in the dependent bunsetsus; o and p (doraiya+de (by using hairdryer)) appear in a bunsetsu which depends on another bunsetsu containingv (atatameru (heat)). The method for acquiring utilization roles is describedin detail in Section 3.2.
3.2

Identiﬁcation of Know-How

The extracted candidates are classiﬁed as either they contain know-how information or not by using a machine learning model. Clue patterns, part-of-speech
information, target objects and their utilization roles (3-tuple) are used as features 4 .
– clue patterns
Clue patterns are manually generated by referring to know-how and nonknow-how information in a development data. The patterns are applied toa sentence or bunsetsu (Japanese phrasal unit) sequences in a sentence ina target passage. The frequency of the sentences matched with each clue
pattern, the total frequency of the sentences matched with the patterns andthe number of types of the matched patterns are used as features.
– part-of-speech information
The appearance frequency of each part-of-speech normalized by the numberof the sentences is used as a feature.
– 3-tuples
The total number of the frequency of the 3-tuples ⟨o, p, v⟩ is used as a feature.
The target objects and their utilization roles are our newly added features.
Henceforth, we call a set of clue patterns and part-of-speech information PT
features. The following subsections describe the method for acquiring utilization.
Acquisition of Utilization Roles Torisawa made the following three assumptions about the characteristics of utilization role ⟨p, v⟩ for a given noun n: 1) An
n marked by p often appears with v. 2) First-person pronouns such as “watashi(I)” often occupy the agent role of v. 3) The postposition “de” is a good candidate of a postposition in a utilization role. Utilization roles were acquired byusing the following formula reﬂecting these assumptions [9].
U (n) = argmax(v′ ,p′ )∈V ×A {U score(n, p′ , v ′ )}V is a set of verbs, which can be a verb in possible utilization roles. As verbs
in V , we used 6,485 verbs which appeared in the Web corpus with the verbal
suﬃx “tai,” which can be translated to “want.” We also manually removed 20
verbs that can never take utilization roles such as “naru (become)” or mean
4

Word n-grams were not used because they did not work well in our preliminary
experiments.

Table 1. Sizes of training and testing data
# of passages containing know-how # of passages
objectsA B C D E
total
air conditioner 14 16 33 5 670492
cell-phone
11 5 11 0 027495
digital camera
8 14 19 5 953487
electric fan
17 29 37 5 11
92
476
iron
34 43 31 9 37
144
468
microwave
24 61 27 13 69
183
482
oven
28 79 32 7 70
208
476
refrigerator
28 56 22 1 60
165
494
vacuum cleaner 14 34 31 5 45117478
washer13 26 14 3 27
78
479
total
201 363 258 53 334
1137
4827

literally “using” and “enjoying” such as “tsukau (use)” from V. A is a set of
postpositions.
U score(n, p′ , v ′ ) =P (n, p′ , v ′ )P (S|AP, v ′ )Bias(p′ )P (n)P (n, p′ , v ′ ) is the co-occurrence probability between the verb v ′ and n marked by
the postposition p′ . As for P (S|AP, v ′ ), S denotes a set of ﬁrst-person pronouns,
and 17 pronouns were used. AP is a set of postpositions which can mark agentroles, and “ga” and “ha” were used. P (S|AP, v ′ ) is the probability that theﬁrst-person pronouns occupy the agent role of v ′ . Bias(p′ ) denotes the bias
concerning the postposition “de.” If p′ is “de,” the bias is 25, otherwise the bias
is 1 by referring to Torisawa’s method [9].

4Evaluation

4.1

Preparation

Construction of Training and Testing Data For our experiment, ten objects were selected from electric products that had appeared 10,000 times ormore in the web corpus [5] and also appear in Japanese WordNet 1.1. We constructed training and testing data from the web corpus using the methods in
Section 3.1. Passages were extracted using 205 as the window size α. We chose
two sets of 100 passages at random from each group of passages extracted byusing the methods A and B, and three sets of 100 passages in decreasing orderof frequency from the groups of passages extracted based on the C, D and E
methods, and judged manually whether they contained know-how information
5

The same window size was used throughout all of the experiments mentioned in thispaper, and it was not well tuned to particular data.

Table 2. Size of development data
# of passages containing know-how # of passages
A B C D E
total
43 44 41 16 34
148
444

or not. As utilization roles, we used top 25 pairs of a proposition and a verb automatically produced by the method described in Section 3.2. In this research,
we assume that the passage is judged to be correct if a given passage contains
know-how information. Note that if a given passage contained only a fragmentof know-how information, the passage was judged as incorrect. The sizes of thedata are shown in Table 1.
Generation of Clue Patterns We constructed the development data in thesame way as training and testing data by extracting candidates for know-how
from the web corpus with the methods described in Section 3.1 and judging them
manually whether they contained know-how information or not. The breakdownof the development data is shown in Table 2.
We manually generated 79 types of patterns by referring to the development
data. Some examples of the patterns are shown in Table 3. The symbols “|”, “+”
and “.∗” in column 2 represent a disjunction, a word boundary and any word
sequences, respectively. Column 3 represents the targets that the patterns areapplied to. S, L, R1, R2 and R3 represent a sentence, the leftmost bunsetsu in a
sentence, the rightmost bunsetsu in a sentence, the rightmost two bunsetsus ina sentence and the rightmost three bunsetsus in a sentence, respectively.
In our experiments, the patterns were generated based on the development
data constructed focusing on the object “hair dryer.” However, we found thatthe patterns tended to depend not on an object but on the types of knowhow information such as procedures and advices according to our preliminary
investigation.
4.2

Settings

In the experiment, the 3-tuples and PT features (clue patterns and part-of-speech
information) were used to train machine learning models. We used the following
two types of 3-tuples:
3Tauto As utilization roles, the top 25 pairs of a postposition and a verb produced by the method described in Section 3.2, which were the same pairs
used for constructing the training and testing data, were used.
3Tman As utilization roles, pairs of a postposition and a verb were manually
selected from among top 100 pairs produced by the method described in
Section 3.2, and were used. The average number of the manually selectedpairs was 25.

Table 3. Clue patterns
No. patterns
target1 mazu|hajimeni|hajime+ha|saisho+ha (ﬁrst, primarily, to begin with) L
2 sore+kara|tsugini|konoato|sonoato|soshite (then, next, secondly)L
3 dekiagari|kansei|shuryo|kanryo (ﬁnish, end, complete, accomplish)
R1
4 verb+hou+ga.*yoi|verb+no+ga.*anshinda (prefer, preferable)S
5 verb+yasui (easier to + verb)
R2
6 kinmotsu|dameda|genkin|kiken (danger, caution, prohibition)S
7 verb+nai+youda.*verb|verb+nu+ni.*verb (never, avoid + verb)S
8 ki+wo+tsukeru|te+wo+nuku+nai|tyuui (see to, warning)
R3
9 wo+taishou|ni+gentei|ni+kagiru (limit, target, restrict)R3
10 hitsuyouda|youi|kakaseru+nai |hissuda (necessary, need, essential, vital) R3
11 yakudatsu|katsuyaku|benrida|kouritsu (useful, helpful, eﬃcient)R2
12 teineida|shintyouda|kinnitsu|shikkari (carefully, advisedly, fastly)
S

Support Vector Machines were used as machine learning models and they weretrained using LibSVM6 . For the experiments, we prepared the following baseline
method:baseline (conventional method) Identify whether a given passage contains
know-how information or not by using the model based only on PT features.
In the next section, the eﬀectiveness of using both an object and its utilization
roles is shown by comparing models with and without the 3-tuples.
4.3

Experimental Results

We split the data for each object in ﬁve and carried out 5-fold cross validation byusing data composed of ten objects. The experimental results are shown in Table
4. In comparison with the model using only PT features (baseline), the models
using both PT and 3T features signiﬁcantly improved in both the precision andthe recall. This indicates that lists of know-how can be eﬃciently acquired byfocusing on an object and its utilization roles. In addition, the model using bothPT and 3Tauto features achieved good results although the rates were a little
lower than ones using both PT and 3Tman features.
In the above experiment, both the training and testing sets contained thesame 10 objects. Thus, the results do not show that the proposed models would
achieve good results for the data containing objects which are not containedin the training set since the patterns and the 3-tuples appearing in know-how
information might depend on an object. Therefore, we carried out 10-fold crossvalidation by training the data containing nine objects out of ten and testing
the data containing the rest (one object). The experimental results are shown in
Table 5. The results show that both the precision and the recall were signiﬁcantly
6

http://www.csie.ntu.edu.tw/∼ cjlin/libsvm/

Table 4. Experimental results by 5-fold cross validation
FeaturePrecision
RecallF-measurePT (baseline) 73.0% (588/805) 51.7% (588/1137)
60.6
PT + 3Tauto 74.6% (660/885) 58.1% (660/1137)
65.3
PT + 3Tman 75.6% (666/881) 58.6% (666/1137)
66.0
Table 5. Experimental results by 10-fold cross validation
FeaturePrecision
RecallF-measurePT (baseline) 71.9% (550/765) 48.4% (550/1137)
57.8
PT + 3Tauto 72.8% (600/824) 52.8% (600/1137)
61.2
PT + 3Tman 73.4% (614/836) 54.0% (614/1137)
62.2

improved by using the 3-tuples. This indicates that using an object and its
utilization roles is useful even though the training set does not contain data
for the target object. The diﬀerences between the proposed methods and thebaseline method are statistically signiﬁcant according to Mcnemar’s test (p <0.01).
The 3-tuples feature of our proposed method is a template or a type offeatures while the patterns used as baseline features are surface strings or tokens.
Therefore, the training data do not have to contain all of the 3-tuples appearing
in the testing data. This indicates that the proposed method works without
creating training data for all objects.
4.4

Discussion

Error Analysis In order to detect the causes of errors, we investigated theresults by 10-fold cross validation with the model that uses both PT and 3Tman .
We investigated 222 passages acquired incorrectly and found following threemain causes:– Passages containing only a fragment of know-how information
There were 93 (41.9%) passages. This is because passages were extracted
inaccurately from the documents. If these passages had been extracted accurately, the passages would have been judged as correct. There were 190
passages containing only a fragment of know-how information in the training
and testing data. It would be important to develop a method for accurately
extracting passages if the acquired passages are separated from documents
and then used. In the case that people refer to the acquired know-how information, it is a matter of no importance that the boundaries of passages areincorrect since they can understand know-how by referring to the know-how
information in conjunction with surrounding contexts.
– Passages containing utilization roles
There were 69 (31.1%) passages. They were either diaries or commercial
articles which contained typical usage of the target object and did not containTable 6. Experimental results using arbitrary pairs and results combining models
FeaturePrec Rec F1
PT (baseline)
71.9% 48.4% 57.8
PT+3Tall
71.5% 49.3% 58.3
*PT+3Tauto
72.8% 52.8% 61.2
*PT+3Tman
73.4% 54.0% 62.2
PT (without the method E) 71.5% 46.0% 56.0

know-how information. This type of errors would be reduced by taking into
account co-occurrences and sequences of the patterns, the target object andits utilization roles.
– Passages containing injunctions
There were 21 (9.5%) passages. We did not target them in this research.
However, it is a type of procedural texts, and it could be a type of knowhow.
We investigated 523 passages that are know-how but could not be acquired.
Objects besides the target object were used in 450 (86.0%) passages. That is tosay, these know-how information might be acquired by targeting other objects.
Furthermore, we expect that the recall will be more improved by simultaneously
considering two or more objects and their utilization roles.
Eﬀect of an Object and Its Utilization Roles In order to show the contribution of our 3-tuples compared with that of arbitrary 3-tuples to acquisition of
know-how information, we carried out 10-fold cross validation using the following
feature:3Tall The total number of the frequency of the 3-tuples ⟨o, p, v⟩, where o is atarget object and ⟨p, v⟩ is the pairs of any postposition and any verb, wasused.
In addition, we carried out experiments by using only conventional methods inboth extraction of know-how candidates and identiﬁcation of know-how. Thatis, know-how candidates were extracted by using the method A through D in
Section 3.1 (without the method E considering 3-tuples) and know-how information was identiﬁed by using the model based on PT features. Experimental
results are shown in Table 6. The models attached with * in Table 6 are statistically signiﬁcant than the baseline method according to Mcnemar’s test (p< 0.01). The model based on PT and 3Tall features is not statistically signiﬁcant than the baseline method. Moreover, the models attached with * in Table
6 are statistically signiﬁcant than the model based on PT and 3Tall according
to Mcnemar’s test (p < 0.01). These results indicate that the 3-tuples should beselected for eﬃciently acquiring know-how information and one of the promising
selection method is based on an object and its utilization roles. The last row inTable 7. Experimental results of open-domain experiments
FeaturePrecision
PT (baseline) 68.4% (171/250)
PT + 3Tauto 70.5% (172/244)
PT + 3Tman 69.0% (176/255)

Table 6 shows the results obtained by using only conventional methods. The results show that the 3-tuples play important roles in both extraction of know-how
candidates and identiﬁcation of know-how although we can not simply comparethem since the size of the data are diﬀerent.
To our knowledge, this study is the ﬁrst trial to show that an object and its
usage play an important role in acquiring know-how information.
4.5

Open-Domain Tests

We performed experiments on data for diﬀerent domains. For the experiments,
we selected eight objects (curtain, hunger, ladder, lighter, mirror perfume, scissors, stove) which are diﬀerent types of objects from electric products. They wereselected almost randomly except that frequent nouns were preferred to others.
The passages were automatically extracted in the same way as the constructionof the training and testing data and classiﬁed whether they contained know-how
information by the models trained using the training and testing data.
Table 7 shows the precisions for acquiring know-how information. The modelusing automatically acquired 3-tuples improved the precision and the recall (thenumber of lists of successfully acquired know-how information).
Thus, we expect that know-how information can be acquired by focusing on
an object and how it is used without depending on domain.

5

Conclusion

This paper presented a method for acquiring know-how information by focusing
on an object and how it is used. First, we extracted know-how candidates foreach object. Then, by using both an object and its utilization roles, passages
containing know-how information were eﬃciently acquired from them. Various
lists of know-how information could be acquired by expanding the number oftarget objects although they were restricted in our experiments as a ﬁrst trial.
In this paper, we used manually extracted patterns to compare our methodwith those in the previous works. We would like to acquire know-how information
using automatically extracted patterns. In the future, we would like to consider(grammatical) objects of verbs for utilization roles since we expect that they are
more useful information for identifying know-how information. For example, to“monitor a temperature with thermometer” is more important than to “monitor
something with thermometer” in the second example in Figure 1. In addition, wewould like to focus on multiple objects and how they are used since it is often thecase that multiple objects are used in know-how. Know-how information would
be more eﬃciently acquired by simultaneously considering two or more objects
and their utilization roles.
In this study, we assume that know-how information could be mostly coveredby the physical entities of WordNet. However, the coverage should be improved
to get statistically valid data for obtaining utilization roles. We are planning to
generalize nouns not found in WordNet.

References
1. Aouladomar, F.: Towards answering procedural questions. In: Proceedings of the
IJCAI Workshop on Knowledge and Reasoning for Answering Questions. pp. 21–31(2005)
2. Delpech, E., Saint-Dizier, P.: Investigating the structure of procedural texts foranswering how-to questions. In: Proceedings of the 6th International Conferenceon Language Resources and Evaluation. pp. 46–51 (2008)
3. Fontan, L., Saint-Dizier, P.: Analyzing the explanation structure of proceduraltexts: dealing with advice and warnings. In: Proceedings of the 2008 Conference
on Semantics in Text Processing. pp. 115–127 (2008)
4. Hearst, M.A.: TextTiling: Segmenting text into multi-paragraph subtopic passages.
Computational linguistics 23(1), 33–64 (1997)
5. Kawahara, D., Kurohashi, S.: A fully-lexicalized probabilistic model for Japanesesyntactic and case structure analysis. In: Proceedings of the 7th Human Language
Technology Conference of the North American Chapter of the Association for Computational Linguistics. pp. 176–183 (2006)
6. Kobayashi, N., Inui, K., Matsumoto, Y., Tateishi, K., Fukushima, T.: Collecting
evaluative expressions for opinion extraction. In: Proceedings of the 2nd International Joint Conference on Natural Language Processing. pp. 584–589 (2004)
7. Takechi, M., Tokunaga, T., Matsumoto, Y., Tanaka, H.: Feature selection in categorizing procedural expressions. In: Proceedings of the 6th International Workshop
on Information Retrieval with Asian Languages. pp. 49–56 (2003)
8. Tamura, A., Takamura, H., Okumura, M.: Classiﬁcation of multiple-sentence questions. In: Proceedings of the 2nd International Joint Conference on Natural Language Processing. pp. 426–437 (2005)
9. Torisawa, K.: Automatic acquisition of expressions representing preparation andutilization of an object. In: Proceedings of the 5th Recent Advances in Natural
Language Processing. pp. 556–560 (2005)
10. Yin, L., Power, R.: Adapting the naive bayes classiﬁer to rank procedural texts.
Lecture Notes in Computer Science 3936, 179–190 (2006)