Semantic analysis apparatus, semantic analysis method and semantic analysis program

A semantic analysis apparatus includes a data obtaining unit that obtains data in which an item name and an item value belonging to the item name are represented in a predetermined data format; an item value extracting unit that extracts the item value from the data based on the data format; a concept storing unit that stores a concept which is a semantic notion to be attached to the item name and an instance which is specific data of the concept in association with each other; a concept specifying unit that specifies the concept, which is stored in the concept storing unit and which is associated with the instance which at least partially matches with a character string of the extracted item value, as the concept for the item name; and an associating unit that associates the concept with the item name.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2005-283477, filed on Sep. 29, 2005; the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a semantic analysis apparatus, a semantic analysis method, and a semantic analysis program, according to which meaning of a term used in a sentence of natural language is analyzed.

2. Description of the Related Art

Conventionally, researches have been widely carried out on natural language processing, in particular on semantic analysis. In the natural language processing, meaning of words used in sentences of natural language is analyzed. An ultimate goal of such researches is to bring out a computer which understands human language.

For example, according to one semantic class analysis technology, sentences written on a target subject are analyzed based on a morpheme dictionary featuring 280,000 words and properties, i.e., 106 kinds of semantic classes and definitions of co-occurring words, included in the morpheme dictionary (see, for example, “A MultiModal Help System based on Question Answering Technology” Information Processing Society of Japan Technical Report, Digital Document, No. 36, 2004).

Further, according to processing disclosed in Japanese Patent Application Laid-Open No. 2001-325284, for example, only a table is set as an analysis target, and a class is determined based on an instance with the use of ontology. Thus, an attribute name region and an attribute value region are extracted. Another disclosed technique allows for classification of words based on the ontology (see, for example, U.S. Pat. No. 6,487,545).

The conventional techniques as described above, however, are disadvantageous in that they carry out the semantic analysis generally at the relevance ratio as low as 60 to 70%. Furthermore, a rule creation and maintenance thereof are complicated.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, a semantic analysis apparatus includes a data obtaining unit that obtains data in which an item name and an item value belonging to the item name are represented in a predetermined data format; an item value extracting unit that extracts the item value from the data based on the data format; a concept storing unit that stores a concept which is a semantic notion to be attached to the item name and an instance which is specific data of the concept in association with each other; a concept specifying unit that specifies the concept, which is stored in the concept storing unit and which is associated with the instance which at least partially matches with a character string of the extracted item value, as the concept for the item name; and an associating unit that associates the concept with the item name.

According to another aspect of the present invention, a semantic analysis method includes obtaining data in which an item name and an item value belonging to the item name are represented in a predetermined data format; extracting the item value from the data based on the data format of the obtained data; specifying a concept, which is stored in the concept storing unit, and which is associated with an instance which at least partially matches with a character string of the extracted item value, as the concept for the item name, the concept storing unit storing a concept which is a semantic notion to be attached to the item name and an instance which is specific data of the concept in association with each other; and associating the concept with the item name.

According to still another aspect of the present invention, a computer program product has a computer readable medium including programmed instructions for performing a semantic analysis process. The instructions, when executed by a computer, cause the computer to perform obtaining data in which an item name and an item value belonging to the item name are represented in a predetermined data format; extracting the item value from the data based on the data format of the obtained data; specifying a concept, which is stored in the concept storing unit, and which is associated with an instance which at least partially matches with a character string of the extracted item value, as the concept for the item name, the concept storing unit storing a concept which is a semantic notion to be attached to the item name and an instance which is specific data of the concept in association with each other; and associating the concept with the item name.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, an embodiment of a semantic analysis apparatus, a semantic analysis method, and a semantic analysis program according to the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the present invention is not limited to the embodiment.

FIG. 1is a block diagram of a structure of a semantic analysis apparatus10. The semantic analysis apparatus10includes a data obtaining unit100, an item value extracting unit102, a concept specifying unit104, a concept attaching unit106, an ontology database (DB)110, a concept extracting unit120, and an instance adding unit122.

The semantic analysis apparatus10generates data in an extensible Markup Language (XML) in a Resource Description Framework (RDF) format from data represented in a predetermined data format.

The data obtaining unit100obtains data from outside, and converts the obtained data into an internal format to be handled in the semantic analysis apparatus10. The data obtained from outside is subjected to the semantic analysis process. When obtained from outside, the data is semi-structured and represented in a predetermined data format. Specifically, the data has a format in which an item name, a colon “:”, and an item value are described sequentially in this order, as “item name: item value”.

The data obtaining unit100obtains data in a Hyper Text Markup Language (HTML) represented in the data format as described above, or meta data represented in a table format. The meta data to be obtained includes at least one combination of the item name and the item value described in the data format as described above.

When the data is represented in a predetermined data format, such format does not limit a syntax of the data but a semantic structure of the data. Therefore, actual data may be represented in a table format other than the text format. For example, the employed data format may be the table format consisting of two rows. In this case, the item name may appear in the left-side row of the two rows, whereas the item value may appear in the right-side row of the two rows.

The item value extracting unit102extracts an item value from a Web page obtained by the data obtaining unit100. More specifically, the item value extracting unit102extracts the item value by searching for a colon based on the above-described data format and extracting content described immediately after the found colon as the item value.

The ontology DB110stores ontology. Here, the “ontology” means a representation of a target world given as a model expressed by a certain knowledge representation language. The embodiment described herein is based on an assumption that the ontology may be represented by, for example, a Web Ontology Language (OWL) whose standardization is underway by the World Wide Web Consortium (W3C). In other words, the use of more detailed ontology, such as a role in a specific situation, should not be considered. Here, the ontology DB110serves to store the concept (concept storing unit).

Specifically, the ontology DB100stores a plurality of concepts. The “concept” means a semantic notion of the data. The ontology DB110further stores an attribute, which is a piece of information indicating a relation between the concepts. Such inter-concept relation may be classified into: a “part of” relation, i.e., where a concept A is a part of a concept B; a “is a” relation, i.e., where the concept A is one type of the concept B; an “instance of” relation, i.e., where the concept A is an abstraction of the concept B, or the like.

The concept specifying unit104refers to the ontology DB110, and specifies the concept of an item name which corresponds to the item value extracted by the item value extracting unit102. Here, the concept specifying unit104limits a search target, i.e., a range of concepts to be searched for specification based on a predetermined condition. In other words, the concept specifying unit104serves to specify the search target (search target specifying unit).

The concept attaching unit106attaches the concept specified by the concept specifying unit104to the item name. In other words, the concept attaching unit106associates the item name with the concept. Specifically, the concept attaching unit106describes the concept in the XML format of the RDF and outputs the item name together with the attached concept. In other words, the concept attaching unit106serves to attach a concept (concept attaching unit).

The concept extracting unit120extracts a concept from the data obtained by the data obtaining unit100. The instance adding unit122newly adds an item value included in the data obtained by the data obtaining unit to the ontology DB110as an instance based on the concept extracted by the concept extracting unit120.

Here, the instance adding unit122serves to search for a concept in the ontology DB110(concept searching unit), to search for an instance in the ontology DB110(instance searching unit), and to add an instance (instance adding unit).

FIG. 2shows a data structure of a concept correspondence table112stored in the ontology DB110. The concept correspondence table112includes concept (class) fields, instance fields, and attribute fields.

In the concept field, the concept to be attached to the item name is stored. In the instance field, specific data included in a corresponding concept is stored. For example, for the concept “Wine Color”, specific data of the “Wine Color” such as “Red”, “Rose”, and “White” are stored in the instance field. The concept can be regarded as data representing the class of the instance.

In the attribute field, the attribute of the concept is stored. The attribute stored in the attribute field corresponding to the pertinent concept is, in other words, the data indicating a relation between the pertinent concept and another concept. In the attribute field, a specific attribute of each concept is also stored in addition to the attribute indicating the above-described relation such as the “part of” relation and the “is a” relation.

For example, as the attribute to the concept “Wine”, the attribute “has Color” is stored in the attribute field. This attribute is specific to the concept “Wine” and indicates that the concept “Wine” has the concept “Color”. Herein, the concept “Wine” and the attribute “has Color” are in relation of the subject and the predicate. Further, each of the instances “Red”, “Rose”, and “White” associated with the concept “Wine Color” is the object in the above-mentioned relation of “Wine” and “has Color.”

Most of the data to be processed by the semantic analysis apparatus10has a content corresponding to a triplet structure of the subject, the predicate, and the object. In view of such characteristic, data indicating the association of the concept and the attribute is previously stored as described above so as to allow for the analysis of the relation among the subject, the predicate, and the object.

Further, a property can be specified. Here, the “property” means a conceptualization of the predicate represented as the attribute. Specifically, the ontology DB110further stores a conceptualization table114in which the attribute is associated with the property, which is a conceptualization of the pertinent attribute.

FIG. 3shows a data structure of the conceptualization table114stored in the ontology DB110. The conceptualization table114includes the attribute fields and the concept fields.

An attribute stored in the concept correspondence table112may be a predicate when the concept which is associated with this attribute is taken as a subject. Further, such predicate may be treated as a concept via conceptualization. The conceptualization table114is employed for a processing of such conceptualization. In other words, the conceptualization table14stores an attribute which is a predicate, and a concept which can be obtained via conceptualization of the attribute, in association with each other.

For example, the attribute “has Color” is associated with the concept “Wine Color”. Further, the attribute “has Body” is associated with the concept “Wine Body”.

FIG. 4is a flowchart of the semantic analysis process by the semantic analysis apparatus10. First, the data obtaining unit100obtains a Web page (step S100).FIG. 5shows a Web page20which is a target of the semantic analysis process. The Web page20ofFIG. 5includes a description “Style: White Wine”. In this manner, the item name and the item value are associated with each other by the colon “:” in the Web page20.

Returning toFIG. 4, after the Web page20is obtained, the item value extracting unit102extracts the item value from the obtained Web page20based on the data format (step S102). Then, the concept specifying unit104searches the ontology DB110for an instance which matches with the extracted item value (step S104).

In the instance search, the concept specifying unit104compares the extracted item value and each of instances stored in the ontology DB110. An order of comparison of the instances is determined based on the attribute of the concept.

For example, when the concept has the attribute indicating the “is a” relation, the search starts from an upper level concept. When the relation is described as “Concept A is a concept B,” the upper level concept is concept B.

When the concept has the attribute indicating the “part of” relation, for example, the search starts from a concept indicating the whole. When the relation is described as “Concept A is a part of concept B,” the concept indicating the whole is concept B.

When the concept has the attribute indicating the “instance of” relation, for example, the search starts from an abstract concept. When the relation is described as “Concept A is an instance of concept B,” the abstract concept is concept B.

The upper level concept in the “is a” relation, the concept indicating the whole in the “part of” relation, and the abstract concept in the “instance of” relation will be hereinafter referred to as major concepts. On the other hand, the lower level concept in the “is a” relation, the concept indicating a part in the “part of” relation, and the non-abstract concept in the “instance of” relation will be referred to as minor concepts. In other words, the major concept is, in general, a concept which has less instances associated therewith and less equal level concepts than the minor concept.

Thus, the numbers of concepts and instances associated with the major concept are smaller than those associated with the minor concept. Hence, the processing load can be reduced when the search starts from the major concept.

When the concept specifying unit104detects an instance which matches with the extracted item value during the search of the major concepts, the concept specifying unit104refers to the attribute of the property corresponding to the detected instance, and continues instance detection by sequentially searching for a property which can be regarded as the minor concept of the pertinent property. Eventually, the concept specifying unit104specifies a property of a minor concept of the lowest level (step S106).

Thus, the sequential search starting from the major concept down to the minor concept enables a high-speed search. Here, a property which is treated as a minor concept is a further ramified concept. Therefore, such property can be considered to be a property, which is most suitable for the item name. Hence, such a property which is a minor concept of the lowest level is specified in the embodiment.

Once the property is specified, the concept attaching unit106attaches the specified property to the item value (step S108). Thus, the semantic analysis process by the semantic analysis apparatus10ends.

FIG. 6is a diagram for explaining the semantic analysis process.FIG. 6shows an example of the semantic analysis process on the Web page20shown inFIG. 5. In the Web page20, the item value “White Wine” which appears immediately after the colon is extracted as the item value in the step S102. Then, the concept specifying unit104searches the concept correspondence table112for an instance, which includes a character string that matches with a character string in the item value by a predetermined amount. Consequently, the concept specifying unit104detects the instance “White” as a match. Here, the term “match” means a complete match or a partial match.

On the other hand, the concept specifying unit104traces down minor concepts while referring to attributes of the detected instance, and specifies an instance corresponding to the minor concept of the lowest level. Here, the concept specifying unit104also searches for a concept which is a conceptualization of the attribute while referring to the conceptualization table114.

Further, a concept which is associated with the instance of the specified minor concept is specified from the concept correspondence table112. In the example here, the concept “Wine Color” is specified. In other words, the concept of the item name “Style” associated with “White Wine” extracted as the item value is specified.

Then, the concept “Wine Color” is attached to the item name “Style”. Specifically, as a result of the process, the description of the Web page20becomes:

Then, the data is output to outside.

In such a manner, when a semi-structured data is the process target, a semantic notion thereof may be judged more precisely with the use of a structured portion thereof, i.e., the colon “:” in this embodiment.

The use of the above described embodiment, however, may be limited since the employable data format is limited compared with an apparatus that employs a free sentence. However, most of bar codes and meta data referred to in Radio Frequency Identification (RFID), which is expected to become more popular, are essentially represented in such data format, in other words, a format in a binary relation in which the data itself is the subject. Hence, practical versatility of the above described apparatus is high.

FIG. 7is a flowchart of the semantic analysis process performed when the Web page20includes meta data indicating the concept of the item name. The data obtaining unit100obtains the Web page20(step S100). Then, the concept extracting unit120searches for the meta data indicating the concept of each of the item names included in the Web page20obtained by the data obtaining unit100. When the meta data of the concept exists (Yes in step S130), the item value corresponding to the item name, the concept of which is specified by the meta data, is specified (step S132). On the other hand, when the meta data of the concept does not exist (No in step S130), the process proceeds to the item value extraction described inFIG. 4(step S102).

After the item value is specified in the step S132, the instance adding unit122searches the ontology DB110for the instance which matches with the specified item value. Here, the search targets are only the instances which are associated with the concept provided as the meta data.

When the instance which matches with the item value is not detected from the ontology DB110, in other words, when an instance corresponding to the item value extracted from the Web page20is not registered in the ontology DB110(No in step S134), the item value is newly registered in the concept correspondence table112as the instance (step S136). Specifically, the item value is registered as the instance which corresponds to the concept of the item name associated with the item value. On the other hand, when the instance which matches with the item value is detected from the ontology DB110(step S134), the process ends.

FIG. 8shows an example of the semantic analysis process ofFIG. 7performed on the Web page20ofFIG. 5. Suppose that the meta data is attached to the Web page20. The meta data is the data indicating that the concept of the item name “Style” described in the Web page20is the concept “Wine Color”.

In this case, the concept that matches with the concept “Wine Color” of the item name “Style” is specified from the ontology DB110based on the meta data. Then the concept “Wine Color” in the concept correspondence table112is specified.

Further, the item value “White Wine” associated with the item name “Style” is searched from among the instances associated with the concept “Wine Color”. Since the item name “White Wine” is not included in the instances, the instance “White Wine” is registered as a new instance for the concept “Wine Color”.

When the instance, which is not registered in the ontology DB110, is specified in the semantic analysis process of the Web page20as described above, the instance is newly registered in the ontology DB110, so that the ontology stored in the ontology DB110is expanded and the newly registered instance can be used in a subsequent semantic analysis.

FIG. 9is a flowchart of the semantic analysis process performed when the concept of the Web page is already known. InFIG. 8, the concept for the item name is attached as the meta data, whereas inFIG. 9the concept of the Web page20itself is already known. In other words, in the case shown inFIG. 9, the concept which is the subject of the meta data, i.e., the concept of the Web page20is already known.

The concept of the Web page20is already known, when, for example, the Web page20is known to correspond to a predetermined concept in advance from a Uniform Resource Locator (URL) of the Web page20. Here, it is assumed that the information on the correspondence between the URL and the concept is previously registered in the ontology DB110of the semantic analysis apparatus10. In other words, the ontology DB110serves to store the concept of the obtained data (obtained data concept storing unit).

Alternatively, the concept for the item name may be described in the meta data of the Web page20. In this case, the concept described in the meta data is extracted.

When the concept in the Web page20obtained by the data obtaining unit100is already known (Yes in step S150), the attribute associated with the known concept in the concept correspondence table112and the concept associated with the attribute in the conceptualization table114are looked up to, and the concept having a certain relation with this concept is specified (step S152).

More specifically, only the concept which corresponds to the minor concept of the lower level than the pertinent concept is specified. Specifically, when the “is a” relation with other concept is stored as the attribute of the concept, a subordinate concept thereof is specified. Further, when the “part of” relation with other concept is stored as the attribute of the concept, a concept indicating the part is specified. Further, when the “instance of” relation is stored, a non-abstract concept is specified.

Then, in the ontology DB110, the detection target in the instance detection, which is the detection of the instance that matches with the item value extracted from the Web page20, is limited to the concepts specified in the step S152(step S154). When the concept of the Web page is known in advance as in the case described above, accuracy of the semantic analysis can be improved since the concept to be searched is limited based on this concept.

Following a search target narrowing process (step S154), the semantic analysis process is performed on the item name already included in the Web page20, and once the concept is specified (Yes in step S156), the concept having a certain relation with the specified concept is specified, and the detection target is narrowed down to the specified concepts (step S158). On the other hand, when the semantic analysis process for the Web page20is not yet performed (No in step S156), the process proceeds to the item extracting process (step S102).

In this manner, if the concept for another item name is known in advance by the semantic analysis process already performed on the same Web page, accuracy of the semantic analysis can be improved by limiting the concept to be searched in the concept search for the item name, to which the semantic analysis is to be performed based on this concept.

FIG. 10shows an example of the process of steps S150to S154described with reference toFIG. 9, on the Web page20shown inFIG. 5. Suppose that it is known that the Web page20is the page about wine from the URL of the Web page20. In this case, only the minor concepts of the concept “Wine” among the concepts stored in the ontology DB110are the search targets.

Suppose that in the ontology DB110, the instance, which matches with the item value “White Wine,” is included both in the instances corresponding to the concept “Wine Color” and in the instances corresponding to the concept “Body Color”.

Since narrowing of the search target concept described with reference toFIG. 9is performed, the concept “Body Color” is not included in the search targets. Therefore, only the concept “Wine Color” is specified from the ontology DB110.

By specifying the search target in this manner, it is able to avoid erroneously specifying the concept “Body Color”.

FIGS. 11 and 12show examples of a process of steps S156and S158described with reference toFIG. 9performed on the Web page20shown inFIG. 5. For example, suppose that the instance which matches with the item value “◯◯◯” extracted from the description “Cellar: ◯◯◯” of the Web page20is detected in the ontology DB110, and the concept, which corresponds to the item name “Cellar,” is specified as the concept “Winery”, as shown inFIG. 11.

Further, suppose that the instance which matches with the item value “xxx” extracted from the description “Area: xxx” of the Web page20is detected in the ontology DB110, and the concept, which corresponds to the item name “Area,” is specified as the concept “Region”.

In this case, from the concepts “Winery” and “Region” specified for the same Web page20, the concept, which corresponds to the major concept thereof, is supposed. In this case, the concept of the Web page20is supposed to be the concept “Wine”. The concepts to be searched can be narrowed down according to the supposed concept. In other words, only the minor concepts of the concept “Wine” will be searched.

Specifically, after the concept of the Web page20is specified to be the concept “Wine,” the concept for the item name in the description “Varietal: Chardonnay” included in the Web page20is specified, as shown inFIG. 12.

As the concept whose instance is the item value “Chardonnay” that corresponds to the item name “Varietal”, the concepts “Wine Grape” and “Manufacturer” are specified. However, since the concept of the Web page20is supposed to be the concept “Wine” as described above, only the minor concepts of the concept “Wine” are searched.

Therefore, only the instances, which correspond to the concept “Wine Grape,” are detected. Further, the concept “Wine Grape” is specified as the concept of the item name “Varietal”. As described above, it is able to avoid performing a wrong semantic analysis by specifying the search target.

Such a mode of use is preferable when a plurality of item names including the item name to which the semantic analysis is difficult to be performed is included in the Web page20. In this case, the semantic analysis is first performed on an item name, for which the concept is easily specified, among the plurality of item names. Then, the concept of the Web page20is supposed based on the specified concept. Thereafter, the semantic analysis to other item names included in the Web page20is performed with the minor concepts of the supposed concept as the search target. Thereby the search target is limited, and the semantic analysis can be performed to the item name, for which the semantic analysis is difficult to be performed, with higher accuracy.

Specifically, suppose that a type of engine, a size of tire, and the like are specified as the properties by the semantic analysis of the item name in the Web page20. In this case, it is supposed that the Web page20is about a car, and, the Web page20is assumed to have the “car” as the property thereof.

Further, as shown inFIG. 11, the concept already specified in the semantic analysis of the item name in the same Web page20can be excluded from the search target. Thereby, the semantic analysis can be performed with higher accuracy.

In the above-described specific example, the type of engine and the size of tire are already specified. Hence, in a subsequent semantic analysis, the minor concepts which correspond to the property of the car, and which is the property other than the type of engine and the size of tire, are selected as the search target.

FIG. 13shows a hardware structure of the semantic analysis apparatus10. The semantic analysis apparatus10includes, as a hardware structure, a Read Only Memory (ROM)52in which a semantic analysis program for executing the semantic analysis process in the semantic analysis apparatus10or the like is stored, a Central Processing Unit (CPU)51which controls each of the units of the semantic analysis apparatus10according to the program in the ROM52, a Random Access Memory (RAM)53which stores various data required for the control of the semantic analysis apparatus10, a communication Interface (I/F)57which is connected to a network for communication, and a bus62which connects the respective units.

The above-mentioned semantic analysis program in the semantic analysis apparatus10may be recorded in a computer readable recording medium, such as a Compact Disc Read Only Memory (CD-ROM), a Floppy (registered trademark) Disc (FD), a Digital Versatile Disk (DVD) or the like, as an installable or an executable file, and provided.

In this case, in the semantic analysis apparatus10, the semantic analysis program is read out from the above-mentioned recording medium and executed to be loaded to a main memory, so that each of the units described in the above software structure is generated on the main memory.

Further, the semantic analysis program of the embodiment may be configured to be stored in a computer connected to a network such as the Internet, and to be provided by being downloaded via the network.