Semantic system for integrating software components

A system and method for integrating databases and/or web services into a searchable ontological structure. The structure allows for free-form searching of the combined system and for the discovery of an execution path through the ontology. The discovered execution path (or paths) provides for the generation of code that integrate databases and services for the purpose of fusing information from disparate databases and Web services.

FIELD OF THE INVENTION

The present application relates to a system for integrating software components using a semantic ontology management system.

BACKGROUND OF THE INVENTION

Web service standards are enjoying widespread adoption in corporations across many industries. Corporations are recognizing the value of making it easier for other applications to consume their data by using web standards such as the Hyper Text Transfer Protocol (HTTP), web addressing, and Extensible Markup Language (XML). Using these standards, software clients written in one programming language can access and retrieve information from a server, irrespective of the technology (e.g., hardware, operating system, and programming language) that the server uses.

However, even with the adoption of these web standards, problems remain. For example, although XML is mature as a syntax for web data exchange, current XML technologies do not supply the capabilities provided by more mature technologies like relational database systems. Also, while solutions that aid in web service discovery (e.g., Universal Description, Discovery and Integration (UDDI), as described at uddi.org/specification.html, incorporated by reference herein) and invocation (e.g., Web Services Description Language (WSDL), incorporated by reference herein) are emerging, they are far from mature. Similarly, technologies that reason with web service description files for the purpose of chaining web services are not available. It is left to the programmer to determine, at design time, which web services to invoke, the order in which they need to be invoked, and the formatting of information necessary to complete an operation. As a result, the programmer writes much of the “glue code” necessary for a particular computation. A need still exists for methods and systems that allow automatic generation of such “glue code,” as well as automatic discovery and integration of available web services and data repositories.

SUMMARY OF THE INVENTION

In one aspect, the invention is a method of generating executable code for linking data in a structured data repository to structured inputs and outputs from a source web service. According to the method, a first domain ontology encoding the structured data repository and a second domain ontology encoding the structured inputs and outputs of the source web service (and optionally also access parameters for the web service) are provided, and are linked to form a merged ontology. Data from the structured data repository and the web service are mapped into the merged ontology to create an expanded ontology. One or more desired input(s) and output(s) are specified for a linked web service, and the expanded ontology is searched for an execution path between the desired input(s) and output(s), which may include at least one concept from the first ontology and one concept from the second ontology. Executable code is then generated that executes steps of the execution path in order to produce the desired outputs. The expanded ontology may be created either by mapping data from the structured data repository and structured inputs and outputs from the web service into the merged ontology, or into the first domain ontology and second domain ontology before merging.

In another aspect, the invention is a method of constructing a new web service that provides a selected output type in response to a selected input. The method includes accepting an input item and an output type from a user, and searching an ontology to find one or more input matches for the input item and output matches for the output type. The ontology comprises structured data from one or more structured data repositories and structured input and output information from one or more existing web services. Once match(es) for the input item and output type are found, the ontology is searched to find execution path(s) that link input match(es) and output match(es). (If multiple execution paths are found, a user may be permitted to select a desired path, and if multiple input and/or output matches are found, a user may also be permitted to select among the matches found). The execution path is then used to generate executable code for the new web service, which allows a user to provide input of a type corresponding to the accepted input item, and provides an output of the accepted output type. The execution path may include at least one concept from a structured data repository and at least one concept from structured input and output information from an existing web service.

In yet another aspect, the invention is a method of selecting and accessing one or more web services, wherein each web service has a set of one or more structured input(s) and output(s). The method includes providing a domain ontology that includes mappings to the structured input(s) and output(s) (and optionally access parameters for a web service in the ontology), specifying one or more desired inputs and outputs, and searching the domain ontology to match the one or more desired inputs and outputs. The domain ontology is also searched for an execution path linking the one or more structured inputs to the one or more structured outputs. The web service(s) having mappings on the execution path are then returned to a user. When a plurality of web services is returned, the web services may form a chain in which a structured output of one web service provides a structured input for another web service.

In still another aspect, the invention comprises a method of mapping a web service having a set of one or more structured input(s) and one or more structured output(s) to a domain ontology. Them method includes identifying an input type for each of the structured input(s), searching the ontology for concepts having each input structure type and adding mappings between the structured inputs and the located concepts in the ontology, and searching the ontology for concepts having each output structure type and adding mappings between the structured outputs and the located concepts in the ontology. A concept is then added to the ontology representing the web service and adding mappings between the added concept, the structured inputs, and the structured outputs. In addition, access parameters for the web service may be added. The ontology may also include concepts and relationships derived from one or more structured data repositories.

In a further aspect, the invention is a software tool for accessing a structured data repository and a source web service having structured inputs and outputs. The tool includes a searchable ontology structure, an interface for a user to specify one or more input items and one or more output types, a search module that searches the searchable ontology for an execution path linking the specified input item(s) and output type(s), and a query module that traverses the execution path to provide output of the output type(s) that is linked to the input item(s) in the ontology. The searchable ontology structure includes concepts and relationships from a selected knowledge domain, concepts and relationships of the structured data repository, and concepts and relationships of the structured input(s) and output(s) of the web service. The concepts and relationships of the structured data repository and of the web service are linked to the concepts and relationships from the selected knowledge domain, and the execution path may include concepts from both the structured data repository and the web service. The tool may also include a code generation module that generated executable code that carries out the execution path, and/or a selection module that allows a user to select among a plurality of execution paths.

DETAILED DESCRIPTION OF CERTAIN PREFERRED EMBODIMENTS

Definitions

As used herein, a “web service” is an application program (for example, a program implemented in Java or PHP on the World Wide Web) that accepts defined input(s) and returns defined output(s), and that exposes its interface, for example using Web Services Definition Language (WSDL). Example web services include the map servers found at MapQuest™ and Yahoo! Maps™, online telephone directories such as Switchboard.com™, and online weather services such as Weather.com™ and the National Weather Service site. While the web services discussed in the specification are available on the World Wide Web, the term “web service” is also intended to include services accessible only on internal intranets (such as an employee “facebook” or directory) or on standalone computers (such as an XML front end to a database or application program).

As used herein, a “database” or “structured data repository” is a collection of data having a formalism (i.e., structured data). Databases may be organized into “tables” of “records” having “fields” as is conventional in the art (see, e.g.,Webster's New World Dictionary of Computer Terms,4thed., Prentice Hall, New York, 1992), or may be more loosely structured (e.g., a structured text file or a tagged file format document).

As used herein, a “triple” or “RDF triple” is a statement that conveys information about a resource, and that can be represented as a subject, a predicate, and an object. As depicted herein, a triple is graphically represented as shown inFIG. 1, wherein subject and object nodes are connected by a directional line representing the predicate. An example statement that could be represented as a triple might be “the bookMoby Dick(subject) has an author (predicate) whose value is Herman Melville (object).” Subject, predicate, and object can all be identified by uniform resource identifiers (URIs), such as a uniform resource locator (URL). Predicates are generally properties of resources, while subjects and objects may be described as “concepts.”

As used herein, an “ontology graph” or simply a “graph” is a collection of related triples that together convey information about the relationships between the concepts represented by the nodes of the graph (the set of objects and subjects of the statements represented by the triples).

As used herein, an “ontology” or “domain ontology” is a dictionary of terms within a given domain of knowledge, formulated in a known syntax and with commonly accepted definitions, including relationships between the terms (e.g., in the form of triples or an ontology graph).

Semantic Web Technologies

Today, Semantic Web technologies are beginning to emerge with promises of enabling a much faster integration of applications and data (see, e.g., Gruber, “A Translation Approach to Portable Ontologies,”J. Knowledge Acquisition,5(2):199-200, 1993; Guarino, et al., “Ontologies and Knowledge Bases: Towards a Terminological Clarification,” inTowards Very Large Knowledge Bases: Knowledge Building and Knowledge Sharing, N. Mars, ed., IOS Press, Amsterdam, 1995; Berners-Lee, et al., “The Semantic Web,”Scientific American, May 2001; and Daconta, et al.,The Semantic Web: A Guide to the Future of XML, Web Services, and Knowledge Management, Wiley US, July 2003, all of which are incorporated by reference herein). However, for that to happen, Semantic Web technologies must facilitate access to large amounts of data with minimal programmer intervention. Web services must be discovered, chained, and invoked automatically, thus relieving the programmer from having to do these steps. Semantic Web standards provide a rich framework for the precise description of data and applications, thereby enabling greater automation in this end-to-end web service execution process. The World Wide Web Consortium (W3C) proposed recommendation for a standard web ontology language (OWL), (incorporated by reference herein), builds on web technologies including XML's ability to define customized tagging schemes and RDF's flexible approach to representing data.

Convergence between web services and Semantic Web technologies is beginning as illustrated by the OWL-Services (OWL-S) effort which is incorporated herein by reference). OWL-S is an effort to develop a web service ontology that could be used to describe the properties and capabilities of web services in unambiguous, computer-interpretable form.

According to the invention, such Semantic Web technologies can be used to discover execution paths spanning the retrieval of data from structured data repositories and execution of web services, to automate the integration of data repositories and web services, and to automatically generate the “glue code” needed to achieve the integration. The example described below shows how a marriage of web services and Semantic Web technologies can further automate this integration process. OWL-S represents a partial solution that can be used according to the invention to orchestrate web services. The example also illustrates how databases can be mapped into ontologies according to the invention, thus making their data available to web services.

An information system will need the capability to integrate new and existing services, e.g., application programs or web services, and incorporate additional information resources, e.g., relational databases or XML documents. In most cases application programs and web services require access to information contained within a data source. Very often the challenge for IS systems is to integrate existing data sources with application programs or web services. The use of IS ontologies and an ontology management system according to the invention can enable this type of integration by generating the integration code automatically.

The existing OWL-S web service ontology model provides features for invoking and accessing web services. However OWL-S does not address the need for web service integration and interaction with an existing database without first building a web service that abstracts the database. The invention provides a methodology to enable the integration of new web services with existing databases.

By representing databases and web services in ontology space and by linking them to the domain ontology, we can now integrate with existing databases, and other web services, without developing additional code. Using IS ontologies in this way not only results in code being generated, but also eliminates the need for creating a web service ontology with composite processes comprised of control flow constructs as defined in OWL-S.

EXAMPLES

The invention is described below with reference to particular examples such as integrating a database with a web service, and “chaining” multiple web services. However, those of ordinary skill in the art will readily see that the inventive techniques may be used in a variety of ways, including rapidly integrating pluralities of legacy databases and/or web services into new systems without a need for extensive programming, and selecting appropriate web services from within an ontology without a need for user familiarity with the available web services. Further, the invention may be used not only with an “ontology viewer” to process individual user queries, but may also be used to construct custom Information System (IS):web/database services that access underlying databases and web services as needed to process queries.

The following example shows how an address database and a map-finding web service (e.g., the map services available at MapQuest™ and Yahoo!™) may be integrated according to the invention, by performing the following steps:Provide a domain ontologyCreate and link a database component ontology to the domain ontologyCreate and link a web service component ontology to the domain ontologyBroker a user request to suggest executable pathsManually view the result of implementing one of the executable paths through the augmented ontology or automatically generate a web service to do so

The domain ontology, D, includes the RDF classes Business, Customer, Name, Location, Street, CityOrTown, State, and PostalZIP. D is represented as a uniform structure of triples, {subject, relationship, object}, as shown in Table 1. This ontology may also be represented graphically, as shown inFIG. 2. (Note that the domain ontology may include other classes and relationships; for simplicity, only seven triples from the ontology are shown).

TABLE 1D::{Business, sellsTo, Customer}D::{Customer, doesBusinessAs, Name}D::{Customer, residesAt, Location}D::{Location, hasA, Street}D::{Location, hasA, CityOrTown}D::{Location, hasA, State}D::{Location, hasA, PostalZIP}
sellsTo, doesBusinessAs, and residesAt are OWL object properties, and hasA is an OWL datatype property with domain rdf:Class and range datatype string. As shown, this domain ontology is manually created, typically by experts in the business domain to which it pertains. It may be possible to automatically create useful domain ontologies using artificial intelligence techniques or other methods; such ontologies may also be used according to the invention.

A database ontology R is then constructed for linking to the domain ontology D to form an augmented ontology D+. R is the conjunction of a database upper ontology RUspecifying the structure, algebra, and constraints of the database, and a database lower ontology RLincluding the data as instances of RU, as follows.

The upper ontology RUspecifies the structure, algebra and constraints of any relational databases in RDF/OWL triples. We define the RDF classes Database, Relation, Attribute, PrimaryKey and ForeignKey. A portion of RUis given in Table 2:

Consider a database having a table ADDRESS as depicted in Table 3. (For brevity, only two of the data records are shown). The relation ADDRESS has Address_ID as the primary key, and Name, Street, City, State, and Zip as attributes. The portion of RLcorresponding to this table may then be constructed (in part) as shown in Table 4.

R is then the conjunction of RUand RLas partially shown inFIG. 3(Note that the fields “City” and “State” have not been shown in R for the sake of brevity, but are linked to the concepts “CityOrTown” and “State” in D in a manner analogous to that shown for “Street” and “Zip” below). If there are entity relationships not captured in the R, these may be inserted manually at this stage.

The concepts in database ontology R are then mapped to the concepts in the domain ontology D to create an augmented domain ontology D+. In this example, this is done using the relationship hasSource, as shown in Table 5. (Note that the linked concepts need not have identical names, as in the mapping between D::PostalZIP and R::Zip).

The map-finding web service is then mapped to an upper ontology WUthat models web services as concepts of Inputs (in Parameters), Output, Classification Conditions, and Effects, as shown inFIG. 4. An instance of this ontology, WI, is created for the map-generation web service of the example. This ontology is then mapped to D+to form augmented ontology D++, for example using the relationship isInputTo, e.g. D++:: {D+::Location, is InputTo, WI::MapQuest}. A portion of D++, showing a link between WIand D+, is shown inFIG. 5.

For automatic code generation, in addition to the ontological representation of the inputs and outputs of the web service, access parameters for the web service may be needed. A generic upper ontology, VU, that models how to access and invoke a web service is shown inFIG. 6. VUalso preserves the structure of any document consumed or produced by the web service. An instance of this ontology, WM, is created to describe the parameters for invoking the web service. As shown inFIG. 7, WMshows parameters for invoking MapQuest™. (For clarity in the drawing, the URL for MapQuest™, is represented as [URL Val]). WMis then mapped into D++, for example using the relationships isValueOf, isServedAt, and hasResult. The isValueOf links the range or object value of the hasLabel relationship in the ontology WMto concepts in the augmented domain ontology D++. The isServedAt relationship links the subject or domain of the hasOutput relationships in the D++ontology to the object of the hasUrl relationship in the WM. The hasResult relationship links the range of hasLabel relationship in the WMto the range of hasOutput relationship in the ontology D++. This relationship is useful when the output of the web service contains the inputs of another, as further discussed below. A portion of D++including WMand WIis shown inFIG. 8, illustrating the links between the web service ontologies and the domain ontology.

Once the ontology D++has been created, a single application (hereinafter, the “Semantic Viewer”) can broker a variety of user requests to return data outputs and/or executable glue code. As implemented in this example, the Semantic Viewer is a web-enabled application. When a user enters any input data, it is linked to concepts in the augmented domain ontology using the userInput, userOutput, and userCriteria relationships. The userOutput links a goal the user is interested in a concept in the augmented domain ontology. The userInput links the user's input to the object value of the in Parameter relationship that are not input to web services in the augmented domain ontology. The userCriteria is used to link user's input to concepts in the augmented domain ontology.

For example, suppose a user provides “255 North Road” and “01824” as inputs, and “map” as a requested output. The Semantic Viewer searches D++for matches to the input values, and locates them in RLas a Street and a Zipcode, respectively. In addition, it searches for “map” and locates it as a concept in WI. It then locates a path through the ontology graph linking the inputs with the output, as shown inFIG. 9. Finally, it generates executable glue code to actually invoke the web service discovered (in this case, MapQuest™) and return a map of the requested address.

The above example user inputs are the same as what would be required if the user were simply to visit MapQuest™ and request the map directly, although the user does not have to know how to access MapQuest™ in order to use the Semantic Viewer as described in this example. If multiple map services were available, the Semantic Viewer would present the user with multiple execution paths, allowing access to whichever map service was desired, again without requiring the user to know the URL or data formatting requirements of each service.

The additional power of the Semantic Viewer can be seen if the user instead enters “MITRE” as an input, and “map” as a requested output. No available web service in D++takes a corporate name and returns a map. However, the Semantic Viewer still locates “MITRE” in the database as an instance of Business_Name, and discovers a path through the ontology graph linking it to the Map output of MapQuest™, as shown inFIG. 10. Thus, the execution path returned now includes a database query to discover the street address of the MITRE Corporation, formats that data for the MapQuest™ map service, and returns a map of the company location.

In practice, the Semantic Viewer may find multiple execution paths for a single query. In this case, the user may be presented with the multiple execution paths and allowed to select the desired path. For example, in the case where a user enters a single input of “01730” and a desired output of “map,” there are two possible connecting paths through the D++described above. According to one path, illustrated inFIG. 11a, the Semantic Viewer recognizes “01730” as a zip code, and passes it to MapQuest™ without a street address, resulting in a map of the general area around that zip code (the recognition of “01730” as a zip code according to this path may be through its existence in the database, but it is also within the scope of the invention to manually indicate that 01730 is a zip code in order for the Semantic Viewer to discover an execution path). However, there also exists a path, illustrated inFIG. 11b, in which the Semantic Viewer finds each instance of “01730” in the database, and passes each Street Address/Zipcode combination (for each listed business having that zip code) to MapQuest™, obtaining maps for every business in the selected zip code area.

In the above examples, a single output of a single web service has been the desired output. However, multiple outputs may also be requested, and these outputs may not all be derived from the same web service. For example, suppose a user enters a single input of “01730” and desired outputs of “map” and “business name.” In this case, an execution path similar to the second path described in the previous paragraph exists. The Semantic Viewer recognizes 01730 as a zip code, and queries the database to find all of the businesses in that zip code, requesting both the business name and the street address. The street addresses and the selected zip code are passed to MapQuest™, and the resulting maps are returned along with the business names, for each business in the database that matches the criteria.

The Semantic Viewer may also “chain” web services as necessary to obtain a desired result. For example, suppose the domain ontology also contains the relationships hasVoiceNumber and hasFaxNumber, and has been further augmented to include a reverse lookup telephone service (such as that found at Switchboard™), which accepts an input “telephone number” and provides a listing name and address. In this case, when a user enters “781-271-2000” as input and “map” as output, one of the returned execution paths will include taking the telephone number from the database listing, passing it to the reverse lookup service to obtain a street address, and passing the street address to MapQuest™ to obtain the requested map.

Similarly, web services may operate in parallel on the same or related inputs to provide multiple outputs. For example, web services that provide maps (as discussed above) and aerial photos (such as that found at TerraServer-USA) may both be called with the same address information, if an address is input and both “map” and “photo” are selected as outputs.

For each of the above-described examples, the Semantic Viewer has located an execution path, and then performed the necessary steps to answer a user query. However, the execution path may also be used to generate glue code necessary to create a new service of the type requested. For example, in the case described above in which a user provided the input “MITRE” and the output “map,” in addition to simply providing a map of the MITRE location, the Semantic Viewer can also return executable “glue code” for a service that accepts a company name, looks it up in the database to find a street address, passes the street address to MapQuest™, and returns the requested map. This static code can then be directly used to create a new standalone web service which is independent of the Semantic Viewer. A flow chart of the resulting web service, including exemplary pseudocode describing the construction of a SQL database query and a web service URL, is shown inFIG. 12. Of course, the code generated will depend on the specific requirements of the database and web service. Further the database type and query syntax may be represented in ontological form and linked to the R in an analogous way to the construction of WMfor accessing the web service, as discussed above.