UDS : A Universal Data Structure Robert Levinson UCSC-CRL-94-15 June 10 , 1994 Board of Studies in Computer and Information Sciences University of California , Santa Cruz Santa Cruz , CA 95064 -LRB-408-RRB-459-2087 FAX : -LRB-408-RRB-459-4829 E-mail : levinson@cse.ucsc.edu abstract This paper gives a data structure -LRB- UDS -RRB- for supporting database retrieval , inference and machine learning that attempts to unify and extend previous work in relational databases , semantic networks , conceptual graphs , RETE , neural networks and case-based reasoning . Foundational to this view is that all data can be viewed as a primitive set of objects and mathematical relations -LRB- as sets of tuples -RRB- over those objects . The data is stored in three partially-ordered hierarchies : a node hierarchy , a relation hierarchy , and a conceptual graphs hierarchy . All three hierarchies can be stored as \ levels '' in the conceptual graphs hierarchy . These multiple hierarchies support multiple views of the data with advantages over any of the individual methods . In particular , conceptual graphs are stored in a relation-based compact form that facilitates matching . UDS is currently being implemented in the Peirce conceptual graphs workbench and is being used as a domain-independent monitor for state-space search domains at a level that is faster than previous implementations designed specifically for those domains.In addition it provides a useful environment for pattern-based machine learning . I am for the delicate dance from parts to wholes and back again . We should not be captured at either end . The dance should go forever . | P. Suppes -LSB- 21 -RSB- 1 Introduction Three popular and competing database paradigms are relational databases , in which data is stored and accessed as a set of tables , conceptual graphs , in which data is stored as a set of graphs , or semantic networks in which data is stored in one large homogeneous network -LRB- graph -RRB- . In this paper we give a data structure that can be seen to replace the others , with all the advantages of each and fewer of the disadvantages . The basic idea will be to store the data as they traditionally are stored in a semantic network as a set of nodes with edges or hyperedges representing n-ary relations . 1 . Each node corresponds to a primitive domain object or value . To store the semantic network one requires a node table that gives for each node all the relation instances -LRB- edges -RRB- it is in and its position in each relation . A relation table stores for each relation instance , pointers to the nodes involved in that instance . One might then think of representing a relational database simply by giving the relation table . What is missing however are the relational schema : the type headings on individual relations in a relational database . Such schema can be created by making the node table and the relation table each into a partially-ordered hierarchy by more-general-than . Once this is done it is seen that the node table corresponds to the traditional type hierarchy used in conceptual graph theory and the more-generalthan relation over the relational hierarchy forms the traditional tables of relational databases : each relational table sits directly under its schema definition . The next key idea is that a conceptual graph -LRB- CG -RRB- can be viewed as a sequence of joins on a set of relations 2 . Thus , a conceptual graphs database can be stored simply as a partial order over ordered lists of relation instances from the relation hierarchy . Thus , the resulting data structure has three hierarchies : ffl The node hierarchy ffl The relation hierarchy ffl The conceptual graphs hierarchy Each of these hierarchies can be stored as one : the node hierarchy and relation hierarchy would sit at the top of the standard conceptual graph hierarchy . However , the distinction between the three levels needs to be retained to allow for the multiple access mechanisms to be supported . Figure 1 below gives a conceptual graphs database of seven graphs and a query graph from which we will be taking our examples . Figure 2 shows the node hierarchy , relation hierarchy and graph hierarchy corresponding to the UDS representation of Figure 1 . To be noted is the compact representation of the graphs in the graph hierarchy . Such compaction of conceptual structures is a hallmark of this new storage and retrieval method and is explained further below . Although not made explicit in the diagram , there are bidirectional pointers from objects in the conceptual graphs hierarchy to the objects they contain in the relation hierarchy and bidirectional pointers from objects in the relation hierarchy to objects in the the node hierarchy . For example , there are pointers from Sue in the node hierarchy to the first argument of relations 4,7 , and 12 in the relation hierarchy -LRB- and hence the label Sue need not actually be stored with the relations -RRB- . Similarly relation 7 points to G4 in the graph hierarchy . Also not made explicit in the diagram is that siblings in the node hierarchy be sorted alphanumerically . 1A hypergraph is a set of nodes and a set of hyperedges which are subsets of or more nodes . Thus a semantic network can be viewed as a directed-labelled hypergraphand partitioned semantic networks as nested-labelled directed hypergraphs -LSB- 2 -RSB- 2Informally : a CG is a very small relational database ! 2 1 . Introduction manr quickly eat eat quickly manr pie cont apples person Query Graph G7 : G6 : G5 : G4 : G3 : G2 : Database Graphs Sue girlman person Dan Frank Type Hierarchy : person eatG 1 : girl eat Sue agent agent agent Sue eat agent quickly manr apples cont pieob j agent Sue eat manr quickly eat objagen t eatDan poss Sue agentQ : obj pie Figure 1.1 : Type hierarchy , database and query graph . This example is adapted from -LSB- 20 -RSB- pp.92-93 and reprinted from -LSB- 12 -RSB- \ manr '' stands for \ manner '' , \ cont '' stands for \ contains '' and \ poss '' stands for \ possesses '' . G1 and G7 are generalizations of the query Q , G2 and G4 are specializations of Q and G3 , G5 , and G6 are incomparable . Dan Frank man girl pie apples foodp erson Sue G7 f11g G1 f2g G2 f3g G5 f12g G6 f6 ,8,10 g G4 f7g G3 f4 ,6 g Node Hierarchy Relation Hierarchy Graph Hierarchy 1 . eat -LSB- agent , obj , manr -RSB- 2 . eat -LSB- person , , -RSB- 11 . eat -LSB- , , quickly -RSB- 3 . eat -LSB- girl , , quickly -RSB- 4 . eat -LSB- Sue , pie , -RSB- 8 . eat -LSB- Dan , pie , -RSB- 7 . eat -LSB- Sue , pie , quickly -RSB- 5 . cont -LSB- whole , part -RSB- 6 . cont -LSB- pie , apples -RSB- 9 . pos -LSB- owner , owned -RSB- 10 . pos -LSB- Sue , pie -RSB- 12 . eat -LSB- Sue , , -RSB- Figure 1.2 : The hierarchies corresponding to the graphs of Figure 1 4 1 . Introduction Also associated with each hierarchy is an additional data structure that allows subsumption testing in the hierarchy to be done in constant time . The three standard data structures for this purpose are sparse arrays , hash tables and bitcodes -LSB- 4 -RSB- . Which scheme is selected depends on storage costs . This use of multiple hierarchies is reminiscent of the data structure used in multi-level hierarchical retrieval -LSB- 13 -RSB- . The major difference is that the node descriptor hierarchy is being replaced by a relation hierarchy that better facilitates CG matching by incorporating the power of a relational database . UDS also incorporates ideas from the original universal graph method -LSB- 11 -RSB- proposed by us in 1984 and the universal relational model -LSB- 19 , 22 -RSB- . Here we add some new twists that take advantage of a better understanding of the relationship between conceptual graphs , relational databases and marker-passing semantic networks -LSB- 9 , 6 -RSB- . The new design exploits the following principles that we feel are critical in designing cost-effective information retrieval systems : ffl Every primitive data object , label or symbol should be stored only once with pointers used to denote the actual uses of the object . ffl Every compound object should be stored with the minimum information required to represent the combinations of its parts . Thus , for example , transitive relations should be stored by only giving the arcs represented in the transitive kernel , and the designation that the relation is transitive . ffl Given no loss of accuracy , objects should be processed at the highest level of abstraction possible . For example , below we see that conceptual graphs can be processed at the \ relational '' level rather than at the node and edge level . The competing methods may each operate over UDS while deriving benefits that were not there in the original methods : ffl Semantic network marker-passing algorithms can make use of the semantic network formed from the node hierarchy and the relation hierarchy , and the partitioning provided by the CG hierarchy . Advantages over traditional semantic network algorithms are that the relation hierarchy allows multi-argument relations to be treated as individual chunks , and the partial order over the nodes and relations allows generalization queries to be handled more quickly and efficiently . ffl Relational database retrieval operations operate directly on the relation hierarchy by making use of the pointers to and from the node hierarchy to enhance the join process during query processing -LRB- it is easy to determine which objects join with each object -RRB- . The type hierarchy formed by the node hierarchy is a facility most relational databases do n't have . Further , the sorting -LRB- see above -RRB- in the type hierarchy can be used to return answers to queries in a desired sorted order . ffl Conceptual graph operations can operate as before on the graphs represented in the conceptual graphs hierarchy supported by the type information in the node hierarchy . For example , the Method III retrieval scheme -LSB- 11 , 12 -RSB- and associated enhancements -LSB- 13 , 4 -RSB- can operate directly on this hierarchy . Graph matching will be much faster than before due to the compact nature of the graphs -LRB- see below -RRB- . In addition to being able to process the data using the multiple viewpoints , other advantages to conceptual graph processing are accrued : ffl Storage of the conceptual graphs has been greatly reduced since nodes and relations are actually stored only once . Redundancy has been removed . Similar ideas involving storage based on canonical rules are given in -LSB- 3 -RSB- . ffl Conceptual graph matching can make use of the chunks provided by processing the graphs at the relational level rather than at the node level . ffl Method V retrieval can also proceed naturally . Method V is a previously unpublished method that proceeds exactly as in Method III -LRB- where a node is not queried unless its predecessors are known to match the query -RRB- except that information about how the predecessors bind to the query is propagated up the hierarchy . These bindings are reflected as relation-torelation bindings in UDS as opposed to node-to-node bindings in a standard CG system , thus leading to substantial complexity reduction . -LRB- For further details , see Section 5 below . -RRB- ffl The fact that the semantic network represents a join over all the relations is a great advantage : Suppose we have one CG : \ Joe eats popcorn '' and another CG : \ popcorn is white '' . If we never take the join of these two CGs we might have difficulty discovering that \ Joe eats something that is white . '' The semantic network representation facilitates the discovery of relationships that may not have been previously recognized . 2 Retrieval In UDS , queries can be answered using either marker passing , relational processing or conceptual graph retrieval . Which method is best depends on the type of query . For instance in the example of Figures 1 and 2 one could use CG retrieval to find immediate generalizations of the object as G1 and G7 with an immediate specialization as G2 and a further specialization as G4 . G4 gives the conclusion that \ Sue ate quickly '' . Ignoring the CG hierarchy and using just the relation hierarchy , one would check for the position of the relation eat -LSB- person , , quickly -RSB- finding generalizations of 2 and 11 and a specialization of 3 that leads to the specialization of 7 that \ Sue ate quickly . '' Marker passing would start with the node hierarchy at nodes eat , person and quickly , finding all relations involving only these nodes as 2 and 11 . Through spreading activation to more specific instances of \ person '' it would be discovered that both \ a girl '' and Sue eat quickly . In this example , in which the query was a single relation , relation-based retrieval worked best . For queries involving multiple relations and highly structured objects it is likely that CG retrieval would work best , having formed the chunks ahead of time . In particular , conceptual graphs can be used to store complex relational database queries for faster processing in the future , much in the same way as views -LSB- 5 -RSB- do currently . Semantic network marker-passing will work best for \ free form '' queries such as \ In which ways are Joe and Sue related ? '' . 3 Insertion and Deletion The costs of insertion and deletion from the structure have decreased on the average due to the indexing properties of the multiple hierarchies and improved matching costs . If inserting a node , one adds it to the node hierarchy exactly as one would do with the type hierarchy in conceptual graphs . Inserting a relation or a new relational scheme definition into the relation hierarchy likewise proceeds exactly as in relational databases while benefiting from the hierarchical organization . Insertion of a conceptual graph also proceeds exactly as before except that the relations in the conceptual graph must also be inserted into the relation hierarchy . This small additional cost is outweighed by the benefits gained due to more efficient matching in the conceptual graphs hierarchy . 4 Compaction of Conceptual Graphs The standard diagrammatic representation of conceptual graphs -LSB- 20 -RSB- makes these graphs appear to have more complexity than they actually do . If one were to implement a conceptual graph based on the diagrammatic representation the costs associated with storage and matching would be much higher than they need to be . For example , a 3-ary relation in a diagrammatic conceptual graph requires 3 concept nodes , 1 relation node and 3 argument arcs . Taking the view -LSB- 2 -RSB- that conceptual graphs are nested directed hypergraphs , the 3-ary relation becomes simply 3 nodes and 1 labeled hyperedge . This is certainly an improvement . But there is yet farther to go , if one takes the relational view of UDS . A 3-ary relation is a single node in the relation table ! A conceptual graph becomes a set of relation nodes where the relations are those stored in the relational table . In those conceptual graphs in which an individual object appears only once it is enough to assume that all relations are joined to other relations in which one 6 5 . Method V retrieval algorithm or more nodes are shared commonly . For those conceptual graphs in which objects may be repeated and take different slots in the relations , explicit joins must be defined between the relation nodes in which it is stated exactly which arguments get joined to which arguments . So , the final view is that a CG is a set of nodes representing individual relations and a set of labeled edges representing joins between these relations . Figure 3 below illustrates a compaction of a conceptual graph representing a molecule involving 2 oxygen atoms , a carbon atom , 2 nitrogen atoms , a double bond and 3 single bonds . Thus the standard chemical diagram contains 5 nodes -LRB- for atoms -RRB- , and 4 edges -LRB- for bonds -RRB- . From the diagram we see that the standard CG diagrammatic form requires 9 nodes and 8 edges . The hypergraph representation of the CG uses 5 nodes and 4 edges -LRB- as in the chemical graph -RRB- . But the relation-based CG representation has managed to store everything as 4 nodes and 3 edges where the nodes point to relations in the relation hierarchy . The labels on the edges denote which arguments in one relation join to which arguments in the next relation . Note further that for the graphs in Figures 1 and 2 no such join information was required and the graphs were reduced to sets -LRB- of relations -RRB- . This compaction has favorable properties for supporting matching : ffl There are fewer nodes and edges . ffl Since nodes denote more specific objects than before , their likelihood of matching is significantly lessened . This leads to fewer false matches that require backtracking in subsumption tests . ffl Since relation nodes are stored only once , it is easy to find node correspondences between two CGs since if a node is shared they will be pointing to the same node in the relation hierarchy , -LRB- or if one node is more-general-than the other there will be a path -RRB- . That is , the first stage of most subgraph-isomorphism tests -LSB- 1 -RSB- in which one finds which nodes could bind -LRB- as \ candidates '' -RRB- to other nodes can be performed by simply reading off correspondences from the hierarchy , or comparing bit encodings -LSB- 4 -RSB- . The same abstraction mechanism that goes from nodes to relations to graphs can be taken one step further to facilitate the storage and retrieval of nested-context conceptual graphs . A graph will point to nested-graphs in which it serves as a context . The encoding of negated contexts and lines of identity is analogous to the encoding of join-structure when going from relations to graphs . The abstraction mechanism may be continued on to multiple layers of nesting . 4.1 Subsumption preservation in compacted form For the relation-based compacted form to be useful it is desirable that the more-general-than relation between two conceptual graphs G1 and G2 be checkable by subgraph-isomorphism on the compacted form . This is the case , since the structure of CGs is such that if G1 is more-general-than G2 each relation in G1 should have a corresponding more-specific relation in G2 . This corresponds directly to comparing two nodes in the relation hierarchy . The edges in the compacted form represents the join of two adjacent relations . Whether edges in G1 can match edges in G2 can be determined by simply checking that the fields on which the relations are joined correspond -LRB- given that the endpoints of the edges are known to correspond -RRB- . 5 Method V retrieval algorithm Method V retrieval is a previously unpublished algorithm that is designed to have the simplicity of Method III retrieval -LSB- 11 -RSB- coupled with the benefits of Method IV . We discuss it here since it can be used to even further enhance the efficiency of retrieval over the conceptual graph hierarchy . Method IV -LSB- 12 , 13 -RSB- introduced the notion of \ node descriptors '' which are more specific descriptions of nodes in graphs that support matching due to their specificity much in the way that relation nodes do so in the above discussion . The idea of Method IV is to do many graph comparisons in parallel by taking advantage of their shared node descriptors . Method V goes one step further by CG Diagramatic Representation Relation-Based CG Representation 1 2 4 3 Oxygen bond Carbon single bond Oxygen single bond Nitrogen single bond Nitrogen single bond single bond CG Hypergraph Representation Oxygen Carbon Oxygen Nitrogen Nitrogen 1 double bond -LSB- Oxygen , Carbon -RSB- 2 single bond -LSB- Oxygen , Carbon -RSB- 3 single bond -LSB- Oxygen , Nitrogen -RSB- 4 single bond -LSB- Nitrogen , Carbon -RSB- Relation Table single bond f2 ,2 g f2 ,2 g f1 ,1 g double bond double Figure 4.1 : Compacting Conceptual Structures 8 6 . State-space search as relation-based hypergraph transformation noting that a graph is itself its best descriptor of its nodes and that if A is a subgraph of B , the nodes in A -LRB- and hence their descriptors -RRB- have bindings to nodes -LRB- and descriptors -RRB- in B . In the conceptual graph hierarchy we store bindings from the nodes of each graph to the nodes of their successors . -LRB- These are additional pointers than those discussed above -RRB- . As a graph may bind in more than one place , there actually may be multiple sets of bindings that would have to be stored and sometimes , such as with star-graphs , an exponential number of bindings may be possible . To prevent this combinatorial explosion and for simplicity sake , for each node in a graph we store all the nodes it could possibly bind to in each of its successor graphs . Compaction as above reduces the number of such possibilities . This information can now be used to dramatically improve the performance of Method III , by reducing the size of the candidate binding lists . For example , suppose graph G has four immediate predecessor graphs H1 , H2 , H3 , H4 . If we query graph G in Phase I of Method III we know that H1 , H2 , H3 and H4 have all matched Q . This gives us a very good indication of how G may fit into -LRB- bind to -RRB- Q. For example , if node 1 of H1 mapped into node 5 of Q , and we know that this same node maps only into nodes 7 and 8 of G , we know that at least one of these nodes must bind to node 5 of Q - if G is to be a subgraph of Q. Further , constraints of the bindings of nodes 7 and 8 of G can be retrieved from the bindings of H1 , H2 , H3 , H4 , thus substantially reducing the candidate binding lists of G , before backtracking is taken up to resolve the remaining bindings . Note that in this framework it is necessary to find ALL bindings of each graph to the query graph -LRB- at least for insertion -RRB- and that finding them is facilitated by the method itself . 6 State-space search as relation-based hypergraph transformation The state space search paradigm in which problems are broken down into initial conditions , terminal conditions -LRB- goals -RRB- and operators continues to be one of the most pervasive problem-solving models used in AI research 3 The fundamental importance of the paradigm is seen when one recognizes that each of automatic programming -LRB- in which the machine translates from one very high-level language into one less \ high '' -RRB- , automatic theorem proving -LRB- in which one tries to prove formulas from axioms and inference rules -RRB- , and pattern recognition -LRB- in which one parses from one representation into another -RRB- are special cases . We now show how all state space search problems as they are normally formalized -LSB- 10 -RSB- can be viewed as relation-based transformations over hypergraphs -LRB- which are a special case of conceptual graphs -RRB- . In the next section we show how this relation-based view can allow these problems to be monitored incrementally and efficiently . To reach a definition of generic search problem we consider notions of domain objects , static relations , dynamic relations , variables and bindings . The framework is inspired by Peirce 's existential graphs -LSB- 18 -RSB- , conceptual graphs , and our own work in experience-based planning -LSB- 14 -RSB- . The following is a description of the components of a search problem , with running examples taken from Tic-Tac-Toe 4 and Towers-of-Hanoi on three disks : 5 ffl Each domain will have a finite set of domain objects . In Tic-Tac-Toe the objects will be squares fS1 , S2 , S3 , S4 , S5 , S6 , S7 , S8 , S9g and pieces fX , O , B -LRB- for blank -RRB- g . In Towers-of-Hanoi , the objects are PegsfP1 , P2 , P3g and Disks fD1 , D2 , D3g . ffl Unary , binary and higher relations may be defined on these objects . An n-ary relation is a set of n-tuples of domain objects . In the finite search domains , rather than defining types explicitly we shall simply note that they are implicitly defined as the set of objects that occur in any single field -LRB- attribute -RRB- of a relation . At the implementation level , relations that 3The objective of a state-space search problem is to find a sequence of operators -LRB- transformations -RRB- that will convert a state that satisfies the initial conditions into one that satisfies the terminal conditions , under various optimality criteria and resource constraints . 4Tic-Tac-Toe is played on a 3x3 board with two players X and O alternating turns selecting cells with X going first . The first player completing a row , column or diagonal wins . 5Towers-of-Hanoi is a single agent game involving three disks \ small '' , \ medium '' , \ large '' placed on top of each other in order on the first peg of three . The object is to move the disks one at a time such that at no time is a disk on top of a disk smaller than it on the same peg , and such that the disks finish all on the third peg . 6 . State-space search as relation-based hypergraph transformation 9 are symmetric or transitive may be abbreviated by specifying a kernel set of tuples and then giving the mathematical properties from which the remaining tuples can be inferred . Other abbreviations and computation of relations are possible such as finding adjacent squares on the chessboard through calculation . We break relations into two classes : static relations are those whose definition -LRB- tuples -RRB- remain constant for a given game , and dynamic relations are those whose content can change from state to state . A frequent use of static relations is in defining board topology . Most domains have a dynamic relation corresponding to ON to say which pieces are on which squares . In Tic-Tac-Toe , we define a \ THREE-IN-A-ROW '' static relation corresponding to winning sets of squares : f -LRB- S1 , S2 , S3 -RRB- , -LRB- S4 , S5 , S6 -RRB- , ... g and the ON dynamic relation initialized as f -LRB- B , S1 -RRB- , -LRB- B , S2 -RRB- etc. g . In Towers-of-Hanoi there is a static relation SMALLER-THAN initialized to : f -LRB- D1 , D2 -RRB- , -LRB- D2 , D3 -RRB- , -LRB- D1 , D3 -RRB- g. Towers-of-Hanoi also has the dynamic relation ON , initialized to : f -LRB- D3 , P1 -RRB- , -LRB- D2 , P1 -RRB- , -LRB- D1 , P1 -RRB- g. ffl Operators define transformations over states by changing the contents of dynamic relations . They are specified by giving sets of preconditions , additions and deletions . Note that each of these sets -LRB- and their combination -RRB- can be viewed as a conceptual graph or hypergraph . For Tic-Tac-Toe we have an operator MOVE -LRB- piece , square -RRB- : Pre : ON -LRB- B , S1 -RRB- , Add : ON -LRB- X , S1 -RRB- , Del : ON -LRB- B , S1 -RRB- . -LRB- we assume the board is always oriented with X to move . -RRB- For Towers-of-Hanoi we have the following MOVE operator : MOVE -LRB- d1 : disk , p1 : peg , p2 : peg -RRB- = pre : ON -LRB- d1 , p1 -RRB- AND ~ -LRB- p1 = p2 -RRB- AND ~ -LRB- -LRB- ON -LRB- d2 , p2 -RRB- OR ON -LRB- d2 , p1 -RRB- -RRB- AND SMALLER_THAN -LRB- d2 , d1 -RRB- -RRB- . add : ON -LRB- d1 , p2 -RRB- del : ON -LRB- d1 , p1 -RRB- ffl States are hypergraphs over domain objects . As the static relations are always true it is only necessary to give the dynamic relations when representing a state . ffl An operator is then applicable in a given state iff there is a 1-1 mapping in variables of the preconditions of the operator to domain objects such that all relations specified in the operator are true -LRB- or false , if negated -RRB- of those domain objects . The result of applying the operator is to remove from the current state those tuples corresponding to the deletions and add those tuples associated with the additions to the contents of the dynamic relations . Since static relations are constant throughout the problem-solving process those bindings of objects to variables that could ever possibly -LRB- constrained by the static relations -RRB- satisfy an operator definition , can in principle be computed ahead of time leaving only the dynamic conditions to be checked . ffl Terminal conditions are defined in exactly the same way as preconditions , addition and deletion conditions of operators . It is automatically assumed that a player having no legal moves is terminal . For tic-tac-toe , we have THREE-IN-A-ROW -LRB- s1 , s2 , s3 -RRB- AND ON -LRB- X , s1 -RRB- AND ON -LRB- X , s2 -RRB- AND ON -LRB- X , s3 -RRB- as terminal . In Towers-of-Hanoi , we have ON -LRB- D1 , P3 -RRB- AND ON -LRB- D2 , P3 -RRB- AND ON -LRB- D3 , P3 -RRB- as terminal conditions . ffl Reward conditions are conditions -LRB- often the same as terminal -RRB- that are coupled with a reward to each player based on the outcome of the game . . For Towers-of-Hanoi and Tic-Tac-Toe , reward conditions are the same as terminals and assign a win to the player who has just moved . ffl Finally , FLIP is a static binary relation over domain objects used to define symmetries so that a game can be encoded from one player 's perspective only . For tic-tac-toe FLIP is : f -LRB- X , O -RRB- , -LRB- O , X -RRB- , -LRB- B , B -RRB- , -LRB- S1 , S1 -RRB- , -LRB- S2 , S2 -RRB- ... g. Tower-of-Hanoi , being a single-agent game , does not require a FLIP operator . 10 7 . Monitoring state-space search incrementally using UDS ffl In summary , an abstract game or search problem is defined as a finite set of domain objects , and finite sets of static relations , dynamic relations , operators , terminal conditions and reward conditions . Finally , for convenience in encoding , we define a symmetry condition -LRB- known as FLIP . -RRB- The conclusion is that a large spectrum of single-agent and multi-agent search problems can be viewed as games of -LRB- hyper -RRB- graph-to-graph transformation directly analogous to organic chemical synthesis : states are graphs , and operators are graph-to-graph productions , terminal and reward conditions may also be expressed as graphs . This conclusion is not surprising , given that conceptual graphs and other semantic network schemes have been shown to carry the same expressive power as first-order logic . The conclusion is significant , however , in that it suggests the potential for graph-theoretic analysis of the rules of a domain and ensuing experience for uncovering powerful heuristics and decision-making strategies . -LSB- 16 -RSB- It also suggests that state-space search can be monitored in a uniform manner , we take up this topic in the next section . 7 Monitoring state-space search incrementally using UDS To claim universality for UDS we must show that it is an effective monitor and executor of the specifications of any given state space search domain . As it turns out , due to the relation-based perspective of UDS the following ideas from the RETE algorithm -LSB- 7 , 17 -RSB- can be exploited with little adjustment to UDS as defined above : ffl The firing of an individual operator does not affect the current state radically . ffl If an operator did not match in the previous cycle it most likely will not match in the current cycle . ffl On each cycle we should only try to re-match operators that could have been affected by the previous operator application . ffl Different operators may share a large amount of the same structure . Thus , separate conditions of operators should only be matched once per cycle . ffl Variable bindings from cycle to cycle remain relatively consistent . UDS -LRB- our implementation is in C++ -RRB- monitors search problems as follows : The relations in the relational hierarchy are used to represent dynamic relations . Specific relations -LRB- tuples -RRB- that are true are stored beneath -LRB- as specifications -RRB- the schema declarations for the relations . A schema declaration and its tuples is equivalent to a table in a traditional relational database and we shall also use the term \ table '' in the following discussion . The preconditions of operators are stored using the graph hierarchy . Repeated parts of operators are only represented once in the relation hierarchy . Figure 7.1 depicts the initial UDS network for monitoring the Towers-of-Hanoi . There may be a difference between UDS and standard RETE implementations in that UDS exploits the relationbased representation of CGs to extend the types of patterns that can be matched and enhances the speed of doing so . However , the important thing is that UDS naturally supports RETE in addition to a variety of other data manipulation methods appropriate to relational databases , CGs and semantic networks . Those dynamic relations that do not depend on any other relations in their definition are known as primitive dynamic relations . The post-conditions of operators work directly on the primitive dynamic relations -LRB- through pointers -RRB- by adding or deleting tuples from their contents . Static relations are compiled away at network generation time since by definition the set of tuples that satisfy them remains constant . Conceptual graphs representing the preconditions of operators are only re-matched if the content of one of their composing relations changes . Only that part of the conceptual graph affected by the change need be re-matched . 7.1 What happens after an operator is selected . 1 . A selected operator corresponds to a tuple in one of UDS 's operator tables , e.g. Move -LRB- D1 , P1 , P3 -RRB- in Towers-of-Hanoi . 7 . Monitoring state-space search incrementally using UDS 11 F SMALLER-THAN -LRB- static -RRB- D1D2 D3 Disk1 Disk2 TERMINAL -LRB- dynamic -RRB- D2 P2 Disk Peg ON -LRB- dynamic -RRB- D2 D1 P1 D3 P3 MOVE -LRB- dynamic -RRB- Disk Peg1 Peg2 Delete Add D1D1 P1P1 P2P3 D1 , P2D1 , P3 D1 D3 P1 P2 P3 D2D1D3 Figure 7.1 : Initial state of UDS network for monitoring Towers-of-Hanoi 2 . This tuple is bound to the given variable-arguments of that operator schema and lists of add and delete tuples are built based on the add and delete conditions of that schema . In our example of firing Move -LRB- D1 , P1 , P3 -RRB- : ON -LRB- D1 , P1 -RRB- will be deleted and ON -LRB- D1 , P3 -RRB- will be added . 3 . These lists of tuples are then added and deleted immediately from the appropriate primitive dynamic relation tables in the net . Since the truth value of the net has been altered because of the immediate adding and deleting of the operator-generated tuples , these new changes are propagated up the net . 4 . Each of the tuples in the add/delete lists created , are iterated thru for each table in the net that they may directly affect -LRB- these are their successors in the hierarchy -RRB- by changing the truth values of tuples in that table . If one of the added or deleted tuples match a pre-condition of a table , then a \ re-join '' procedure is called to update the tuples currently being stored in the table . Those tables not affected by the new add/delete tuples are ignored . Thus , in our example , the firing of Move -LRB- D1 , P1 , D3 -RRB- removes -LRB- D1 , P1 , P2 -RRB- and -LRB- D1 , P1 , D3 -RRB- from the Move table and adds -LRB- D1 , P2 , P1 -RRB- and -LRB- D2 , P1 , P3 -RRB- to the Move table . 7.2 How Join works . A relation table defined as the conjunction of two or more subtables can have its tuples calculated by taking a join of its subtables as in a relational database . In UDS we calculate the contents of a 12 7 . Monitoring state-space search incrementally using UDS table incrementally when one or more of its subtables have their contents changed -LRB- by deleting or adding tuples -RRB- . Obviously , Join is a critical operation and must be as efficient as possible . A join of an individual tuple is similar to a traditional subgraph-isomorphism test -LSB- 1 -RSB- , but by working at the table -LRB- or schema -RRB- level a large number of subgraph bindings are found simultaneously , -LCB- illustrating another advantage of the UDS organization . Further advantages are accrued due to the incrementality . For each table T , Join is called once for each pre-condition table P that has changed . Join works in two phases : If the current precondition table P has variables that have already been bound previously -LRB- on earlier preconditions in T 's precondition list -RRB- , then in Phase II , the system checks to see if the current tuples in T are valid given P 's tuples and already bound variables , and removes any that are not . Otherwise -LRB- one or more variables are unbound -RRB- a Merge operation is done in Phase II based on iteration rather than Look-up of pre-condition Tuples . For example , consider a table , T , to be updated which contains the following tuples : -LRB- s1 , WP , S5 -RRB- , -LRB- s1 , BN , S45 -RRB- , and -LRB- s1 , WK , S4 -RRB- . Variables s2 and s3 have already been bound for all three tuples -LRB- constants are in uppercase -RRB- , while s1 remains unbound . Now suppose Join is iterating upon the following pre-condition argument P : ON -LRB- WP , s3 -RRB- . Then Phase I immediately determines that all variables in pre-condition ON -LRB- WP , s3 -RRB- -LRB- fs3g in this case -RRB- are already bound , and so PHASE II will be a Look-up phase . In this phase , tuples in T will be iterated over , and by combining the bound variables with ON -LRB- WP , s3 -RRB- , the following subtuples will have there existence checked in ON : -LRB- WP , S5 -RRB- , -LRB- WP , S45 -RRB- and -LRB- WP , S40 -RRB- . If the subtuple queried is not present in ON , the corresponding tuple of T will be eliminated . Merge . If a Merge is required in Phase II , then Join iterates through the tuples in T , binding each tuple to all possible variables in turn , then iterating through P 's tuples , and building a new tuple based on comparing the current T tuple to the pre-condition tuple , seeing if currently bound variables match , and adding new objects to the unbound variables . For example , again start with table T above . Suppose that join is iterating upon pre-condition argument : CAPTURE -LRB- s1 , WP -RRB- . We see that \ s1 '' , the only variable in the pre-condition , is unbound in T. Merge is then called to iterate over the CAPTURE tuples , stopping at any tuples where the second argument is \ WP '' -LRB- the constant -RRB- and will then bind \ s1 '' to the first argument . Thus an \ s1 '' bind with -LRB- for example -RRB- CAPTURE -LRB- S33 , WP -RRB- , would create new Tuples in T : -LRB- S33 , WP , S5 -RRB- , -LRB- S33 , BN , S45 -RRB- and -LRB- S33 , WK , S4 -RRB- . Note that \ s1 '' can be bound more than once , and in this particular example , three new tuples would then be created each time a bind occurred . 7.3 Implementation and Extensions Join , as defined above , is used to combine subtables that are connected through conjunction -LRB- AND -RRB- . Tables that are combined through OR use an implicit UNION operation -LRB- as in relational databases -RRB- . Since UNION simply involves including all tuples in each subtable in a new table , UNION is implemented by simply storing a list of subtables -LRB- rather than creating a new table -RRB- . NOT is also simulated -LRB- as opposed to creating a new table of tuples -RRB- by returning exactly the opposite answers about tuple existence as its subtable . Relation tables in our implementation of UDS are stored as hashtables over their tuples that also allow iteration -LRB- through a linked-list of buckets -RRB- over all tuples . Note that the pointers from the node hierarchy into the relation hierarchy can be used as indices to further facilitate the Look-Up and Merge operations of join . For example , the WP node will point to all tuples involving WP . 7.4 Performance results . With this scheme , we have been able to monitor a variety of domains including Tic-Tac-Toe , Tower-of-Hanoi , 8-puzzle , and Hexpawn at a level of efficiency that is faster -LRB- in some cases up to 10 times -RRB- than previous programs of ours that had been written specifically for these domains but were not incremental . In Figure 7.4 below we give initial timings for UDS in playing -LRB- at various search depths -RRB- and monitoring a variety of search domains that have been compiled into its tuple network from rule definitions as in Section 6 . As UDS is domain-independent and not hard-coded for any of these domains it clearly should not be as fast as any domain-specific system . Still , through incrementality , its results , as shown , are reasonable and would likely beat -LRB- for domains with high branching factors -RRB- a hard-coded program that does not exploit incrementality . As this is an early implementation with several optimizations not yet completed , we expect even stronger results in the near future . The results reported are using a C++ implementation on a SUN SparcII with 64 megabytes of main memory . 8 Support for pattern matching A common problem-solving operation used in case-based reasoning systems is to find all generalizations of previously seen situations that match a given state . This information is used to reason by analogy with the past , perhaps by retrieving the previous situations themselves . Other systems such as Morph -LSB- 8 , 15 -RSB- construct an evaluation function that is a combination of the weights of the most-specific patterns that match a state . Both types of reasoning require matching a large database of patterns against the current state . Methods III-V were designed exactly with this operation in mind . However , until the integration of the ideas from RETE these algorithms were not necessarily as fast as they could be since they did not support incrementality -LRB- in which there is little change from one query to the next -RRB- . Thus the Morph chess system , using UDS , after a pawn moves need only rematch those patterns affected by the pawn move rather than starting from scratch by reconsidering its database of -LRB- up to 50000 -RRB- patterns . Such savings due to incrementality should provide great benefits in most inference settings . 9 Support for machine learning Not only does UDS support retrieval , problem-space monitoring and pattern-matching , it also provides a structure that is highly convenient for developing machine learning algorithms for learning evaluation functions -LSB- 16 -RSB- . The multi-level hierarchical view of UDS shows directly how nodes are combined into relations and in turn how relations are combined into conceptual graphs . This hierarchical decomposition into wholes-and-parts is also fundamental in machine learning . Much of the work in neural networks , statistics and pattern recognition deals directly with \ how '' the values of parts combine to create the value of the whole . Such methods include gradient-descent , linear-regression and nearest neighbor algorithms -LSB- 23 -RSB- . UDS gives one the capability of employing these methods to produce a combining rule for each decomposition as one traverses the hierarchy . For example , in chess one might like to determine the value of having a piece between the king and an attacking piece . In our declarative representation of the rules of chess , we have defined the BETWEEN-KING-ATTACKER relation as follows : BETWEEN-KING-ATTACKER -LRB- SQUARE : s1 , SQUARE : s2 , SQUARE : s3 ; BLACKPIECE : p1 , WHITEPIECE : p2 -RRB- = ATTACK -LRB- s1 , s2 -RRB- & ON -LRB- WK , s2 -RRB- & BETWEEN -LRB- s1 , s2 , s3 -RRB- & MOVEABLE -LRB- s2 , s3 -RRB- & MOVEABLE -LRB- s1 , s3 -RRB- & ON -LRB- p1 , s1 -RRB- & ON -LRB- p2 , s3 -RRB- . This relation represents a conceptual graph composed of more primitive relations . Informally , the meaning is a white piece p2 on s3 is between a white king on s2 and an attacking piece p1 on s1 , if s3 is between s1 and s2 and it is possible to move p1 from s1 to s2 and then to s3 . The table for this relation will sit below -LRB- in the CG hierarchy -RRB- the tables for the relations it is composed of . The APSII domain-independent game-playing system -LSB- 16 -RSB- uses UDS to evaluate a BETWEEN-KING - ATTACKER tuple , say -LRB- 10,11,12 , br , wp -RRB- as follows : 14 9 . Support for machine learning MONITORING AND EXECUTION TIME FOR VARIOUS DOMAINS SINGLE AGENT GAMES -LRB- IN MOVES TAKEN -RRB- : TOWERS-OF-HANOI 100 Turns 500 Turns -LRB- With Game Restarts -RRB- --------- --------- total | ave total | ave -LRB- min : secs -RRB- . R vs R < :01 | - :02 | 0.004 1 ply :03 | 0.03 :15 | 0.03 2 ply :19 | 0.19 1:42 | 0.20 3 ply 1:50 | 1.10 10:18 | 1.24 8 PUZZLE 100 Turns 500 Turns -LRB- Randomly , only 1 Game -RRB- --------- --------- total | ave total | ave -LRB- min : secs -RRB- . R vs R < :01 | 0.01 :02 | 0.004 1 ply :04 | 0.04 :22 | 0.04 2 ply :42 | 0.42 3:25 | 0.41 3 ply 4:58 | 2.98 26:45 | 3.21 DOUBLE AGENT -LRB- IN GAMES -RRB- : HEX PAWN 100 Games 500 Games --------- --------- total | ave total | ave -LRB- min : secs -RRB- . R vs R :02 | 0.02 :10 | 0.02 1 ply :06 | 0.06 :30 | 0.06 2 ply :22 | 0.22 1:50 | 0.22 3 ply 1:20 | 0.80 6:45 | 0.81 TIC-TAC-TOE 100 Games 500 Games --------- --------- total | ave total | ave -LRB- min : secs -RRB- . R vs R :03 | 0.03 :15 | 0.03 1 ply :09 | 0.09 :45 | 0.09 2 ply :55 | 0.55 4:50 | 0.58 3 ply 6:55 | 4.15 - R vs. R means `` random-player vs. random-player '' . n-Ply means one agent is searching this deep , the other agent is playing randomly . 1 . The tuple is looked up in a table of previously seen tuples -LRB- each relation table has a history table for learning purposes -RRB- . If the tuple is found , the learned weight for the tuple is returned , else the system continues on to steps 2 and 3 . 2 . Subtables are called to get the weights of subtuples -LRB- based on the table 's definition -RRB- of the original tuple . In our example , the weights of ATTACK -LRB- 10,11 -RRB- , ON -LRB- WK ,11 -RRB- , MOVEABLE -LRB- 11,12 -RRB- , MOVEABLE -LRB- 10,12 -RRB- , ON -LRB- BR ,10 -RRB- , and ON -LRB- WP ,12 -RRB- are asked for . In general , such calls may themselves induce deeper recursive calls to subtables of subtables and so on , until tuples are found or we bottom out in the node tables that represent individual domain objects -LRB- and serve as arguments to the primitive dynamic relations -RRB- . A bottom level table returns a default -LRB- neutral -RRB- value if a tuple is not found in its history table . Note that BETWEEN is not called on -LRB- 10,11,12 -RRB- since it is a static relation . 6 3 . Once the values of all subtuples are found , they are combined linearly using coefficients -LRB- the higher the coefficient the more accurate -LRB- with respect to feedback sent to the BETWEEN - KING-ATTACKER table -RRB- the subtable has been in the past -RRB- that have been learned for the table . The result of the linear combination is returned . In our example , suppose weights of 0.55,0.45,0.43,0.42,0.50 had been returned from the subtables . If the learned coefficients for the BETWEEN-KING-ATTACKER table are 0.3 , 0.3 , 0.2 , 0.1 , 0.1 -LRB- the coefficients are normalized to sum to 1 -RRB- the value returned for the tuple -LRB- 10,11,12 , br , wp -RRB- will be 0.478 . 10 Conclusion Here we have shown how UDS can be used to combine the best features of semantic network marker passing retrieval , relational databases and conceptual graphs . A standout features of this new data structure is the degree to which it is able to simplify and compact the storage of a conceptual graphs database . In addition to giving Method V as an improvement over Methods III-IV retrieval we have shown how UDS can be used to incrementally monitor state-space search domains -LRB- since they are based on transformations of conceptual graphs -RRB- and to support pattern-matching and machine learning . Although there are details still to be worked out in how to best integrate the major database paradigms in UDS , we hope that the ideas presented here will lead to a further unification of currently disparate information retrieval and problem-solving methods . Future directions are directed at the fact that in UDS as in its underlying components , query processing time is too heavily influenced by exactly which structures and joins are pre-stored in the database . We feel that a truly universal model should not be as susceptible to the original representation of the facts and should be able to reorganize itself to make better use of them . Along similar lines , the determination of key intermediate concepts remains a critical and fundamental issue in machine learning and one that we are studying . 11 Acknowledgements This research was supported in part by NSF research grant IRI-9112862 . Thanks to Don Roberts who provided diagrams and feedback during the construction of the paper . Thanks to Yuxia Zhang for her assistance with UDS and search-space definition . Finally , thanks to John Amenta who is largely responsible for the current UDS implementation , helped obtain the performance results , and supplied some algorithmic descriptions for the paper . 6The presence of a tuple in a static relation is invariant across positions and hence supplies no new information to base an evaluation on . 16 References References -LSB- 1 -RSB- J.M. Barnard . Problems of substructure search and their solution . In Wendy Warr , editor , Chemical Structures the International Language of Chemistry , . Springer-Verlag , 1988 . -LSB- 2 -RSB- H. Boley . Declarative operations on nets . Computers and Mathematics with Applications , 23 -LRB- 6 - 9 -RRB- :601 -LCB- 638 , 1992 . Part 2 of Special Issue on Semantic Networks in Artificial Intelligence , Fritz Lehmann , editor . Also reprinted on pages 601 -LCB- 638 of the book , Semantic Networks in Artificial Intelligence , Fritz Lehmann , editor , Pergammon Press , 1992 . -LSB- 3 -RSB- G. Ellis . Compiledhierarchical retrieval . In E. Way , editor , Proceedings of Sixth Annual Workshop on Conceptual Structures -RRB- , pages 187 -LCB- 208 , SUNY-Binghamton , 1991 . To Appear . -LSB- 4 -RSB- G. Ellis . Efficient retrieval from hierarchies of objects using lattice operations . In G. Mineau and B. Moulin , editors , Proceedings of First International Conference on Conceptual Structures -LRB- ICCS-93 -RRB- , Montreal , 1993 . To Appear . -LSB- 5 -RSB- R. Elmasri and S.B. Navathe . Fundamentals of Database Systems . Benjamin/Cummings , Redwood City , California , 2 edition , 1994 . -LSB- 6 -RSB- S. Fahlman . NETL : A System for Representing and Using Real-World Knowledge . MIT Press , Massachusetts , 1979 . -LSB- 7 -RSB- C.L. Forgy . Rete : A fast algorithm for the many pattern/many object patern match problem . Artificial Intelligence , 19 -LRB- 1 -RRB- :17 -LCB- 37 , 1982 . -LSB- 8 -RSB- J. Gould and R. Levinson . Experience-based adaptive search . In R. Michalski and G. Tecuci , editors , Machine Learning : A Multi-Strategy Approach , volume 4 , pages 579 -LCB- 604 . Morgan Kauffman , 1994 . -LSB- 9 -RSB- J.A. Hendler . Massively-parallel marker-passing in semantic networks . In Fritz Lehmann , editor , Semantic Networks in Artificial Intelligence , pages 277 -LCB- 292 . Pergamon Press , 1992 . -LSB- 10 -RSB- R. E. Korf . Planning as search . Artificial Intelligence , 1987 . -LSB- 11 -RSB- R. Levinson . A self-organizing retrieval system for graphs . In AAAI-84 , pages 203 -LCB- 206 . Morgan Kaufman , 1984 . -LSB- 12 -RSB- R. Levinson . Pattern associativity and the retrieval of semantic networks . Computers and Mathematics with Applications , 23 -LRB- 6-9 -RRB- :573 -LCB- 600 , 1992 . Part 2 of Special Issue on Semantic Networks in Artificial Intelligence , Fritz Lehmann , editor . Also reprinted on pages 573 -LCB- 600 of the book , Semantic Networks in Artificial Intelligence , Fritz Lehmann , editor , Pergammon Press , 1992 . -LSB- 13 -RSB- R. Levinson and G. Ellis . Multilevel hierarchical retrieval . Knowledge-Based Systems , 5 -LRB- 3 -RRB- :233 -LCB- 244 , September 1992 . Special Issue on Conceptual Graphs . -LSB- 14 -RSB- R. Levinson and K. Karplus . Graph-isomorphism and experience-based planning . In D. Subramaniam , editor , Proceedings of Workshop on Knowledge Compilation and Speed-Up Learning , Amherst , MA. , June 1993 . -LSB- 15 -RSB- R. Levinson and R. Snyder . Adaptive pattern oriented chess . In Proceedings of AAAI-91 , pages 601 -LCB- 605 . Morgan-Kaufman , 1991 . -LSB- 16 -RSB- R.A. Levinson . Exploiting the physics of state-space search . In Proceedings of AAAI Symposium on Games : Planning and Learning , pages 157 -LCB- 165 . AAAI Press , 1993 . -LSB- 17 -RSB- D. P. Miranker . Treat : A better match algorithm for ai production systems . In Proceedings of AAAI-87 , pages 42 -LCB- 47 , 1987 . -LSB- 18 -RSB- D.D. Roberts . The existential graphs . In Semantic Networks in Artificial Intelligence , pages 639 -LCB- 664 . Roberts , 1992 . -LSB- 19 -RSB- E. Sciore . A complete axiomatization for join dependencies . JACM , 29 -LRB- 2 -RRB- :373 -LCB- 393 , April 1982 . -LSB- 20 -RSB- J. F. Sowa . Conceptual Structures . Addison-Wesley , 1983 . -LSB- 21 -RSB- P. Suppes . Models of data . In Logic , Methodology and Philosophy of Science , pages 252 -LCB- 261 . Stanford , East Lansing , 1962 . -LSB- 22 -RSB- J.D. Ullman . The u.r. strikes back . In Proceedings of the ACM SIGACT-SIGMOD Symposium on Principles of Database Systems , pages 10 -LCB- 22 , 1982 . -LSB- 23 -RSB- S. Watanabe . Pattern Recognition : Human and Mechanical . Wiley , New York , 1985 .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

conceptual graphs
conceptual structures
machine learning
pattern matching
relational database
rete
state space search