GET : An Implementation of Conceptual Graphs Michel Wermelinger ? -LRB- mw@fct.unl.pt -RRB- Centro de Intelig ^ encia Artificial/UNINOVA Quinta da Torre , 2825 Monte da Caparica , PORTUGAL Abstract This paper describes GET -LRB- Graph Editor and Tools -RRB- , a partial implementation of Sowa 's conceptual structures . First , a brief overview of X-Prolog , an X-Windows based Prolog programming environment , is given . A summary of GET 's graphical interface and linear notation follows . Finally , an example application using the graphs to represent sentences in Portuguese will be presented . 1 Introduction Conceptual structures are an ambitious attempt to represent knowledge in a natural and expressive way . The formalism was first presented in -LSB- Sow84 -RSB- but has its roots in Peirce 's existential graphs . As Sowa puts it himself , -LSB- Conceptual graphs are -RSB- a system of logic with a graph-based formalism that aims for a very wide expressive power . Its primary purpose is to serve as an intermediate language between natural language and other formalisms including database query languages , frames , production rules , and predicate calculus . GET is intended to be used as a tool for experimenting Sowa 's conceptual graphs as a basis for semantic representation of Portuguese sentences . In its actual state , GET is a collection of Prolog predicates that implement several of the operations on conceptual graphs and a graph editor for X-Windows . The former is collectively called CGT -LRB- Conceptual Graph Tools -RRB- and the latter is known as CGE -LRB- Conceptual Graph Editor -RRB- . For the reader unfamiliar with conceptual structures , a very brief overview is given in the next section . The rest of the paper will just make references to particular parts of the formalism using e.g. \ Definition 3.4.1 '' as an abbreviation for \ see -LSB- Sow84 , Definition 3.4.1 -RSB- '' . Then , the paper will briefly describe Quintus X-Prolog , the language used to implement GET , stressing the graphical user interface . The fourth section will deal with CGT , focusing on the changes made to the linear notation . Next , CGE is presented , emphasizing the visualization of graphs and the operations one can perform on them . Finally , the last section will show an application that uses conceptual graphs as the semantic representation of sentences in Portuguese . For a more detailed account on all of these topics , see -LSB- Wer91a -RSB- and -LSB- Wer91b -RSB- . ? Owns a scholarship from JNICT . 2 Conceptual Graphs Conceptual graphs consist of two kinds of nodes : concepts and relations . Concepts are graphically represented as boxes and relations as labeled circles . For example , the sentence \ John loves Mary '' could be expressed by the following graph PERSON : John oe OEAE Oo AGNT oe LOVE - OEAE Oo OBJ - PERSON : Mary whose linear notation is -LSB- PERSON : John -RSB- < - -LRB- AGNT -RRB- < - -LSB- LOVE -RSB- - > -LRB- OBJ -RRB- - > -LSB- PERSON : Mary -RSB- . Concepts have two fields , separated by a colon , denoting the type and the referent . There are several kinds of referents , some of which are indicated below : Referent Notation Reading generic -LSB- CAT : * -RSB- or -LSB- CAT -RSB- a cat individual -LSB- CAT : # -RSB- the cat name -LSB- CAT : Morris -RSB- Morris set -LSB- CAT : fMorris , Felixg -RSB- Morris and Felix universal -LSB- CAT : 8 -RSB- all cats Whenever the graph does n't fit into a single line , the relations -LRB- concepts -RRB- linked to the same concept -LRB- relation -RRB- may be written on separate lines . The dash is used to introduce such a list of items which are separated by newline characters . For instance , the above example could also be written as -LSB- LOVE -RSB- - -LRB- OBJ -RRB- - > -LSB- PERSON : Mary -RSB- -LRB- AGNT -RRB- - > -LSB- PERSON : John -RSB- . A context is a concept whose type is ` PROPOSITION ' . The referent field of a context is the generic marker -LRB- ` * ' -RRB- or a set of graphs . Coreference links are used to connect concepts denoting the same individual . Visually they are represented with dotted lines and the linear notation uses multiple referents . For example , \ Rosalie is not Rosann '' is represented by the graph PERSON : Rosalie PERSON : Rosann oe aess oe ? NEG whose linear notation is -LSB- PERSON : Rosalie -RSB- -LRB- NEG -RRB- - > -LSB- PROPOSITION : -LSB- PERSON : Rosann = Rosalie -RSB- -RSB- . which can be further abbreviated to -LSB- PERSON : Rosalie -RSB- ~ -LSB- -LSB- PERSON : Rosann = Rosalie -RSB- -RSB- . The concept types form a hierarchy , like the classes in object-oriented programming languages . To define new types and relations one can use lambda abstractions , which are graphs using concepts as formal parameters . The linear notation is e.g. ` type WHITE-CAT -LRB- x -RRB- is -LSB- CAT : * x -RSB- - > -LRB- ATTR -RRB- - > -LSB- WHITE -RSB- . ' where ` * x ' is called a variable . Abstractions may be written directly in the type field , using ` > = ' instead of a name . Lambda abstractions are also used to define schemata , which are graphs associated to concept types specifying background knowledge . The most important operations on conceptual graphs are the canonical formation rules and the propositional and first-order inference rules . The former are used to derive new canonical graphs from the ones given by the user . Canonical graphs define selectional constraints on the way concepts and relations may be combined . As such , there is a canonical graph for each concept and relation type , stating how other concepts and relations link to it . The fundamental distinction between canonical formation rules and inference rules is that the latter always derive true graphs -LRB- i.e. graphs stating true assertions -RRB- from true graphs while the former do n't . For example -LRB- taken from -LSB- Sow84 -RSB- -RRB- , from -LSB- PERSON : Sue -RSB- < - -LRB- AGNT -RRB- < - -LSB- EAT -RSB- - > -LRB- OBJ -RRB- - > -LSB- PIE -RSB- . and -LSB- GIRL -RSB- < - -LRB- AGNT -RRB- < - -LSB- EAT -RSB- - > -LRB- MANR -RRB- - > -LSB- FAST -RSB- . the formation rules derive -LSB- PERSON : Sue -RSB- < - -LRB- AGNT -RRB- < - -LSB- EAT -RSB- - -LRB- MANR -RRB- - > -LSB- FAST -RSB- -LRB- OBJ -RRB- - > -LSB- PIE -RSB- which makes sense -LRB- i.e. is possible -RRB- but may not be true . Therefore , the inference rules are unable to derive the same graph . 3 X-Prolog X-Prolog was created as a result of the ESPRIT project \ Advanced Logic Programming Environments '' -LRB- -LSB- Con89 -RSB- -RRB- . It consists of regular Prolog and the Widget1 Description Language -LRB- -LSB- Abr89b -RSB- -RRB- enabling the programmer to access the X Toolkit functions -LRB- -LSB- MAS88 -RSB- -RRB- and to use the Athena widgets -LRB- -LSB- Pet89 -RSB- -RRB- in a declarative way . The first versions were built on top of C-Prolog , and included a programming environment with a graphical interface featuring editor windows , a widget class browser , menu and dialog packages , and many other goodies -LRB- -LSB- Abr89a -RSB- -RRB- . The most recent version runs under Quintus Prolog and is for X Version 11 Release 4 -LRB- the previous were for release 3 -RRB- , but its user interface consists only of the so called top-level shell , which keeps the history of the launched goals and enables the user to format their output . As the Conceptual Graph Editor was entirely coded in Quintus X-Prolog , it takes advantage of the underlying graphical interface , providing an easy way to edit graphs using windows , dialog boxes , icons , buttons , and the combination of mouse and keyboard . Furthermore , the choice of the X-Windows standard increases portability and decreases the learning time for users already familiar with other graphical interfaces . 1Window gadget | a graphical object in X-Windows terminology . 4 The Conceptual Graph Tools The Conceptual Graph Tools are a relatively portable collection of Prolog -LRB- -LSB- CM87 -RSB- -RRB- predicates implementing the most important operations on conceptual graphs -LRB- see 5.3 -LSB- Editor Commands -RSB- , page 9 -RRB- , a simple contextual mark - & - sweep memory management system , and a linear notation parser and generator using Definite Clause Grammars -LRB- -LSB- PS87 -RSB- -RRB- . 4.1 The Linear Notation The linear notation of conceptual graphs as parsed and generated by GET is a bit different from the one used by Sowa . The formal definition in -LSB- Wer91a -RSB- extends the one given in -LSB- Sow84 , Appendix A. 6 -RSB- , especially in what concerns the type and referent fields , including nested contexts . 4.1.1 Changes Several things Sowa did n't need to concern about , because he was n't describing any real program which interacted with a real user who might give erroneous input , had to be clearly defined . Most of these things are of semantic nature and do n't concern the syntax . Other changes are more of a cosmetic nature -LRB- like using ` \ ' for ` > = ' and ` V ' for ` 8 ' -RRB- . Some of the syntactic changes concern : Separator The \ newline '' character as separator between multiple items of concept and relation lists was considered a poor choice because it should be used only to format the linear input according to the user 's taste . A more fundamental reason from the implementor 's point of view is the need for a clear distinction between the tokeniser and the synctactic parser of the linear notation . This would be impossible if some newlines had a special meaning while others were optional . For these reasons , the character ` j' is used instead . Referents In GET , a restricted form of multiple referents is implemented . In -LSB- Sow84 , section 4.2 -RSB- each of the multiple referents may be a referent as defined before . But this poses some efficiency problems during parsing . For instance , how could the coreference links in the following example be quickly detected ? -LSB- PERSON : -LCB- John , Rose , Mary -RCB- -RSB- ; -LSB- PERSON : -LCB- Sowa , Smith , Robinson -RCB- -RSB- ; -LSB- PROPOSITION : -LSB- PERSON : -LCB- Robinson , Sowa , Smith -RCB- = -LCB- Rose , Mary , John -RCB- -RSB- -RSB- . To overcome this problem , only the first of multiple referents can be a non-variable . This restriction does n't reduce their expressive power and it is efficient to implement , using the usual variable scoping rules of procedural programming languages . As such , the above example must be rewritten as -LSB- PERSON : -LCB- John , Rose , Mary -RCB- = * x -RSB- ; -LSB- PERSON : -LCB- Sowa , Smith , Robinson -RCB- = * y -RSB- ; -LSB- PROPOSITION : -LSB- PERSON : * x = * y -RSB- -RSB- . Another restriction concerns the contexts -LRB- Definition 4.2.1 -RRB- . Although it seems that only propositions may have graphs as their referents , a type ` SITUATION ' appears to exist which would also have that propriety . But the Conceptual Catalog -LRB- -LSB- Sow84 , Appendix B -RSB- -RRB- is n't very coherent with the rest of the book in this respect and does n't describe the ` SITUATION ' type , although it is used in some examples as supertype of ` EVENT ' , which in turn is a direct subtype of the universal type in the Catalog . To clear things up , it was decided that a context must be of a subtype of ` PROPOSITION ' and only contexts may have graphs as referents . In this version of GET , contexts may not be coreferenced , unless they are generic . Otherwise it would be difficult to check if incompatible propositions were being coreferenced . Type Labels In GET , the type label may also contain lowercase letters , digits , and the underscore character . In fact , as long as the label is unique , there should be no problem with not using exclusively uppercase letters and the dash . The type labels of the builtin types are all uppercase , in conformity with -LSB- Sow84 , Appendix B -RSB- . 4.1.2 Errors The parser used in this version is of the SAFE kind -LRB- Stop At First Error -RRB- : it does n't perform any error recovery , stopping with the first error found . It still needs to be made more robust regarding syntax errors . However the following kinds of semantic errors are already accounted for : Ambiguities It was decided that concepts which are coreferenced must be of compatible types , i.e. one must be a subtype -LRB- Assumption 3.2.3 -RRB- of the other . Another ambiguity arises when the same variable is used for different concepts in the same graph . Undefinitions Possible undefinitions are an unknown type label or an unknown parameter in a lambda abstraction -LRB- Definition 3.6.1 -RRB- , i.e. the variable does n't appear in any referent in the body . Bad Arcs The proper use of arcs was relaxed a little bit . Briefly said , the arc number may be dropped if it may be computed from the context . The precise rules are : Each relation may only have a fixed number n of arcs attached to it , where n is given by the relation 's type . The highest-numbered arc must point away from the relation , all the others must point to it . If n is bigger than 2 , at least n-2 of the n-1 incoming arcs must be numbered . The outgoing arc never needs to be numbered . If the relation is monadic -LRB- 1 arc -RRB- or dyadic -LRB- 2 arcs -RRB- , no numbering is necessary . Bad Referents A bad referent may result of breaking one of the following rules : ffl A concept whose referent is of the form ` @ ` Name must be of a subtype of ` DIMENSION ' . ffl A context must have as referent a set of graphs or else it must be generic -LRB- ` * ' -RRB- . Conversely , a concept whose referent is a set of graphs must be of a subtype of ` PROPOSITION ' . Redefinitions In GET , you ca n't change neither the definition of a concept or relation type nor its associated canonical graph . The reason is simple : if the canonical graph associated to a type X changes , all graphs with a concept or relation of type X must probably change , too . And if a type Y is defined in terms of X , the graphs involving concepts or relations of type Y would have to change too , and so on . To keep things simple and efficient , it was decided that redefinitions are forbidden -LRB- at least in this version -RRB- . Breaking the selectional constraints imposed by the canonical graph associated with a relation type is also considered to be an error of this kind . If the canonical graph states that arc n links the relation to a concept of type t , any graph using that relation must link it to a concept of a subtype of t . This forces the created graphs to also be canonical , as it corresponds to the \ restrict '' formation rule -LRB- Assumption 3.4.3 -RRB- . 4.2 Databases CGT provides facilities to manipulate graph databases made up of : ffl a type hierarchy -LRB- Assumption 3.2.3 -RRB- ffl a set of graphs , where each may have some text associated to it ffl for each concept type , a -LRB- possibly empty -RRB- set of schemata -LRB- Definition 4.1.1 -RRB- ffl for each type , the associated canonical graph and/or definition Each database is independent of any other . Several databases can be built on top of a \ core '' database . A sample one comes with GET ; it contains all relations defined in the Conceptual Catalog -LRB- -LSB- Sow84 , Appendix B -RSB- -RRB- , and the most important concept types . Finally , only one database may be open at any time , even if multiple editors are active . 4.3 Future Work As to be expected , several parts of the formalism have not yet been implemented in this first version of the Graph Tools including the first-order rules of inference -LRB- Assumption 4.3.5 -RRB- , and the OE operator -LRB- Assumptions 3.3.2 , 3.6.2 , 4.2.3 , and 4.2.6 -RRB- , which translates graphs into first-order logic formulas . Some of the missing operations could be adapted to the working environment . For example , a modified version of the OE operator could be used to assert graphs as Prolog clauses in order to use Prolog 's inference engine to do deductions . Other things still need to be improved , like the canon -LRB- Assumption 3.4.5 -RRB- , individual markers -LRB- Assumption 3.3.1 -RRB- , and coreference links -LRB- Assumption 4.2.5 -RRB- . Most of these changes will also require theoretical work ; the referents , particularly , will need some reworking -LRB- see e.g. -LSB- GTS89 -RSB- and -LSB- PH90 -RSB- -RRB- . Also , coreference links must be carefully analysed with respect to their side-effects on operations such as the canonical formation rules . 5 The Conceptual Graph Editor CGE is the graphical interface to CGT , and can be considered to be a kind of \ syntaxoriented '' editor , as most commands correspond to operations provided by the formalism , thereby enforcing the resulting graph to be canonical . The alternative would be to have a \ visual '' editor allowing to operate on single nodes and arcs , but its implementation would be more difficult because incomplete and ill-formed graphs would have to be taken into account . 5.1 The Editor Window A Conceptual Graph Editor is a window comprising five areas -LRB- see Figure 1 , page 7 -RRB- : header This area consists of a single line of text displaying a description of the shown graph . The text may be a user defined string -LRB- as in the figure -RRB- , the usual description Figure 1 : A Conceptual Graph Editor -LRB- e.g. ` schema for BUS -LRB- x -RRB- is ' , ` relation AGNT -LRB- x , y -RRB- is ' , etc. -RRB- or simply the word ` graph ' . graphical display It is under the header and shows the edited graph -LRB- see 5.2 -LSB- Graphical Display -RSB- , page 8 -RRB- . linear display It shows the same graph as the graphical display but in linear notation . It is a normal text widget , enabling the user to edit its contents using normal Emacs commands -LRB- -LSB- Sta85 -RSB- -RRB- in order to create graphs which can not be obtained with the menu commands -LRB- e.g. to create lambda abstractions in type fields -RRB- . menus Under the two display areas , all possible commands to edit graphs and their nodes are provided -LRB- see 5.3 -LSB- Editor Commands -RSB- , page 9 -RRB- . buttons Located to the left of the graphical display , buttons and icons provide access to some commands without keyboard shortcuts as well as an easy control over the way graphs are drawn in the graphical display . There is also a ` modified ' label in the bottom left corner , appearing only when a graph has been added to the database but the latter has n't been saved on disk . Several editor windows may be open at the same time . To make better usage of the display area of the monitor , the editor windows may be iconified . In order to distinguish the various editors in an easy way , both the icons and the windows are numbered . 5.2 The Graphical Display The main area within an editor window is occupied by the graphical display of graphs . In CGE , the visual appearance of graphs may be controlled by the user , either semi-automatically or manually for full control . As in -LSB- Sow84 -RSB- , relations are displayed as circles and concepts as rectangles . Contexts and single-use types -LRB- with lambda abstractions -RRB- are also supported -LRB- see Figure 1 , page 7 -RRB- , but not coreference links -LRB- see below -RRB- . Two words have a special meaning in the context of the graphical display : node This refers to a relation , concept or context . It is possible to select , move , and perform other operations on a single node -LRB- see 5.3 -LSB- Node Menu -RSB- , page 10 -RRB- . viewer A piece of the screen used to display graphs . In this version , all graphs in the same context , and only them , are displayed in the same viewer . To make the display of conceptual graphs easier , a generic widget to handle the visualization of arbitrary graphs was created -LRB- -LSB- Qua90 -RSB- -RRB- . Each viewer is implemented through a Graph widget . This correspondance between contexts , viewers , and widgets gives a lot of flexibility , as graphs in different contexts may be displayed in different ways . The relevant attributes of the Graph widget for CGE users are : layout mode It indicates if the display of the graph is to be done automatically -LRB- by the widget -RRB- or manually -LRB- by the user , dragging the nodes with the mouse to the desired position -RRB- . layout function This is the algorithm used in automatic mode to calculate the positions of the graph 's nodes . layout style It may be one of the four available styles -LRB- left-right , right-left , top-down , bottom-up -RRB- for automatic layout . There are three pre-defined layout functions : Hierarchy This function is mainly used for hierarchical graphs and it is the one that provides the best results for conceptual graphs . Tree This function can only be used for a single graph that is in fact a tree -LRB- see Figure 2 , page 9 -RRB- . If used to display disconnected graphs , a mess will appear on the screen . Spring This is the only iterative function , i.e. the visual appearance of the graph will depend on the original position of its nodes , whereas the other functions always display the same graph in the same way . Figure 2 : Tree display In this version , CGE does n't display lines of identity -LRB- Assumption 4.2.5 -RRB- because it would require some dirty code , like having a transparent window where only the coreference links would be drawn , as they span over several contexts . The problem is that whenever there 's an automatic relayout of the nodes , this widget would have to somehow notice it and recalculate all links accordingly . To match the speed of the Graph widget , which is written in C , would require to code it in C , too . It is therefore easier to wait for the plans for the Graph widget to become true , namely to have direct access from Prolog to the primitives to draw arcs . To the left of the graphical display there 's a layout control box consisting of eight buttons . The top one controls the layout mode , the next three control the layout function and the bottom four arrows control the layout style . The buttons have a twofold purpose . By clicking on them with the mouse , the user may set the corresponding attributes in the selected viewer -LRB- s -RRB- . On the other hand , the state of the buttons reflects the attributes of the selected viewers . 5.3 The Editor Commands The commands available in CGE may be issued from the keyboard , selecting an entry of a menu , or clicking with the mouse on an icon . Often , there are at least two ways to invoke the same command . Most of the commands use dialog boxes -LRB- to interact with the user -RRB- and selections -LRB- to show the graphs or nodes on which to operate -RRB- . There are several types of the former -LRB- acknowledge dialogs to display error messages , choice dialogs to present a set of options , etc. -RRB- , taking into account the various needs for user input -LRB- see Figure 3 , page 10 -RRB- . Also , two kinds of selections are provided to enable some commands to distinguish their operands -LRB- e.g. the insertion operation needs to know the graph to insert and the context in which to insert it -RRB- . All available commands , except those involving the way graphs are displayed , appear in the following three menus : Editor Menu The Editor Menu contains commands that do n't belong to the conceptual graph formalism , like loading and saving a graph , change the current graph database , deleting an arbitrary graph , and quitting the editor . Figure 3 : A confirm dialog box Graph Menu The commands in this menu operate on whole graphs . They include the canonical formation rules -LRB- Assumption 3.4.3 -RRB- and the propositional rules of inference -LRB- Assumption 4.3.1 -RRB- . Furthermore , there are commands to compute the depth of a graph -LRB- Definition 4.2.4 -RRB- and to check whether one graph is a generalization , a specialization -LRB- Definition 3.5.1 -RRB- , or a copy of another graph . Node Menu This menu groups all commands that operate only on relations and concepts . They are divided into three groups : the restrict operation from the canonical formation rules -LRB- Assumption 3.4.3 -RRB- , type expansions -LRB- Definitions 3.6.6 , 3.6.7 , and 3.6.15 -RRB- , and referent expansions and contractions -LRB- -LSB- Sow84 , section 3.3 -RSB- -RRB- . There is n't an \ undo '' command , but most of the implemented ones have a counterpart , like drawing vs. erasing a double negation , iteration vs. deiteration , etc. . To cancel the effect of any operation , the ` Clear Graph ' command is provided , but it should be used only in case of a mistake -LRB- e.g. the wrong graphs were joined -RRB- as it is not a canonical or propositional rule . 5.4 Future Work The Graph widget needs some recoding before being of practical use for the display of conceptual graphs : the algorithms must take the size of the nodes into account , make better usage of space , and maybe a new one must be developped for nested graphs . Besides these problems , not concerning directly CGE , some more specific enhancements are : ffl Make a type lattice -LRB- Assumption 3.2.5 -RRB- editor/viewer which would provide an easy way to create new types or selecting existing ones . ffl Show coreference links and enable the user to edit them in a simple way -LRB- e.g. by pointing and dragging -RRB- . ffl Enable the user to choose for each context whether it should be displayed in normal or reduced size , thus improving the effective usage of the available display area . ffl Enable the user to do some things -LRB- e.g. lambda abstraction in the type field -RRB- in a more graphical way , instead of having to write the corresponding linear notation for it . 6 A Semantic Interpreter As an example of a possible use of conceptual graphs , a small semantic interpreter for Portuguese sentences was built . The approach taken is similar to the one described in -LSB- SW86 -RSB- : there 's a canonical graph associated to each possible meaning of a word and the graph corresponding to a particular part of the sentence is obtained by combining the graphs for its subparts , following the structure of the syntactic tree . The syntactic parsing of the sentences is done by the Prolog grammar described in -LSB- LR90 -RSB- , which covers a vast portion of the Portuguese language . In its actual state , the semantic interpreter only \ understands '' a tiny subset of that fragment . On the other hand , it performs some deduction on the database constructed from the input sentences . The interpreter understands three types of sentences : Declarative sentences They denote assertions to be added to the database , stating a simple negative or positive fact -LRB- e.g. \ The cat does n't eat . '' -RRB- or a rule of the form \ A if B '' , where A and B are simple facts -LRB- e.g. \ The cat eats the mouse if it is hungry . '' -RRB- . The former are represented by negative or positive contexts , respectively , and the latter uses the ` IMP ' relation -LRB- see -LSB- Sow84 , section 4.2 -RSB- -RRB- . It is to be noted that as there is no anaphora resolution , sentences like \ The cat eats the mouse if it does n't run away . '' must be rewritten into \ The cat eats the mouse if the mouse does n't run away . '' and even in this case no coreference link between the two ` MOUSE ' concepts will be drawn . Therefore , whenever the individual is not specified the interpreter makes the simplifying assumption that the user is always referring to the same one . Deductions upon graphs are made according to the following rules , where G/X is a fact represented by a proposition of polarity X containing graph G , A -RRB- B represents the rule \ B if A '' , and < = denotes specialization -LRB- Definition 3.5.1 -RRB- : ffl G1/pos and G2/pos -RRB- G3/X implies G3/X if G1 < = G2 . ffl G1/neg and G2/neg -RRB- G3/X implies G3/X if G2 < = G1 . ffl G1/pos and G2/X -RRB- G3/neg implies G2 / : X if G1 < = G3 . ffl G1/neg and G2/X -RRB- G3/pos implies G2 / : X if G3 < = G1 . The first two rules are modus ponens , the other two implement modus tolens . For example , the third rule states : if the database contains a positive fact A and a rule stating \ not C if B '' , then the fact \ not B '' will also be in the database if C is a generalization of A. Whenever a new fact or rule is input by the user , the interpreter tries to match it with the antecedent and consequent of every rule in order to determine if modus ponens or modus tolens may be applied . This process is recursively applied to every deduced fact . As soon as a fact -LRB- deduced or not -RRB- is about to be added , the database is searched for a more general one stating the opposite . If such a fact is found , the sentence must be incoherent with the previously stated premisses , forcing the interpreter to issue a message and to retract all the facts asserted during the deduction process . Interrogative sentences They may be simple questions -LRB- e.g. \ What does the cat eat ? '' -RRB- or of the form \ A if B '' stating hypotheses -LRB- e.g. \ Does the mouse die if the cat eats ? '' -RRB- . The graph representing the question will have the same form as for a declarative sentence , whereby wh-pronouns are simply dropped or substituted by generic concepts . In the case of a simple question , looking for an answer consists in searching the database for a specialization of the graph representing the question . In the other case -LRB- \ A if B '' -RRB- , the hypothesis B is temporarily put as a normal assertion in the database -LRB- i.e. it is tested for coherence with the known facts and all possible deductions are performed -RRB- and then A is treated like a simple question . Imperative sentences They are interpreted as instructions to the interpreter . In its actual version , commands consist of a single verb in the imperative form , e.g. \ stop ! '' . The graph representing the command must be known to the interpreter , i.e. the program searches its internal command list for an exact copy of the graph . Only three different commands are known in this version : \ mostra ! '' -LRB- show -RRB- , \ apaga ! '' -LRB- erase -RRB- , \ para ! '' -LRB- stop -RRB- . The first shows one by one on demand the current facts in the database , the second clears the database , and the third is used to quit the interpreter . 6.1 Interpreting Sentences The interpreter can be used with or without CGE , the difference being how user input and program output is handled . Some of the main aspects of the interpreter are : ffl The backtracking facility of Prolog is used to find alternative syntactic and semantic representations for the sentences . ffl Fillmore 's order of preference -LRB- agent , instrument , object -RRB- is used to join the verb and subject graphs . ffl A sentence is required to contain a verb , but everything else is optional . The interpreter also handles conjunctions , prepositions , adjectives , common and proper nouns , and some types of determiners . The considered modifiers are preposition phrases and relative and subordinate clauses . ffl Relative clauses are translated into abstractions of the type corresponding to the noun they modify , like in -LSB- SW86 -RSB- . ffl Verb arguments and modifiers are distinguished in that the former restrict the concepts of the verb graph while the latter join new graphs , namely those of the modifiers . This means that the graph for the verb must already make provision for its possible arguments . If a relation is used for an argument , it wo n't be used for any modifier that may come ahead . This prevents sentences like \ : : : cuts the cheese with the knife '' to be interpreted as ` -LSB- KNIFE -RSB- < - -LRB- INST -RRB- < - -LSB- CUT -RSB- - > -LRB- INST -RRB- - > -LSB- CHEESE -RSB- ' , where \ cheese '' is the subject and \ : : : '' the topicalized object . 6.2 Some Examples The following is an excerpt of an actual session with the semantic interpreter without using CGE as the visual graph notation would take too much space . The examples illustrate mainly deduction and question answering . User input begins with ` | : ' . | : o gato joao come o rato se o rato nao fugir . -LRB- If the mouse does n't run away , John the cat will eat it . -RRB- Syntactic analysis done ! Semantic analysis done ! -LSB- PROPOSITION : -LSB- ESCAPE -RSB- - -LRB- AGNT -RRB- - > -LSB- MOUSE : # -RSB- -LRB- SRCE -RRB- - > -LSB- ENTITY -RSB- -RSB- - -LRB- IMP -RRB- - > -LSB- PROPOSITION : -LSB- EAT -RSB- - -LRB- AGNT -RRB- - > -LSB- CAT -RSB- - > -LRB- NAME -RRB- - > -LSB- `` Jo ~ ao '' -RSB- -LRB- OBJ -RRB- - > -LSB- MOUSE : # -RSB- -RSB- -LRB- NEG -RRB- . Another interpretation ? -LRB- y/n -RRB- | : n | : um rato nao foge se o rato come queijo . -LRB- If a mouse is eating cheese , it wo n't run away . -RRB- Syntactic analysis done ! Semantic analysis done ! ~ -LSB- PROPOSITION : -LSB- ESCAPE -RSB- - -LRB- AGNT -RRB- - > -LSB- MOUSE -RSB- -LRB- SRCE -RRB- - > -LSB- ENTITY -RSB- -RSB- < - -LRB- IMP -RRB- < - -LSB- PROPOSITION : -LSB- EAT -RSB- - -LRB- AGNT -RRB- - > -LSB- MOUSE : # -RSB- -LRB- OBJ -RRB- - > -LSB- CHEESE -RSB- -RSB- . Another interpretation ? -LRB- y/n -RRB- | : n | : quem come o rato se o rato comer queijo branco ? -LRB- Who eats the mouse if it eats white cheese ? -RRB- Syntactic analysis done ! Semantic analysis done ! -LSB- PROPOSITION : -LSB- EAT -RSB- - -LRB- AGNT -RRB- - > -LSB- MOUSE : # -RSB- -LRB- OBJ -RRB- - > -LSB- CHEESE -RSB- - > -LRB- ATTR -RRB- - > -LSB- WHITE -RSB- -RSB- - > -LRB- IMP -RRB- - > -LSB- PROPOSITION : -LSB- EAT -RSB- - -LRB- AGNT -RRB- - > -LSB- ANIMATE -RSB- -LRB- OBJ -RRB- - > -LSB- MOUSE : # -RSB- -RSB- . Another interpretation ? -LRB- y/n -RRB- | : n -LSB- PROPOSITION : -LSB- EAT -RSB- - -LRB- AGNT -RRB- - > -LSB- CAT -RSB- - > -LRB- NAME -RRB- - > -LSB- `` Jo ~ ao '' -RSB- -LRB- OBJ -RRB- - > -LSB- MOUSE : # -RSB- -RSB- . Another answer ? -LRB- y/n -RRB- | : y I do not know . | : os gatos que comem queijo nao comem ratos . -LRB- Cats that eat cheese do n't eat mice . -RRB- Syntactic analysis done ! Semantic analysis done ! ~ -LSB- PROPOSITION : -LSB- EAT -RSB- - -LRB- AGNT -RRB- - > -LSB- \ a -LSB- CAT : * b = * a -RSB- ; -LSB- PROPOSITION : -LSB- EAT -RSB- - -LRB- AGNT -RRB- - > -LSB- CAT : * b -RSB- -LRB- OBJ -RRB- - > -LSB- CHEESE -RSB- -RSB- -RSB- -LRB- OBJ -RRB- - > -LSB- MOUSE -RSB- -RSB- . Another interpretation ? -LRB- y/n -RRB- | : y Sorry ... 6.3 Future Work The current state of the interpreter is not very satisfactory as far as speed and flexibility are concerned . The coverage can also be much improved , but that was n't the purpose of this application . The lack of speed is mainly due to the complexity of graph operations and to the constant copying of the intermediate graphs during the process to make backtracking possible . Furthermore , the interpreter has to cope better with unknown syntactic trees , because it sometimes enters an infinite loop . Other minor aspects that could be improved upon are : ffl Answer hypothetical questions , even if the supposition is known to be false . In fact , if the database contains fact F and rule : F -RRB- G , question : F -RRB- ? wo n't get G as an answer because : F ca n't be temporarily asserted as a fact . ffl Give short answers , i.e. to a question like \ Who ate the mouse ? '' just answer \ The cat . '' instead of \ The cat ate the mouse . '' as it is done now . This will be tricky to implement , and perhaps it will be only feasible for simple questions . ffl Check if a fact is already in the database . For the moment , stating G and then asking G -RRB- ? , will assert -LRB- although temporarily -RRB- G again and perform the same deductions . However , such a case seldom occurs and it is probably not worth constantly checking for copies , which imposes some overhead . ffl Use schemata and the supertypes ' canonical graphs to increase flexibility and semantic coverage . 7 Conclusions GET is a first attempt at providing a collection of tools to work with conceptual graphs in a well known graphical environment , X-Windows . Some options had to be taken concerning some less clear points of the formalism , others were deliberately postponed until the researching community agrees on them -LRB- e.g. the referent field -RRB- . A toy semantic interpreter for Portuguese sentences was built proving that the mapping between natural language and conceptual graphs is relatively straightforward . Acknowledgements I would like to thank Irene Rodrigues for many fruitful discussions ; Salvador Abreu for answering all my questions about X-Prolog , Emacs , LaTEX and the like | even the stupid ones ; Paulo Quaresma for providing his Graph widget which eased the programming of CGE ; Claudia Ventura and Sabine Gruninger for reviewing this document and polishing my English ; and , specially , Gabriel Lopes for his continuous support . References -LSB- Abr89a -RSB- Salvador Pinto Abreu . A Prolog Interface to the X Window System . In David Bowen and Arun Lakhotia , editors , Logic Programming Environments : The Next Generation , 1989 . -LSB- Abr89b -RSB- Salvador Pinto Abreu . ALPES X-Prolog Programming Manual . Centro de Intelig ^ encia Artificial , UNINOVA , 1989 . -LSB- CM87 -RSB- W. F. Clocksin and C. S. Mellish . Programming in Prolog . Springer Verlag , 3rd edition , 1987 . -LSB- Con89 -RSB- Alpes Consortium . ALPES Final Report . Universidade Nova de Lisboa , 1989 . -LSB- GTS89 -RSB- David A. Gardiner , Bosco S. Tjan , and James R. Slagle . Extended conceptual structures notation . In Proc . of the 4th Annual Workshop on Conceptual Structures , 1989 . -LSB- LR90 -RSB- Gabriel Pereira Lopes and Irene Pimenta Rodrigues . Descri?c ~ ao parcial da sintaxe do Portugu ^ es . Technical report , CRIA/UNINOVA , June 1990 . -LSB- MAS88 -RSB- Joel McCormack , Paul Asente , and Ralph R. Swick . X Toolkit Intrinsics | C Language Interface . MIT Project Athena , 1988 . For X Version 11 , Release 4 . -LSB- Pet89 -RSB- Chris D. Peterson . Athena Widget Set | C Language Interface . MIT X Consortium , 1989 . For X Version 11 , Release 4 . -LSB- PH90 -RSB- Heather D. Pfeiffer and Roger T. Hartley . Additions for set representation and processing to conceptual programming . In Proc . of the 5th Annual Workshop on Conceptual Structures , pages 131 -LCB- 140 , 1990 . -LSB- PS87 -RSB- Fernando Pereira and Stuart Shieber . Prolog and Natural Language Analysis , volume 10 of CSLI Lecture Notes . Center for the Study of Language and Information , 1987 . -LSB- Qua90 -RSB- Paulo Miguel Quaresma . Graph | um novo widget para o X-Windows . Dep . de Inform?atica , FCT-UNL , September 1990 . -LSB- Sow84 -RSB- John F. Sowa . Conceptual Structures : Information Processing in Mind and Machine . The System Programming Series . Addison-Wesley Publishing Company , 1984 . -LSB- Sta85 -RSB- Richard Stallman . GNU Emacs Manual . Free Software Foundation , 1985 . -LSB- SW86 -RSB- John F. Sowa and Eileen C. Way . Implementing a semantic interpreter using conceptual graphs . IBM Journal Res . Develop. , 30 -LRB- 1 -RRB- :57 -LCB- 69 , January 1986 . -LSB- Wer91a -RSB- Michel Wermelinger . GET : Graph Editor and Tools | the incomplete reference . Technical Report RT-3 / 91-CIUNL , Centro de Inform?atica da UNL , January 1991 . -LSB- Wer91b -RSB- Michel Wermelinger . GET | some notes on the implementation . Technical Report RT-4 / 91-CIUNL , Centro de Inform?atica da UNL , January 1991 .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

conceptual structures
graphical interfaces

knowledge representation
natural language processing


Kimmel Bruckstein 14 4 . a . When propagating an equal height contour with respect to ^ l on P l , the underlying image which specifies the propagation velocity shifts in time with a constant velocity of sin . b . The constant velocity of the shifting image may be converted to a constant shifting of the evolving curve . 5 . Reconstruction of a synthetic images given on 64 ? 64 grid . a . The light source direction is from above -LRB- ^ l = ^ z -RRB- . b . A tilted light source direction l = -LRB- ? 0:364 ; ; 1 -RRB- -LRB- = 20o -RRB- . c . The light source direction is from above -LRB- ^ l = ^ z -RRB- . d . A tilted light source direction l = -LRB- ? 0:364 ; ; 1 -RRB- -LRB- = 20o -RRB- . 6 . C l is evolving with a constant unit velocity along its normals directions , and C is the projection of C l to P . The propagation rule that determines the evolution of C is specified by a velocity vn along its normals . vn is achieved by first projecting the C l normals to P and then taking only the planar components of that projection . Kimmel Bruckstein 13 -LSB- 13 -RSB- A. P. Pentland , \ Local shading analysis , '' IEEE Trans . on PAMI , vol . 6 , no. 2 , pp. 170 -LCB- 187 , 1984 . Also in Horn and Brooks , 1989 . -LSB- 14 -RSB- P. Dupuis and J. Oliensis , \ An optimal control formulation and related numerical methods for a problem in shape reconstruction , '' Department of Computer Science COINS TR93-03 , University of Massachusetts at Amherst , Amherst MA , 01003 , 1993 . -LSB- 15 -RSB- J. A. Sethian , \ A review of recent numerical algorithms for hypersurfaces moving with curvature dependent speed , '' J. of Diff . Geom. , no. 33 , pp. 131 -LCB- 161 , 1989 . -LSB- 16 -RSB- C. L. Epstein and M. Gage , \ The curve shortening flow , '' in Wave Motion : Theory , Modeling , and Computation -LRB- A. Chorin and A. Majda , eds . -RRB- , New York : SpringerVerlag , 1987 . -LSB- 17 -RSB- G. Sapiro and A. Tannenbaum , \ Affine invariant scale -LCB- space , '' to be published in Int . J. Comp . Vision. , 1993 . -LSB- 18 -RSB- R. Kimmel , A. Amir , and A. M. Bruckstein , \ Finding shortest paths on graph surfaces , '' Center for Intelligent Systems Report CIS # 9301 , Technion -LCB- Israel Institute of Technology , Israel , January 1993 . -LSB- 19 -RSB- R. J. LeVeque , Numerical Methods for Conservation Laws . Lectures in Mathematics , Basel : Birkhauser Verlag , 1992 . -LSB- 20 -RSB- J. A. Sethian and J. Strain , \ Crystal growth and dendritic solidification , '' J. of Computational Physics , no. 98 , 1992 . -LSB- 21 -RSB- S. Osher and C. W. Shu , \ High -LCB- order essentially nonoscillatory schemes for Hamilton -LCB- Jacobi equations , '' SIAM J. Numer . Analy. , vol . 28 , pp. 907 -LCB- 922 , August 1991 . Figure Captions 1 . The case where the viewer and the light source are at different directions -LRB- ^ l < > ^ z -RRB- is solved by propagating curves which are \ equal height '' contours with respect to the given light source direction -LRB- ^ l -RRB- . 2 . On a patch of a surface , the brightness under Lambertian shading rule is given by the cosine of the angle between the surface normal and the light source direction E = cos = ^ N ? ^ l. 3 . a . When the light source is from above -LRB- ^ l = ^ z -RRB- the ambiguity cone is directed upwards . b. For other light source directions the ambiguity cone is tilted towards the light source . Kimmel Bruckstein 12 We have noted in Section 4 that the planar unit normal to any level set of a given continuous function OE can be written as ^ n = ? rOE krOEk . Using this notation in equation -LRB- 14 -RRB- we obtain vn = 1 q OE2x + OE2y q 1 + p2l + q2l q OE2x -LRB- 1 + q2l -RRB- + OE2y -LRB- 1 + p2 -RRB- ? OExOEy2plql : References -LSB- 1 -RSB- B. K. P. Horn , \ Obtaining shape from shading information , '' in The Psychology of Computer Vision -LRB- P. H. Winston , ed . -RRB- , pp. 115 -LCB- 155 , New York : McGraw Hill , 1987 . -LSB- 2 -RSB- B. K. P. Horn , \ Height and gradient from shading , '' Int . J. of Comput . Vision , vol . 5 , pp. 37 -LCB- 75 , 1990 . -LSB- 3 -RSB- E. Rouy and A. Tourin , \ A viscosity solutions approach to shape-from-shading , '' SIAM . J. Numer . Analy. , vol . 29 , pp. 867 -LCB- 884 , June 1992 . -LSB- 4 -RSB- S. Osher and L. Rudin , \ Rapid convergence of approximate solutions to shape-fromshading problem , '' tech . rep. , Cognitech Inc. , 1993 . never appeared . -LSB- 5 -RSB- A. M. Bruckstein , \ On shape from shading : Some theoretical considerations methods , '' EE Report 524 , Technion -LCB- Israel Institute of Technology , June 1985 . -LSB- 6 -RSB- A. M. Bruckstein , \ On shape from shading , '' Comput . Vision Graphics Image Process. , vol . 44 , pp. 139 -LCB- 154 , 1988 . -LSB- 7 -RSB- S. J. Osher and J. A. Sethian , \ Fronts propagating with curvature dependent speed : Algorithms based on Hamilton-Jacobi formulations , '' J. of Comp . Phys. , vol . 79 , pp. 12 -LCB- 49 , 1988 . -LSB- 8 -RSB- R. Kimmel , \ Shape from shading via level sets , '' M.Sc . thesis -LRB- in Hebrew -RRB- , Technion - Israel Institute of Technology , June 1992 . -LSB- 9 -RSB- R. Kimmel and A. M. Bruckstein , \ Shape from shading via level sets , '' Center for Intelligent Systems Report CIS # 9209 , Technion -LCB- Israel Institute of Technology , Israel , June 1992 . -LSB- 10 -RSB- R. Kimmel , K. Siddiqi , B. B. Kimia , and A. M. Bruckstein , \ Shape from shading : Level set propagation and viscosity solutions , '' submitted , 1993 . -LSB- 11 -RSB- S. J. Osher , \ A level set formulation for the solution of the Dirichlet problem for Hamilton -LCB- Jacobi equations , '' Department of Math , UCLA CAM Report 92-39 , UCLA , Los Angeles , CA 90024-1555 , 1992 . -LSB- 12 -RSB- C. H. Lee and A. Rosenfeld , \ Improved methods of estimating shape from shading using the light source coordinate system , '' Arificial Intelligence , vol . 26 , pp. 125 -LCB- 143 , May 1985 . Kimmel Bruckstein 11 Appendix Planar Normal Velocity Calculation Given the simple propagation rule on the plane defined by the light source direction P l C lt = ^ t ? ^ l ; we want to project this evolving curve to the -LRB- x ; y -RRB- - plane P , and take only the normal component of the velocity vector of the curve in P , see Figure 6 . The above equation may be written as -LRB- xt ; yt ; zt -RRB- = -LRB- xu ; yu ; zu -RRB- q x2u + y2u + z2u ? -LRB- ? pl ; ? ql ; 1 -RRB- q 1 + p2l + q2l = -LRB- w1 ; w2 ; w3 -RRB- ; where u is the parameter of the curve C l -LRB- u ; t -RRB- = -LRB- x -LRB- u ; t -RRB- ; y -LRB- u ; t -RRB- ; z -LRB- u ; t -RRB- -RRB- . The projection of this curve evolution on the -LRB- x ; y -RRB- -LCB- plane may be written using Ct = vn ^ n , or -LRB- xt ; yt -RRB- = vn -LRB- ? yu ; xu -RRB- q x2u + y2u : We are looking for the velocity vn , which is the projection of -LRB- w1 ; w2 -RRB- on the planar normal ^ n . This may be written as vn = h -LRB- w1 ; w2 -RRB- ; ^ ni = ? w1yu + w2xu q x2u + y2u : By calculating the vector product and eliminating -LRB- due to the projection -RRB- the ^ z component , we get -LRB- w1 ; w2 -RRB- = -LRB- yu + qlzu ; ? xu ? plzu -RRB- q x2u + y2u + z2u q 1 + p2l + q2l : Now , we are ready to calculate vn , vn = -LRB- yu + qlzu ; ? xu ? plzu -RRB- q 1 + p2l + q2l q x2u + y2u + z2u ? -LRB- ? yu ; xu -RRB- q x2u + y2u : Using the chain rule zu = zxxu + zyyu = plxu + qlyu , we get vn = vuut x2u -LRB- 1 + p2l -RRB- + y2u -LRB- 1 + q2l -RRB- + 2plqlxuyu -LRB- 1 + p2l + q2l -RRB- -LRB- x2u + y2u -RRB- : Writing the planar normal as its components ^ n = -LRB- n1 ; n2 -RRB- = -LRB- ? yu ; xu -RRB- px2u + y2u , we conclude with vn = 1 q 1 + p2l + q2l q n21 -LRB- 1 + q2l -RRB- + n22 -LRB- 1 + p2l -RRB- ? n1n22plql : -LRB- 14 -RRB- Kimmel Bruckstein 10 If -LRB- OEni ; j ? OEn ? 1 i ; j < 0 -RRB- then heighti ; j = ? -LSB- ? t -LRB- n ? OEni ; j OEni ; j ? OEn ? 1 i ; j -RRB- -RSB- q 1 + p2l + q2l + i?xpl + j?yql . Using the above procedure each grid point gets its height at the \ time '' when the OE functions ' -LCB- zero level passes through it . 5 Examples and Results We demonstrate the performance of the proposed algorithm by applying it to several shaded images . The images were generated for surfaces assumed to be Lambertian , the size of these images being quite small -LRB- 64 ? 64 pixels -RRB- . The initialization is achieved by using gray -LCB- level thresholding to specify initial \ singular areas '' . The reconstruction of the synthetic models is displayed in Figure 5 where the upper right is the given shading image , the upper left surface is the original model , the lower left surface is the reconstructed surface , and at the lower right is a description of the curve evolution staring from the gray level contour . For each surface model two light direction were applied , the first is the vertical ^ l = -LRB- 0 ; ; 1 -RRB- -LRB- that is = o -RRB- , and the second is a tilted light source direction of l = -LRB- ? 0:364 ; ; 1 -RRB- -LRB- that is = 20o -RRB- . Figures 5 . a and 5 . b show two examples of reconstruction for a \ volcano '' surface , starting from a small curve around the singular area at the top of the mountain . In the equal height contours picture of the reconstructed surface one can observe the way topological problems like the saddle on the lowest left corner are inherently solved through this \ down -LCB- going '' process . Note however that starting at the base of the volcano would require special treatment to proceed beyond the saddle point , such issues are discussed in -LSB- 8 , 9 -RSB- . Figures 5.c and 5 . d , demonstrate the reconstruction of part of a sphere surface . 6 Concluding Remarks We have described a method for recovering the shape of an object from its shaded image by an equal height contour propagation method and gave a new formulation to the general light source direction problem . A numerically stable implementation in which topological problems in the propagated height contours are often inherently avoided was presented . In this method shocks , cusps and other singularities formed in the contours are also readily dealt with in an efficient numerical scheme originally developed to solve wavefront propagation . Acknowledgments We thank Prof. James Sethian and Prof. Stanley Osher for several interesting discussions on numerical methods of front evolution . We thank Prof. Moshe Israeli , Prof. Allen Tannenbaum and Prof. J. Rubinstein for their help in introducing us to the world of numerical analysis and moving surfaces . And last but not least , it is a great pleasure to thank Dr. Guillermo Sapiro for the long and interesting discussions we had . Kimmel Bruckstein 9 and the same for OE2y . Here Dx?OEni = OEni ? OEni ? 1 ? x , Dx + OEni = OEni +1 ? OEni ? x , and the same for the y derivatives . The minmod -LSB- 19 -RSB- function is used to approximate OEx and OEy : OExjx = i?x ; y = j?y ss minmod -LRB- Dx?OEi ; j ; Dx + OEi ; j -RRB- ; and the same for OEy , where minmod -LRB- a ; b -RRB- = -LRB- sign -LRB- a -RRB- min -LRB- jaj ; jbj -RRB- if ab > otherwise All the above approximations yield a so -LCB- called first order numerical scheme that approximates equation -LRB- 13 -RRB- . This scheme automatically enforces the entropy condition , and frees one from the need to take care of topological changes . In fact this formulation deals with the topology of all up going -LRB- or down going -RRB- surfaces with respect to the light source direction without any external control or outside interference . The algorithm also deals with shock formation in the propagating contours which indicates sharp corners in the reconstructed surfaces within the numerical flow . One of the great advantages of the Eulerian formulation is that the coordinate system of the propagated OE function is fixed , thereby avoiding the stability problems of the Lagrangian formulation . More sophisticated higher order schemes are presented in -LSB- 7 , 21 -RSB- . In our problem the velocity G is position and normal dependent . In order to avoid numerical problems in which F ! 1 , we truncate the brightness function to the maximum values of Emax < 1 , which yields Fmax = Emax p1?E2max . 4.1 Initialization and Height Assignment Every OE function which obeys the demands described earlier provides a good initialization . We present a simple way to initiate the OE function , obeying smoothness , continuity so that OE -LRB- x ; y ; 0 -RRB- = gives the initial contour . Let us define the initial height contour by first thresholding the gray -LCB- levels in the picture and separating all the \ singular '' areas -LRB- the singular areas is the set of points in the image for which E -LRB- x ; y -RRB- ss 1 , at such points the normal is known to be ^ l and therefore may be used as boundary conditions or candidates to start the propagation -RRB- . Then use the gray -LCB- level function to initiate the OE function in a simple way . For example , if the singular areas are defined by f -LRB- x ; y -RRB- : E -LRB- x ; y -RRB- > Tg -LRB- where T 2 -LRB- 0 ; 1 -RRB- is the selected threshold -RRB- , then the first level contour can be approximated as the level set E -LRB- x ; y -RRB- = T . In this case we can take OE -LRB- x ; y ; 0 -RRB- = E -LRB- x ; y -RRB- ? T near the selected singular areas , as the required initialization , making direct use of the continuity of the gray levels in the picture , without any extra calculations . This is the initialization method used in our examples . After initialization has been completed the OE is propagated according to the above described algorithm . Our goal is finding the elevation at each grid point while the OE function is propagated on the grid . A way to achieve accurate results using a simple linear interpolation is as follows : At every iteration step , for each grid point , check Kimmel Bruckstein 8 in the exterior of the level set OE -LRB- x ; y ; 0 -RRB- = 0 . Furthermore OE has to be smooth and Lipschitz continuous . The idea is to determine an evolution of the surface OE -LRB- x ; y ; t -RRB- so that the level sets OE -LRB- x ; y ; t -RRB- = provide the height contours C -LRB- t -RRB- as if propagated by -LRB- 8 -RRB- and obeying the entropy condition . If OE -LRB- x ; y ; t -RRB- = along C -LRB- t -RRB- then , by the chain rule , we have OEt + rOE ? Ct -LRB- t -RRB- = : -LRB- 9 -RRB- The scalar velocity of each curve point in its normal direction is G = Ct -LRB- t -RRB- ? ^ n -LRB- t -RRB- : -LRB- 10 -RRB- In our case the velocity is given by the scalar function G -LRB- x ; y ; ^ n -RRB- as defined in equation -LRB- 7 -RRB- , which is a function of the local image brightness and the given normal of the propagating curve . The gradient rOE ? -LRB- @@ x ; @@ y -RRB- OE is always normal to the curve given by OE -LRB- x ; y ; t -RRB- = so that ^ n -LRB- s ; t -RRB- = ? rOE krOEk , the minus sign indicating the inward direction of propagation , hence G = Ct ? ? rOE krOEk : -LRB- 11 -RRB- Substituting this into -LRB- 9 -RRB- yields OEt ? GkrOEk = ; -LRB- 12 -RRB- or explicitly OEt = 1 q 1 + p2l + q2l ? -LRB- 13 -RRB- ? F -LRB- x ; y -RRB- q OE2x -LRB- 1 + q2l -RRB- + OE2y -LRB- 1 + p2l -RRB- ? OExOEy2plql ? -LRB- plOEx + qlOEy -RRB- ? : Sethian called this approach based on propagating the OE function , Eulerian , since the coordinates here are the natural physical coordinates -LRB- x ; y -RRB- . Therefore , if we have a surface OE propagating according to -LRB- 13 -RRB- with the level set OE -LRB- x ; y ; 0 -RRB- = coinciding with C -LRB- 0 -RRB- , then OE -LRB- x ; y ; t -RRB- = will produce C -LRB- t -RRB- propagated according to -LRB- 8 -RRB- and solving topological problems . In order to drive a numerical scheme for the surface propagation equation which obeys the natural \ entropy condition '' , we modify results obtained in -LSB- 7 -RSB- -LRB- see also -LSB- 20 -RSB- -RRB- based on Hamilton Jacobi equations , weak solutions and conservation laws . Define OEni ; j ? OE -LRB- i?x ; j?y ; n?t -RRB- . We use a simple forward derivative approximation in time OEtjt = n?t ss OEn +1 ? OEn ? t ; and slope limiter approximations -LSB- 19 -RSB- for the spatial derivatives . The squared terms are approximated by OE2xjx = i?x ; y = j?y ss max -LRB- Dx + OEi ; j ; ? Dx?OEi ; j ; 0 -RRB- 2 ; Kimmel Bruckstein 7 Another manipulation on the evolution equation frees us from the need to take care of these problems . Instead of shifting the image forward it is possible to include a backwards propagation constant velocity term in the curve evolution rule . This way we can keep the image in its place and transfer the constant motion to the evolving curve , see Figure 4 . b . It also frees us from the need to project the heights to the P l plane , and the height at each grid point may be calculated without any interpolation . The new evolution rule is given by Ct = F -LRB- x ; y -RRB- vn ^ n ? sin -LRB- ? pl ; ? ql -RRB- q 1 + p2l + q2l ; -LRB- 5 -RRB- and again , taking only the normal component of the second term -LRB- the tangential components have no effect on traces of the evolving curves -RRB- , we obtain : Ct = @F -LRB- x ; y -RRB- vn + -LRB- pln1 + qln2 -RRB- q 1 + p2l + q2l 1 A ^ n : -LRB- 6 -RRB- where ^ n = -LRB- n1 ; n2 -RRB- . By using the vn obtained in the Appendix , we define G -LRB- x ; y ; ^ n -RRB- = 1 q 1 + p2l + q2l ? ? F -LRB- x ; y -RRB- q n21 -LRB- 1 + q2l -RRB- + n22 -LRB- 1 + p2l -RRB- ? n1n22plql + -LRB- pln1 + qln2 -RRB- ? : -LRB- 7 -RRB- Then , equation -LRB- 6 -RRB- may by written as Ct = G -LRB- x ; y ; ^ n -RRB- ^ n ; -LRB- 8 -RRB- which is again a \ Lagrangian '' formulation of the new evolution rule . Note that equation -LRB- 3 -RRB- is a special case of equation -LRB- 8 -RRB- . Direct numerical implementations of Lagrangian formulations suffer from instability and topological problems due to the fact that local representations of propagating fronts are followed , see -LSB- 7 , 15 -RSB- . Control algorithms are needed where topological changes occur . If , for example , we start with two separate closed contours that grow up to a merging point from which they continue to grow as a single contour , it is necessary to handle this merging process by an external control procedure . To avoid the various problems that occur in this approach , a novel and miraculous \ Eulerian formulation , '' described below , was developed by Osher and Sethian -LSB- 7 -RSB- . 4 Solution via the Eulerian Formulation The Eulerian scheme is a recursive procedure that propagates height contours while inherently implementing a so-called entropy condition . This entropy condition is necessary to select the \ physically correct '' propagation -LSB- 19 -RSB- among all possible weak solutions . Introduce a function OE -LRB- x ; y ; t -RRB- initialized so that OE -LRB- x ; y ; 0 -RRB- = yields the curve C -LRB- 0 -RRB- . Assume that C -LRB- 0 -RRB- is a closed curve and restrict OE to be negative in the interior and positive Kimmel Bruckstein 6 Define C -LRB- 0 -RRB- = fx -LRB- s ; 0 -RRB- ; y -LRB- s ; 0 -RRB- g as the smooth -LRB- and , in some cases , closed -RRB- initial equal height curve , and C -LRB- t -RRB- as the one -LCB- parameter family of curves generated by moving C -LRB- 0 -RRB- along its normal vector field with a speed given by F . Here , F is a given scalar function of the brightness E. Using this notation , equation -LRB- 3 -RRB- can be written as a planar curve evolution equation @@ t C -LRB- t -RRB- = F -LRB- x ; y -RRB- ? ^ n . Sethian -LSB- 15 -RSB- called the direct implementation of such propagation models \ Lagrangian '' evolution equations because the physical coordinate system moves with the propagating front . 3.1 General Light Source Direction When the light source direction is ^ l -LRB- as defined in -LRB- 1 -RRB- -RRB- , the brightness map under the Lambertian shading rule is E = ^ l ? ^ N . In this case the surface normal is on a tilted ambiguity cone as described in Figure 3 . b . In -LSB- 8 , 9 -RSB- it was discussed how to propagate an equal height contour in this situation . It was also noted of possible ambiguities in recovering the normal direction that had to be solved by adding smoothness constraints along the equal height contour or along the surface . In the rest of this section we present a different kind of curve propagation that overcomes these ambiguities and frees us from the need to consider additional constraints while reconstructing the shape . Consider the plane P l defined by the light source direction ^ l , see Figure 1 . The equal height contours with respect to this plane are given by the previous propagation rule C lt = F l -LRB- xl ; yl -RRB- ^ t ? ^ l ; where ^ t is the tangent vector to C l , and F l is the velocity which is a function of the image as it would have been seen from the light source direction . This evolution may be projected to the -LRB- x ; y -RRB- -LCB- plane by first projecting the velocity and then taking only the normal component of the projected velocity - vn , since in the case of planar curve propagation the tangent component of the velocity affects only the internal parameterization of the curve while the trace of the curve is determined only by the normal component , see -LSB- 16 , 17 -RSB- for justification . The projected evolution is then given by Ct = ~ F vn ^ n ; where the correction factor vn is calculated in the Appendix -LRB- see also -LSB- 18 -RSB- for a similar approach to solve the shortest path problem on surfaces -RRB- . The fact that the contours lie on parallel planes , constant heights with respect to ^ l , enables us to predict the \ deformation '' in time of ~ F . Define cosfi = ^ z ? ^ l -LRB- where ^ z ? -LRB- 0 ; ; 1 -RRB- -RRB- , then ~ F -LRB- x ; y ; t -RRB- = F -LRB- x ? tpl = q 1 + p2l + q2l ; y ? tql = q 1 + p2l + q2l -RRB- , which means that the image shifts in time along the direction defined by -LRB- ? pl ; ? ql -RRB- = q p2l + q2l with a constant velocity of sin , see Figure 4 . a . This formulation forces us to interpolate the image intensity between the pixels as the image moves in time . Another problem is the fact that we calculate the heights with respect to P l so that in the last stage we should project the heights from P to P l , or stretch the elevation array given on a uniform grid in P with respect to the light source direction . This stretching results in a non uniform elevation array . Kimmel Bruckstein 5 3 Shape from Shading via Equal -LCB- Height Contours Let us first briefly review Bruckstein 's shape from shading approach , -LRB- see -LSB- 5 , 6 -RSB- -RRB- . An equal -LCB- height contour or a level curve is a continuous curve in the -LRB- x ; y -RRB- -LCB- plane on which the function z -LRB- x ; y -RRB- is constant . Defining fx -LRB- s -RRB- ; y -LRB- s -RRB- g , s 2 -LSB- 0 ; S -RSB- , as the parametric representation of the contour , we have @z @s = @z @x @x @s + @z @y @y @s = or1 p xs + q ys = : The unit normal to the equal height contour on the -LRB- x ; y -RRB- -LCB- plane is given by ^ n -LRB- s -RRB- = 1 q x2s -LRB- s -RRB- + y2s -LRB- s -RRB- -LRB- ys -LRB- s -RRB- ; ? xs -LRB- s -RRB- -RRB- : Clearly ^ n is in the direction of the projection of ^ N on the image plane . Consider the case where ^ l = ^ z -LRB- that is ^ l = -LRB- 0 ; ; 1 -RRB- -RRB- and define dz as the height we climb while progressing a distance D in the normal direction ^ n in the -LRB- x ; y -RRB- -LCB- plane . From elementary geometry we have D = dz cot ff ; where is the surface orientation angle . Under the simple Lambertian shading rule where R -LRB- p ; q -RRB- = cos = 1 p1 + p2 + q2 , we have D = dz cot = dz 1 pp2 + q2 = dz E p1 ? E2 : If , from the first contour , we uniformly climb dz , we get to the next equal height contour via fx -LRB- s ; dz -RRB- ; y -LRB- s ; dz -RRB- g = fx -LRB- s ; 0 -RRB- ; y -LRB- s ; 0 -RRB- g + D -LRB- s -RRB- ? ^ n This yields the propagation of the equal height contours as a nonlinear initial value P.D.E. problem . Given fx -LRB- s ; 0 -RRB- ; y -LRB- s ; 0 -RRB- g the evolution equations are -LSB- 5 , 6 -RSB- -RRB- : 8 >> > < >> > : xt -LRB- s ; t -RRB- = F -LRB- x ; y -RRB- ys px2s + y2s yt -LRB- s ; t -RRB- = F -LRB- x ; y -RRB- ? xs px2s + y2s -LRB- 3 -RRB- where t ? z and F -LRB- x -LRB- s ; t -RRB- ; y -LRB- s ; t -RRB- -RRB- = E -LRB- x -LRB- s ; t -RRB- ; y -LRB- s ; t -RRB- -RRB- q 1 ? E2 -LRB- x -LRB- s ; t -RRB- ; y -LRB- s ; t -RRB- -RRB- : -LRB- 4 -RRB- 1The subscripts s ; t ; x ; y ; refer to partial derivatives , -LRB- e.g. xt ? @x = @t -RRB- , and the subscripts -LRB- and superscripts -RRB- l ; n refer to the light source direction and the planar normal , respectively . Kimmel Bruckstein 4 is to recover the depth function z -LRB- x ; y -RRB- , from the image E -LRB- x ; y -RRB- . Let us first specify the surface orientation using the components of the surface gradient p = @z = @x and q = @z = @y . The surface unit normal can then be written as ^ N = 1 p1 + p2 + q2 -LRB- ? p ; ? q ; 1 -RRB- T : In case of a surface with so -LCB- called Lambertian , or diffuse reflection properties and uniform illumination , E -LRB- x ; y -RRB- is proportional to the cosine of the angle between the surface normal ^ N and the direction to the light source ^ l -LRB- see Figure 2 -RRB- . In the general case we define the light source direction as ^ l = 1 q 1 + p2l + q2l -LRB- ? pl ; ? ql ; 1 -RRB- T : -LRB- 1 -RRB- The dependence between brightness and surface orientation -LRB- the reflectance map -RRB- can be written as a function mapping the surface normal direction ^ N to the brightness image E -LRB- x ; y -RRB- as follows E -LRB- x ; y -RRB- = Function of -LRB- ^ N -RRB- = R -LRB- p -LRB- x ; y -RRB- ; q -LRB- x ; y -RRB- -RRB- : -LRB- 2 -RRB- This is the image irradiance equation . For example in the Lambertian case we have E -LRB- x ; y -RRB- = R -LRB- p ; q -RRB- = ^ l ? ^ N = 1 + plp + qlq p1 + p2 + q2q 1 + p2l + q2l In this case the normal direction lies on an ambiguity cone whose main axis is directed towards the light source , see Figure 3 . b. For the simple case where ^ l = -LRB- 0 ; ; 1 -RRB- T we have R -LRB- p ; q -RRB- = 1 p1 + p2 + q2 ; and the ambiguity of the surface normal direction is an upward directed cone , see Figure 3 . a. Equation -LRB- 2 -RRB- is a nonlinear partial differential equation that has to be satisfied by the surface z -LRB- x ; y -RRB- . Therefore , solving the shape from shading problem amounts to solving a nonlinear partial differential equation . Clearly boundary conditions are necessary . Any shape from shading scheme reconstructs a function z -LRB- x ; y -RRB- by implicitly or explicitly recovering its normal ^ N -LRB- x ; y -RRB- everywhere . The surface normal at each point is represented by two numbers , and the only constraint we have so far is the image 's irradiance equation -LRB- 2 -RRB- . The two variables representing the surface normal direction at each point can only be computed by using more than one equation . The \ art '' of recovering a shape from its shaded image requires introducing boundary conditions and local constraints that follow from reasonable assumptions concerning the relation of each point on the surface to its neighborhood . Kimmel Bruckstein 3 1 Introduction The problem of reconstructing three dimensional objects from their shaded images has been extensively investigated by computer vision researchers over the last two decades , starting from the classic \ shape from shading '' work of Horn -LSB- 1 -RSB- . Until recently iterative schemes to solve this problem were derived by regularization versions of the direct problem formulation , see -LSB- 2 -RSB- for a survey on these methods . A new iterative method was recently suggested by Rouy , Tourin , Osher and Rudin based on viscosity solutions to partial differential equations , -LSB- 3 , 4 -RSB- . They represented the shape from shading problem as a first order Hamilton -LCB- Jacobi equation . Viscosity solutions to the Hamilton -LCB- Jacobi equation guarantee uniqueness and existence . A direct approach to solving the shape from shading problem based on characteristic strip expansion method was provided by Horn in his original work . A different direct approach , using equal -LCB- height contour propagation , was suggested by Bruckstein -LSB- 5 , 6 -RSB- . An accurate numerical scheme for implementing this method based on a level set propagation algorithm devised by Osher and Sethian -LSB- 7 -RSB- was then developed in -LSB- 8 , 9 -RSB- and was also compared with viscosity solutions and other traditional methods in -LSB- 10 -RSB- . This novel level set based formulation for the equal height contour propagation method for the shape from shading problem turned out to be a general method for solving Dirichlet problems for Hamilton -LCB- Jacobi equations , see S. Osher -LSB- 11 -RSB- . In this paper we first summarize the level set based numerical approach to the equal height contour propagation method for solving the shape from shading problem . As mentioned above it is based on some nice results on numerical approximation of front propagation , published by Osher and Sethian in -LSB- 7 -RSB- . Then , we derive the contour propagation equation for the case of general light source direction , so that the contours are \ equal -LCB- height '' with respect to the light source direction instead of the height axis , see Figure 1 . This new formulation solves some ambiguity problems that were encountered in our previous work -LSB- 8 , 9 , 10 -RSB- . Light source coordinate were used in -LSB- 12 -RSB- to improve the shape from shading method of -LSB- 13 -RSB- , and later in -LSB- 14 -RSB- as a natural coordinate system with respect to the singular points -LRB- the brightness points in image -RRB- . In the next section we give a formal definition of the shape from shading problem under Lambertian reflectance . In section 3 , the equal height contour approach to the shape from shading problem is summarized and the new formulation to the general light source direction is presented . Section 4 , presents the level set based algorithm for tracking the equal height contours . Section 5 , presents results of applying the new algorithm to some shading images . 2 Problem Formulation and the Reflectance Map Suppose we are given a continuous function of two variables , z -LRB- x ; y -RRB- describing the surface of an object . The shaded image of that surface is defined as a brightness distribution E -LRB- x ; y -RRB- . The brightness values are defined by the properties of the surface , by its orientation at each point and by the illumination . The brightness E -LRB- x ; y -RRB- is determined via a so -LCB- called shading rule or reflectance map , which characterizes the surface properties and provides an explicit connection between the image and the surface orientation . The shape from shading problem Kimmel Bruckstein 2 Abstract A new algorithm for recovering depth to a Lambertian surface given its gray -LCB- level image under uniform illumination from the viewing direction is presented . To recover depth , an almost arbitrarily initialized bivariate function is numerically propagated in time on a rectangular grid , so that a level set of this function tracks the height contours of the surface . The image shading controls the propagation of the bivariate function . When the light direction is tilted with respect to the viewing direction the problem is solved by tracking the projection of equal height contours defined with respect to the light source direction . This projection approach provides a nice solution that overcomes ambiguity problems encountered in previous work . Tracking Level Sets by Level Sets : A Method for Solving the Shape From Shading Problem Ron Kimmel Technion , Electrical Engineering Department Technion -LCB- City , Haifa 32000 , Israel Tel : + 972 4 294699 Email : ron@tx.technion.ac.il FAX : + 972 4 323041 Alfred M. Bruckstein Technion , Computer Science Department Technion -LCB- City , Haifa 32000 , Israel Email : freddy@cs.technion.ac.il

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

curve evolution
equal height contours
lambertian shading rule
level sets
shape from shading