EXPERIMENTS WITH AN ON-LINE PICTURE LANGUAGE

During the investigation of diverse pattern recognition problems it has become increasingly apparent that recognition of objects in a complex picture cannot be treated as an open loop process consisting of feature extraction and classification. A paradox is encountered in that reliable features can only be extracted if the object in the picture has been recognized, but the object cannot be recognized before its features are identified. The recognition process is better characterized by a closed-loop process containing an hypothesis testing procedure. Starting even from a single feature, which has been detected and recognized only to a certain degree as being a known feature, the machine will have to form an hypothesis as to which objects this feature may belong to and what additional information has to be gathered from the picture before the feature can be recognized with greater reliability. Only then can the machine improve on feature detection, search out other features expected to belong to the hypothesized object and decide whether its hypothesis can be maintained in the context of the given recognition problem. Some of the problems encountered in trying to realize such a system are discussed. The proposed solutions have been tested to some degree by using an online picture language program. A preliminary version of the online picture language has been completed. By using a controllable flying spot scanner the computer has direct access to a picture of about 4000 × 4000 resolution elements. The picture is treated as a read only memory and it may contain many different objects. The entire program consists of three major subdivisions labelled (a), (b), (c). (a) Atom Formation, Description and Recognition: A complicated object is fragmented (“atomized”) by using an algorithm which operates with or without guidance. If guided, the operator may only point out the approximate location of the atom that he proposes to the machine. The algorithm decides whether the area can be considered to be an atom and how much of the object it should include. A description of the atom is formed and normalized, before it is compared against already known “ideal” atoms. If the match is sufficiently good, the presented fragment is considered to be a realization of the best matching ideal atom. If the match is inadequate, the fragment becomes a new “ideal” atom. (b) Teaching of Object-Inter-relationships: When all the fragments (or a sufficient number of them) have been extracted, the operator instructs the computer how these atoms are to be interrelated for a given object. The operator can teach the machine expressions of the type: “If you see atom a1 (at location (xo, yo)), go search for atom a2 (at location (x1, y1))”, or “an object 02 consists of atoms a1, a2 and a3 (01 = a1 . a2 . a3)”, or “object 1 (01) is the same as object 2 (02), i.e., 01 = 02”. The position, size and rotation relationships between the atoms and objects are retained. (c) Recognition of Objects: In the recognition part of the system, the taught relationships are used to guide the machine from the identified atoms to the expected atoms until an object of the desired class has been identified or all the usable taught relationships have been exhausted. During running of the recognition programs the machine appeared to behave as if it “understood” the problem by going from one atom to the next in a systematic exploration of the picture. However, it also became apparent that the machine needs a goal-seeking algorithm in the recognition stage to guide it through the maze of instructions via the shortest path. At present the object fragmentation stage (a) can be run without human interference and there appear to exist no unsurmountable programming problems to automate step (b), i.e., to allow the machine to teach itself. The present programs are written for two-dimensional objects. Extension to three-dimensional objects is conceptually simple.