Method for diagram recognition by using recognizing rules and system for implementing the method

A diagram recognizing system is capable of automatically recognizing a diagram prepared, for example, through a manual operation or through drawing software, independently from a CASE tool or the like, and delivering a recognized logic to the CASE tool or the like without the aid of any manual operation. A recognizing rule for recognizing the diagram as the object of recognition is entered and stored. A symbol, a connecting line, a character string, and a relationship of connection between the symbols, each structuring the diagram, are recognized on the basis of the recognizing rule by a diagram recognizing means. Logical data, indicative of the kind of each of the symbols, the character string, and the relationship of connection between the symbols, are generated, and the logical data are converted into and generated as a predetermined file type so as to be delivered directly to the CASE tool or the like.

BACKGROUND OF THE INVENTION 
1. Field of the Invention 
The present invention relates to a diagram recognizing system and, more 
particularly, to a diagram recognizing system so adapted as to recognize a 
diagram to be employed for describing a specification for data processing, 
such as flowcharts, data flow diagrams, and the like, of documents for 
developing and maintaining a data processing system, to convert the 
recognized diagram into logical data indicative of its logical meaning, 
and to generate the logical data. 
2. Description of the Related Art 
Recently, there have been proposed and employed techniques for analyzing 
and designing a system, having both accuracy in the specification for data 
processing and ease in understanding it, by representing a procedure of 
the data processing for a data processing system or the like. 
For instance, such techniques may include a structured analysis technique 
proposed by T. DeMarco (DeMarco, T.; Structured Analysis And System 
Specification; Prentice-Hall, 1978) and a technique for information 
engineering, systematized by J. Martin (Martin, J.; Information 
Engineering, Book I: Introduction, Prentice-Hall, 1989). 
As a specific example of these techniques, there is commercially employed a 
reverse engineering system for forming an entity relationship diagram from 
definition data of DB2, IDMS, VSAM, and IMS, which are IBM's database and 
files, as a tool for forming a logical structure of data from the database 
definition data of text file data such as a source program of a developed 
information system, database definition data, job control statement, 
linkage parameter, input-output map, and the like, as indicated by Data 
Analyst of Buchman, Inc. (Sato, M.; CASE Tool; Explanation of Its 
Functions and Know How of Its Application, page 219, Nov. 1, 1989). 
On the other hand, as CASE (Computer Aided Software Engineering) tools for 
aiding operations for forming a source program on the basis of these 
techniques, there have been commercialized Excelerator (Index Technology, 
Inc.) and IEW (Information Engineering Workbench: KnowledgeWare, Inc.), 
which are employed mainly in the U.S. and Europe (Nakamura, M.: Evolving 
CASE; Exploding "Walls" Among Steps & Providing Smooth Development 
Environments: Nikkei Computer; pages 78-101, Nov. 5, 1990). 
It can be noted, however, that the CASE tools, which set fourth the 
formation of a diagram as a premise, are thought to spread even in the 
U.S. at a rate as high as 10%, so that the situation is not yet ripe in 
which system analysts and system engineers can always utilize the CASE 
tools. Hence, system analysts and system engineers are usually employing 
techniques for manually drawing diagrams with rulers, templates and the 
like and allowing operators to enter the diagrams into the CASE tools or 
for forming diagrams by making use of drawing software, which is far less 
expensive than the CASE tools, and entering the formed diagrams into the 
CASE tools. 
The diagrams formed manually or through the drawing software in the manner 
as described hereinabove, however, are independent from the CASE tools, so 
that the delivery to downstream steps, such as the verification of 
matching with the specification as the CASE tool, the editing to be done 
with the logical meaning of the diagrams taken into account, and the 
automated formation of engineering data or a source program, should be 
conducted by manual operations. Hence, these techniques present the 
problems that the verification of accuracy of conversion operations, 
compliance of the specification with the program, maintenance of the 
diagrams, and the like are rendered very difficult. 
To this end, demands have been made to develop and realize a system that 
can automatically recognize a diagram formed manually or through drawing 
software and deliver the diagram to the CASE tool without any manual 
assistance. 
It can be noted herein that, as a similar technique, for instance, Hitachi, 
Ltd. has commercialized an intelligent system for recognizing account 
books and vouchers (BELIEVE) (Tsuchiya, M.; Jimu to Keiei: An Intelligent 
System for Recognizing Account Books & Vouchers As An Instant Power 
Potential; pages 8-10, September, 1990). This system, enters accounting 
vouchers and slips as data of an image, allows a recognition of a voucher 
style and a recognition of fields, and automatically forms programs for 
implementing operations for issuing the vouchers and printing the books. 
Further, Japanese Patent Publication Laid-open (kokai) No. 71,366/1990 
discloses a system for forming an E-R model, which is so adapted as to 
enter an entity relationship diagram (an E-R model diagram) for forming a 
model of data within an information system, as data of an image, and to 
allow a graphic recognition to thereby form the E-R model in an automatic 
way. 
In addition, as indicated by M. Hirata (An Input System for Automatically 
Entering Drawings of Logical Circuits: Demonstrating Its Effect by 
Restricted Uses; Nikkei Computer Graphics; page 18, February, 1988), a 
system for automatically recognizing a drawing of a logical circuit, 
developed by Matsushita Electric Industries, Co., Ltd., is so adapted as 
to read the drawing of the logical circuit of hardware as data of an 
image, to recognize symbols, characters, and relationships of connection 
between the symbols, and to store them in a database. 
The system for automatically recognizing the drawing of the logical 
circuit, however, is so adapted as to enter the drawing of the logical 
circuit of the hardware, so that it cannot be applied to the recognition 
of a diagram describing a data system. The diagram describing the data 
system offers the characteristics that it is important to distinguish the 
directivity of the relationship of connection between the symbols, that 
the number of connecting lines to be connectible to each of the symbols is 
optional, that there is regularity among the connectible symbols, that the 
shape and the meaning of each of the symbols can be modified or added for 
each user, and the like. Hence, the problem resides in the fact that the 
automated system cannot be applied to the recognition of diagrams such as 
data flow diagrams, entity relationship diagrams, flowcharts, and the 
like. 
Further, the intelligent system for recognizing the account books and 
vouchers, as described hereinabove, is so designed as to recognize 
formats, such as account books and vouchers composed of spreadsheets which 
are delimited and surrounded by straight lines only, to detect positions 
of the characters within the format, and to recognize the data. This 
system, however, cannot recognize diagram data unless the positions of 
symbols, connecting lines or a sequence of characters have been defined in 
advance. 
Furthermore, the system for forming the E-R model as described hereinabove 
is designed exclusively for forming the entity relationship diagram in 
which the number of symbols to be employed is limited to several kinds; 
hence, it has the problem that, although it can be applied only to the 
recognition of the shape and the meaning of a predetermined diagram, it 
cannot be applied to the recognition of a diagram that should be defined 
by each user in a manner as have conventionally been conducted. 
SUMMARY OF THE INVENTION 
Therefore, the object of the present invention is to solve the problems 
presented by the prior art techniques and to provide a diagram recognizing 
system capable of automatically recognizing a diagram formed manually or 
with drawing software, or formed independently from the CASE tool, 
particularly a diagram which has a directivity with respect to the 
relationship of connection between the symbols, in which the positions of 
the symbols are not defined in advance, and which is defined optionally by 
each of the users, and is capable of delivering the recognized logic to 
the CASE tool or the like without any manual operation. 
In order to achieve the aforesaid object, the present invention consists of 
a diagram recognizing system comprising a diagram input means for entering 
a diagram to be employed for describing a specification of data processing 
as data of an image, which is composed of a symbol to be represented by a 
drawing such as a circle, a rectangle and the like, a connecting line 
having a directivity indicative of a connection between symbols, and a 
character string to be provided on the symbols and the connecting line, 
such as a flowchart, a data flow diagram or the like; a recognizing rule 
input means for entering a recognizing rule for recognizing the diagram to 
be entered by the diagram input means; a recognizing rule storing means 
for storing the recognizing rule entered by the recognizing rule input 
means; a diagram recognizing means for recognizing the symbol, the 
connecting line, and the character string, each structuring the diagram 
entered from the diagram input means, as well as the connected 
relationship of the connection of the symbols, on the basis of the 
recognizing rule stored by the recognizing rule storing means, and for 
generating the kind of the symbol and the character string as well as the 
connected relationship of the connection between the symbols as logical 
data; and a conversion means for converting the logical data generated 
from the diagram recognizing means into a predetermined file format. 
With the arrangement as described hereinabove, the recognizing rule for 
recognizing the diagram as the object of recognition is entered from the 
recognizing rule input means, and it is stored in the recognizing rule 
storing means. Then, the diagram as the object of recognition is entered 
as the data of the image from the diagram input means. Upon the input of 
the diagram as the data of the image, the diagram recognizing means 
recognizes the symbols, the connecting lines, and the character string, 
each structuring the diagram entered from the diagram input means, as well 
as the relationship of connection between the symbols on the basis of the 
recognizing rule, and it generates the kind of the symbols and the 
character string as well as the relationship of connection between the 
symbols as the logical data. The logical data generated from the diagram 
recognizing means are converted into a predetermined file format by the 
conversion means so as to be delivered directly to the CASE tools or the 
like and then generated. 
Other objects, features and advantages of the present invention will become 
apparent in the course of the description of the preferred embodiments, 
which follows, with reference to the accompanying drawings.

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
The present invention will be described more in detail with reference to 
the accompanying drawings. 
FIG. 1 is a block diagram showing the data processing system according to 
the first embodiment of the present invention. The data processing system 
according to the first embodiment of the present invention comprises an 
image input unit 1 for entering a diagram as the object of recognition as 
data of an image, a data processing unit 2 for implementing the processing 
for recognizing the diagram, an input display unit 3 for allowing the user 
to correct the result of recognition, and an output file 4 for storing the 
result of recognition. 
The data processing unit 2 comprises an image data input section 21 for 
entering data of an image of the diagram that is represented by logic 1 
and logic 0 corresponding to the presence and the absence of the image of 
a pixel on a plane of the diagram on which the diagram is read, 
respectively; an image data storing section 22 for storing the data of the 
image entered by the image data input section 21; a data converting 
section 23 for converting the data of the image stored in the image data 
storing section 22 into data of a vector as an aggregate of line segments; 
a vector data storing section 24 for storing data of the vector; a 
recognizing rule storing section 25a for describing a logical recognizing 
rule, such as the kind of diagram corresponding to a graphic 
characteristic, the relationship of connection between the diagrams, and 
the like, in order to recognize the diagram; a diagram recognizing section 
25b for recognizing the diagram by matching the recognizing rule stored in 
the recognizing rule storing section 25a with the data of the vector 
stored in the vector data storing section 24; an intermediately recognized 
data storing section 26 for storing the result of recognition obtained by 
the diagram recognizing section 25b; a correction section 27 for 
correcting an unrecognizable portion of the result of recognition stored 
in the intermediately recognized data storing section 26 through a manual 
operation by the user on the image display unit 3; a recognized data 
storing section 28 for storing the corrected result of recognition 
corrected by the correction section 27; and a recognized result converting 
section 29 for converting the contents of the recognized data storing 
section 28 into a data format in the output file 4. 
For the data processing unit 2 having the structure as described 
hereinabove, the recognizing rule is stored in the recognizing rule 
storing section 25a and entered prior to the recognition processing from 
the image display unit 3 through the manual operation by the user. 
Further, the recognizing rule can be modified or added thereafter through 
the image display unit 3. 
FIG. 2 is a flowchart showing an outline of the processing procedure 
through the data processing unit 2. First, at step 100, a diagram as the 
object of recognition is read by the image input unit 1 and data of an 
image of the diagram is stored by the image data storing section 22 
through the image data input section 21. Then, at step 200, the data of 
the image stored in the image data storing section 22 is read by the data 
converting section 23; it is then subjected to the processing for removing 
noises, thinning lines, and the like, by the data converting section 23; 
it is further converted into the data of the vector which is a collection 
of data indicative of an aggregate of line segments and representing a 
closed graphic or an open graphic; and it generates the graphics table 802 
from said data of vector, and its graphics table 802 is stored in the 
vector data storing section 24. It is noted herein that a description on a 
detail of the technique for converting the data of the image into the data 
of the vector will be omitted because the technique is known as is 
described, for example, in a paper entitled "Making Input of Data for CAD 
Efficient: Highlighting An Automated Input Device for Drawings" by M. 
Hirata (Nikkei Computer, page 82, issued Mar. 3, 1986). Further, a known 
method for classifying the data of the vector into one of a group of open 
graphics and a group of closed graphics is described, for example, in a 
book entitled Computer Image Processing Guide, by H. Tamura (Soken 
Publishing Company, 1985), pages 220-222. This book describes an 
abstracting method of closed graphics by detecting an isolated diagram. 
Then, the vector data of the character string, for example, "ORDER 
PROCESSING" in FIG. 6(a) is also stored in the graphics table 802, as the 
group of open and closed graphics. 
Then, at step 300, the diagram recognizing section 25b is so adapted as to 
read the data of the vector stored in the vector data storing section 24 
and the recognizing rule for recognizing the diagram, stored in the 
recognizing rule storing section 25a; to implement the processing for 
matching the data of the vector with the recognizing rule so as to 
recognize the diagram represented in the data of the vector; and to store 
the result of recognition in the intermediately recognized data storing 
section 26 as an intermediate result of recognition. 
Further, at step 400, an inquiry is made through the image display unit 3 
about an unrecognizable portion of the intermediate result of recognition 
stored in the intermediately recognized data storing section 26, thereby 
requiring the user to enter necessary data, and the unrecognized portion 
is corrected by the correction section 27 to provide a complete result of 
recognition of the diagram entered. The corrected result of recognition is 
then stored in the recognized result storing section 28. 
At step 500, the recognized data converting section 29 is so adapted as to 
convert the data of the result of recognition stored in the recognized 
result storing section 28 into the logical data in a data format 
predetermined by the output file 4, for example, in such a data format as 
capable of being delivered directly to the CASE tool, and stores the 
resulting logical data in the output file 4. 
Hence, the CASE tool as described hereinabove can carry the data over 
immediately to a downstream step without any manual operation on the basis 
of the logical data stored in the output file 4. 
Next, a description will be made of a specific example of the recognition 
processing for recognizing the diagram with reference to FIG. 3 showing an 
example of a data flow diagram entered as the data of the image through 
the image input unit 1. This diagram may be written manually or drawn 
through drawing software and the logical relationship between the symbol 
and the connecting line for connecting the symbols can be judged only by 
the user. The diagram recognizing system according to the present 
invention is so adapted as to recognize such a diagram and to form logical 
data representing the name and the kind of the symbols and the 
relationship of connection between the symbols, as shown in FIG. 4. 
FIG. 5(a) indicates an example of an image corresponding to a portion of 
the data flow diagram, as shown in FIG. 3, stored in the image data 
storing section 22. In FIG. 5(a), reference numeral 701 represents an 
example corresponding to a portion of the data of the image around a 
diagram indicative of a character string "ORDER PROCESSING" of FIG. 3, 
surrounded by a circular symbol 704. FIG. 5(b) indicates a state of the 
pixels storing the data of the image corresponding to the portion as 
represented by reference numeral 702 in FIG. 5(a). The pixels indicated by 
"0" represent that no image exists, and the pixels indicated by "1" 
represent that an image exists. 
FIGS. 6(a), 6(b) and 6(c) show an example of the data of the vectors 
converted from the data of the image, as shown in FIG. 5, which is stored 
in the vector data storing section 24. In FIG. 6(a), reference numeral 801 
indicates an example of the data of the vector converted from the image 
data 701, which is displayed on the image display unit 3. The circular 
symbol 704 surrounding the sequence of the characters "ORDER PROCESSING" 
may be broken down into twelve line segments L1, L2, . . . , L12, and each 
of the line segments is represented by data in a vector type. Likewise, an 
arrow 705 may be divided into four line segments L13, L14, L15, and L16, 
and each of the line segments is represented by data in a vector type. 
The vector data 801 as represented hereinabove are stored as a closed 
graphic or an open graphic as the unit of storage in a graphic table 802 
of the vector data storing section 24. As specifically shown in FIG. 6(b), 
the graphic table 802 comprises data 803 of identification numbers of 
graphics (Graphic ID Nos.), indicative of an aggregate of line segments, 
data 804 of a class indicative of closed graphic or open graphic, data 805 
indicative of an arrangement for the line segments structuring the graphic 
classified in the class data 804, and data 806 of a branched point or 
points of the corresponding graphic, indicative of the coordinates of the 
branched point or points. Further, as shown in FIG. 6(c), each of the line 
segments is so structured as to be managed by a line segment table 807 
which in turn stores each of the line segments and comprises data 808 of 
the identification number of the line segment, data 809 of coordinates of 
a starting point of the line segment, and data 810 of coordinates of an 
ending point thereof. 
It is to be noted herein, however, that the coordinates (x1, y1) of a 
branched point branched from the line segment L11 to the line segment L15 
in FIG. 6(a) are represented on the drawing for reference only, and they 
are not displayed actually on a display screen. 
FIGS. 7, 8 and 9 are each an example of the recognizing rule for 
recognizing the data flow diagram stored in the recognizing rule storing 
section 25a. The recognizing rule for recognizing the diagram comprises a 
symbol recognizing rule 1000a for recognizing symbols, a flow recognizing 
rule 1010a for recognizing connecting lines (flows), and a relationship 
recognizing rule 1020a for recognizing the relationship of connection 
between the symbols, respectively. 
As shown in FIG. 7, the symbol recognizing rule 1000a contains data 1001a 
of the kind of symbol representing an external agent such as a user, 
process, and file, and the like; data 1002a of classification of the 
symbol into a simple graphic such as a circle, a rectangle, etc., or a 
composite graphic such as a combination of simple graphics; data 1003a of 
shapes of the symbols, indicative of a graphic characteristic of the 
corresponding symbol; data 1004a of positions indicative of the location 
of the character string inside or outside the corresponding symbol; and 
data 1005a of the directivity of the characters, (horizontal or vertical), 
in which the characters are written on the symbol. 
For example, when the data 1003a of the shape of the symbol identify the 
symbol as a rectangle, the symbol is defined by data consisting of four 
line segments of a long vector, no line segment of a short vector, four 
right angles, and two pairs of parallel lines. Further, the shape of the 
symbol recognized by the symbol recognizing rule 1000a is illustrated by 
reference numeral 1006a as shown in FIG. 7. 
As the recognizing rule representing the data 1003a indicative of the shape 
of a symbol, as shown in FIG. 7, there may be employed a number of long 
vectors, a minimum number of short vectors, a minimum and a maximum number 
of acute angles, a minimum and a maximum number of right angles, a minimum 
and a maximum number of obtuse angles, a sum of angles between the long 
vectors, a minimum and a maximum number of pairs of parallel lines, and 
the like. A class of the symbol indicative of the closed graphic or the 
open graphic may be added thereto. Further, as the recognizing rule, there 
may be added a minimum of the length to be recognized as a long vector and 
a maximum of the length to be recognized as a short vector. In addition, 
rules may be added in accordance with the symbols to be recognized or the 
graphic characteristics of the flows, in order to improve a rate of 
recognition. 
As shown in FIG. 8, the flow recognizing rule 1010a contains data 1011a of 
the kind of flow, indicative of a single direction of the connecting line 
or two directions thereof; data 1012a of the position of the arrow, when 
the connecting line is provided with the arrow; data 1013a of the shape of 
the flow, indicative of the graphic characteristic of the connecting line; 
data 1014a of the position of the character string corresponding to the 
connecting line; and data 1015a of the direction, horizontal or vertical, 
in which the character string is written on the connecting line. 
In the data 1013a of the shape of the flow, for instance, the connecting 
line in the single direction may be defined by data consisting of one or 
more line segments for a long vector, one acute angle, and two line 
segments for a short vector. The shape of the connecting line to be 
recognized by the flow recognizing rule 1010a is illustrated by reference 
numeral 1016a, as shown in FIG. 8. 
As shown in FIG. 9, the relationship recognizing rule 1020a for recognizing 
the relationship of connection between the symbols consists of a 
combination of data 1021a of the kind of the input symbol with data 1022a 
of the kind of the output symbol; the data 1021a indicative of the kind of 
the input symbol may contain an external agent such as a user and the 
like, the process, and the file, and the data 1022a indicative of the kind 
of the output symbols may likewise contain the external agent, process, 
and the file. Further, the relationship recognizing rule 1020a is so 
arranged as to represent data 1023a and data 1024a in a matrix type, the 
data 1023a being indicative of an unconnectable combination of the data 
1021a of the kind of the input symbol with the data 1022a of the kind of 
the output symbol and the data 1024a being indicative of the kind of 
connectable lines in the combination of the data 1021a with the data 
1022a. As shown in FIG. 9, for instance, it is defined that the external 
agent, such as a "user", as the input symbol cannot be combined with the 
external agent, such as "user", as the output symbol and the file, such as 
"shipment file", as the input symbol cannot be combined with the file such 
as a "shipment file" as the output symbol. Hence, in this case, for 
instance, a cell that exists under the column 1022a "EXTERNAL AGENT" "KIND 
OF OUTPUT SYMBOLS" and that exists in the row "EXTERNAL AGENT" under the 
column 1021a "KIND OF INPUT SYMBOLS" is indicated as "UNCONNECTABLE", as 
indicated by reference numeral 1023a, because the data on the external 
agent under the column 1022a is not connectable with the data on the 
external agent in the row located under the column 1021a. Likewise, when 
the data existing under the column 1022a is unconnectable with the data 
existing in the row under the column 1021a, the corresponding cells are 
filled with the word "UNCONNECTABLE". For instance, the cell corresponding 
to the file as the kind of the output symbol and to the external agent as 
the kind of the input symbol is filled with the word "UNCONNECTABLE". On 
the other hand, the data 1021a of the kind of the input symbols relating 
to the external agent are to be connected with the data 1022a of the kind 
of the output symbols relating to the process, as shown in FIG. 9. Hence, 
in this case, the cell corresponding to both of the data in the row 
"EXTERNAL AGENT" and under the column "PROCESS" is filled with words 
"SINGLE-DIRECTED FLOW OR DOUBLE-DIRECTED FLOW", as indicated by reference 
numeral 1024a. Likewise, all the cells defined in this manner are filled 
with the same words. 
FIG. 10 is a flowchart showing a detail of the diagram recognizing 
processing at step 300 in FIG. 2, and FIG. 11 is a flowchart showing a 
detail of the processing for recognizing the symbols and the flows at step 
350 in FIG. 10, without using any recursive processing. 
Now, a description will be made of the operations for recognizing the 
diagram represented in a vector type, with reference to the flowcharts as 
shown in FIGS. 10 and 11. 
First, at step 310, the diagram recognizing section 25b reads data of a 
vector relating to the diagram as the object of recognition, which is 
stored in the vector data storing section 24, followed by proceeding to 
step 320 at which the recognizing rule stored in the recognizing rule 
storing section 25a is read. Then, at step 330, the diagram recognizing 
section 25b is so adapted as to separate the data of the vector indicative 
of a line segment having a length shorter than a predetermined length as a 
candidate of a character string. Further, at step 340, a search is made 
for the graphics indicated in the class data 804 of the graphic table 802 
stored in the vector data storing section 24, and one closed graphic is 
fetched and set as a first object of recognition. 
More specifically, for example, in the data of the vector as shown in FIG. 
6(a), the class data 804 indicative of the symbol 704 surrounding the 
character string "ORDER PROCESSING" indicates the closed graphic, as shown 
in FIG. 6(b), so that the symbol 704 is set as the first object of 
recognition. 
Then, at step 350, the recognition processing for recognizing the symbols 
and the flows, as shown in FIG. 11, is called using the symbol as an 
argument, thereby recognizing the kind of the symbol set as the first 
object of recognition on the basis of the symbol recognizing rule 1000a. 
For example, for the symbol 704 for the character string "ORDER PROCESSING" 
as shown in FIG. 6(a), the graphic table 802 is so arranged as to store 
the line segments L1 to L12, inclusive, of the symbol 704, and the line 
segment table 807 is so arranged as to store a detail of the positions 
indicated by the data 809, indicative of the coordinates of the starting 
point of each line segment, and the data 810, indicative of the 
coordinates of the ending point thereof. 
On the other hand, as shown in FIG. 7, the symbol recognizing rule 1000a 
for recognizing the symbol is so arranged as to define the shape of the 
symbol under the column "SHAPE OF SYMBOL", as indicated by reference 
numeral 1003a, corresponding to the row "PROCESS", as indicated by 
reference numeral 1001a, by no long vectors, a large number of short 
vectors, a large number of obtuse angles, and the like. In the recognition 
processing at step 350, the length of each of the line segments L1 to L12, 
inclusive, is given by the data 809 indicative of the coordinates of the 
starting points and the data 810 indicative of the coordinates of the 
ending points, thereby determining the numbers of the long vectors and the 
short vectors. In addition, the direction of each of the line segments is 
determined by the data 809 and the data 810, and an angle of the adjacent 
line segments is given for each pair of the adjacent line segments, 
thereby determining the numbers of acute angles, obtuse angles, right 
angles, and pairs of the parallel lines. 
Then, data of the numbers of the long vectors, the short vectors, the acute 
angles, the obtuse angles, the right angles, and the pairs of the parallel 
lines are compared with the data 1003a of the shapes of the symbols in the 
symbol recognizing rule 1000a. When it is found as a result of comparison 
that the symbol 704 for the character string "ORDER PROCESSING" agrees 
with the shape of the symbol under the column 1003a corresponding to the 
row "PROCESS" under column 1001a indicative of the kind of the symbol, 
then the symbol 704 is recognized as the kind of the symbol under the 
column 1001a "PROCESS". 
Thereafter, it is decided to determine if the data 806 indicative of 
branched points exists in the data of the vector relating to the symbol as 
the object of recognition contained in graphics table 802. When it is 
decided that the data 806 exists, the data of vector corresponding to the 
data 806 is read from graphics table 802, and the data is recognized as 
the candidate for the connecting line. 
Then, it is decided to determine if the candidate for the connecting line 
includes additional data indicative of branched points. If the candidate 
for the connecting line has such additional data, the data of vector 
corresponding to the additional data is read from the graphics table 802, 
and set up as the partner symbol which is the next object of recognition. 
The partner symbol is also recognized on the basis of the symbol 
recognition rule 1000a, and the data 1012a indicative of the position of 
the arrow and the data 1013a indicative of the shapes of the flow in the 
flow recognizing rule 1010a are matched with the data of the vector 
corresponding to the candidate for the connecting line, and the candidate 
for the connecting line is recognized and decided as the connecting line. 
Then, the logic of the relationship of connection of the symbol as the 
object of recognition with the partner symbol is investigated on the basis 
of the relationship recognizing rule 1020a for recognizing the 
relationship of connection between the symbols. 
Thereafter, at step 360, a character string corresponding to the symbol 704 
is detected, and the relationship of correspondence is stored in a column 
"ID OF CORRES. CHARACTER STRING", as indicated by reference numeral 904, 
in a symbols table 900 prepared in the intermediately recognized data 
storing section 26, as shown in FIGS. 12 and 13. 
The intermediately recognized data storing section 26 comprises the symbols 
table 900, a flow table 910, a character string table 920, a vector data 
storing section 930, and a stack 940. 
As shown in FIG. 12, the symbols table 910 may contain data of the 
identification of symbols under column 901 indicated by "SYMBOL ID", which 
is represented, for example, by `S1`, `S2`, and the like; data of the kind 
of the symbols under column 902 indicated by "KIND OF SYMBOLS", which is 
represented, for example, by `EXTERNAL AGENT`, `PROCESS` and the like; 
data of the appropriateness or inappropriateness of the result of 
recognition of the symbol under column 903 indicated by "DECISION OF 
RECOGNITION", which is represented, for example, by `OK` or `NG`; data of 
the identification of corresponding character string under column 904 
indicated by "ID OF CORRES. CHARACTER STRING"; and data of the vector 
corresponding to the connected line determined as the object of 
recognition, under column 905 indicated by "CORRES. VECTOR DATA". 
FIG. 12 further shows the flow table 910 which may contain data of the 
identification of flow under column 911 indicated by "FLOW ID", which is 
represented, for example, by `f1`, `f2`, and the like; data of the 
direction of the flow under column 912 indicated by "DIRECTION OF FLOW", 
which is represented by `SINGLE` for a single direction, `BOTH` for both 
directions, or `NONE` for no direction; data of the identification of 
input symbols showing the input symbols under column 913 indicated by 
"INPUT SYMBOL ID"; data of the identification of output symbols showing 
the output symbol under column 914 indicated by "OUTPUT SYMBOL ID"; data 
of the results of recognition of the flow under column 915 indicated by 
"DECISION OF RECOGNITION"; data of the relationship of connection between 
the symbols connected through the connected line, under column 916 
indicated by "DECISION OF CONNECTION"; data of the identification of the 
corresponding character string provided on the connected line, under 
column 917 indicated by "ID OF CORRES. CHARACTER STRING"; and data of the 
vector corresponding to the connected line determined as the object of 
recognition, under column 918 indicated by "CORRES. VECTOR DATA". 
As shown in FIG. 13, the character string table 920 may contain a column 
921 indicative of data of the identification of character strings; a 
column 922 indicative of data of the results of recognition of the 
character strings; a column 923 indicative of data of codes of the 
characters of the recognized character string; and a column 924 indicative 
of the data of the vector corresponding to the character string determined 
as the object of recognition. 
FIG. 13 further shows the vector data storing section 930 which is so 
adapted as to correspond the data of the vector prior to recognition to 
the symbols, flows and character strings subsequent to recognition. Data 
of each of the symbols, flows and character strings subsequent to 
recognition are stored in substantially the same type as in the graphics 
table 802 and the line segments table 807 as shown in FIG. 6. 
In addition, as shown in FIG. 13, the stack 940 is so adapted as to store 
data 941 indicative of the identification of partner symbols for the 
symbols as the object of recognition. The example as shown in FIG. 13 
indicates that partner symbols "ORDER PROCESSING" and "SHIPMENT 
PROCESSING", whose identification is S2 and S3, respectively, are 
temporarily stored during the course of the processing for the symbol 
"USER" as the object of recognition, whose identification is S1. 
Turning back to FIG. 10, the relationship of correspondence of the 
character string corresponding to the symbol as the object of recognition 
is stored at step 360 under the column 904 indicative of the 
identification of the corresponding character string of the symbol table 
900 and then, at step 370, the character string corresponding to the 
connected line is detected on the basis of the vector data 809 and 810 
indicative of the coordinates of the starting point and the ending point, 
respectively, of the connected line recognized, and the data 1014a 
indicative of the position of the character string in the recognizing 
rule, and this relationship of correspondence is stored in the flows table 
910 of FIG. 12 as data 917 indicative of the identification of the 
corresponding character string. Thereafter, at step 380, the corresponding 
character string is recognized by a character recognizing technique which 
has been conventionally utilized for an optical character reader (OCR) or 
so on, the suitability or unsuitability of the result of recognition is 
stored in the character string table 920 of FIG. 13 as data 922 for 
judging the result of recognition, and, when it is found that the result 
of recognition is suitable, the code of the character is stored in the 
character string table 920 as character code data 923. It is to be noted 
herein that steps 360 and 370 are exchangeable. 
In FIGS. 12 and 13, reference numerals (1) to (5), inclusive, are intended 
to mean the relationship of correspondence between the data connected 
through the broken lines indicated by the respective reference numerals 
(1) to (5). 
A description will now be made of a detail of the processing at step 350 of 
FIG. 10 with reference to the flowchart as shown in FIG. 11. 
At step 351a, the data of the vector of the symbol as the object of 
recognition, to be delivered as an argument, is matched with the data 
1003a indicative of the shape of the symbol, defined in the symbols 
recognizing rule 1000a, and the symbol identification data 901, the data 
902 indicative of the kind of the symbol, the data 903 indicative of the 
decision of the result of recognition, and the data 905 indicative of the 
corresponding data of the vector are stored in the symbols table 900 of 
the intermediately recognized data storing section 26. 
Then, at step 352a, it is decided to determine if the data 806 indicative 
of branched points exists in the data of the vector relating to the symbol 
as the object of recognition contained in graphics table 802. When it is 
decided that the data 806 exists, the data of vector corresponding to the 
data 806 is read from the graphics table 802, and recognized as the 
candidate for the connecting line. 
Then, at step 353a, it is decided to determine if the candidate for the 
connecting line has additional data indicative of further branched points. 
If the candidate for the connecting line has such additional data, the 
data of vector corresponding to the additional data is read from the 
graphics table 802, and set up as the partner symbol which is the next 
object of recognition. 
Then, at step 354a, the partner symbol set up at step 353a is recognized in 
the same manner as step 351a, on the basis of the symbol recognition rule 
1000a. 
Then, at step 355a, the data 1012a indicative of the position of the arrow 
and the data 1013a indicative of the shapes of the flow in the flow 
recognizing rule 1010a are matched with the data of the vector 
corresponding to the candidate for the connecting line, and the candidate 
for the connecting line is recognized and decided as the connecting line. 
Thus, the decided data, including the data 911 of the identification of the 
flow, the data 912 of the direction of the flow, the data 913 of the input 
symbol, the data 914 of the output symbol, the data 915 for deciding the 
result of recognition, and the vector data 918 corresponding to the 
connected line, are stored in the flow table 910. 
Then, at step 356a, the logic of the relationship of connection of the 
symbol as the object of recognition with the partner symbol is 
investigated on the basis of the relationship recognizing rule 1020a for 
recognizing the relationship of connection between the symbols as shown in 
FIG. 9, and the result of recognition is stored in the flow table 910 of 
FIG. 12 as the data 916 for judging the connection. Thereafter, at step 
357a, the partner symbol is stored temporarily in the stack 940 of FIG. 
13, followed by returning to the processing at step 352a. 
It is then investigated to determine if the connecting line connected to 
the symbol set as the object of recognition further exists, and this 
processing is repeated until no connecting line exists any more. When it 
is decided at step 352a that no connected line exists any more, on the 
other hand, then the program flow goes to step 358a at which it is 
determined if the stack 940 is empty. If even one datum of the kind of 
symbol is still left in the stack 940, then one symbol indicated by the 
data of the kind of the symbol is fetched from the diagram recognizing 
section 25b, and the symbol is set as a symbol as the new object of 
recognition in step 359a, then the program flow goes back to step 352a. 
The processing is allowed to end when the stack 940 becames vacant. 
The order of fetching one symbol from the stack 940 may be based on any 
method, such as a first-in first-out method or a first-in last-out method. 
Next, a description will be made of the processing at step 350 in FIG. 10, 
which employs recursive processing, with reference to the flowchart as 
shown in FIG. 14. 
First, at step 351b, the symbol as the object of recognition delivered as 
an argument is matched with the symbols recognizing rule 1000a. Then, at 
step 352b, it is decided to determine if the data 806 indicative of 
branched points exists in the data of the vector relating to the symbol as 
the object of recognition contained in graphics table 802. When it is 
decided that the data 806 exists, the data of vector corresponding to the 
data 806 is read from the graphics table 802, and recognized as the 
candidate for the connecting line. 
Then, at step 353b, it is decided to determine if the candidate for the 
connecting line has additional data indicative of further branched points. 
If the candidate for the connecting line has such additional data, the 
data of vector corresponding to the additional data is read from graphics 
table 802, and set up as the partner symbol which is the next object of 
recognition. 
Then, at step 354b, the partner symbol set up at step 353b is recognized as 
in step 351b, on the basis of the symbol recognition rule 1000a. 
Then, at step 355b, the data 1012a indicative of the position of the arrow 
and the data 1013a indicative of the shapes of the flow in the flow 
recognizing rule 1010a are matched with the data of the vector 
corresponding to the candidate for the connecting line, and the candidate 
for the connecting line is recognized and decided as the connecting line. 
Thus, the decided data, including the data 911 of the identification of the 
flow, the data 912 of the direction of the flow, the data 913 of the input 
symbol, the data 914 of the output symbol, the data 915 for deciding the 
result of recognition, and the vector data 918 corresponding to the 
connected line, are stored in the flow table 910. Thereafter, at step 
356b, the logic of the relationship of connection between the symbol as 
the object of recognition and the partner symbol is investigated on the 
basis of the relationship recognizing rule 1020a indicative of the 
relationship of connection between the symbols. Then, at step 357b, the 
partner symbol is assigned to a symbol as a new object of recognition, and 
the processing for recognizing the symbol and the flow, as shown in FIG. 
14, is called in a recursive manner, followed by the processing at step 
352b. When it is found at step 352b that no connecting line to be 
connected to the symbol as the new object of recognition exists, on the 
other hand, then this processing is allowed to end. 
The data stored in the intermediately recognized data storing section 26 by 
the processing made in the manner as described hereinabove are read by the 
correction section 27 for correcting the result of recognition, and it is 
inspected as to whether the data contain any unrecognizable portion. This 
inspection is conducted on the basis of the data 903 for judging the 
recognition in the symbol table 900 as shown in FIG. 12, the data 915 for 
judging the recognition in the flow table 910 as shown in FIG. 12, and the 
data 922 for judging the recognition in the character string table 920 as 
shown in FIG. 13. If the symbol, the flow or the character string is 
represented by "NG", it is displayed on the image display unit 3, 
requiring the user to make a necessary input recognizable in a correct 
way. 
When it is found that there is neither symbol nor flow nor character string 
which is represented by `NG`, the correction section 28 stores the result 
of correction in the recognized result storing section 28. The result of 
recognition stored therein is then converted by the recognized result 
converting section 29 into the logical data, as shown in FIG. 4, which 
have the data format for the output file 4. 
As have been described hereinabove, the diagram recognizing system 
according to the embodiment of the present invention is configured in such 
a manner that the diagram recognizing rule for recognizing the diagram as 
the object of recognition is entered from the image display unit 3 and 
stored in the recognizing rule storing section 25a; the diagram as the 
object of recognition is then entered as data of an image from the image 
input unit 1; the symbol, the connecting line, and the character string, 
each structuring the diagram entered, as well as the relationship of 
connection between the symbols, are recognized on the basis of the 
recognizing rule by the diagram recognizing section 25b; the symbol, the 
connecting line, the character string, and the relationship of connection 
between the symbols, are generated as the logical data indicative of the 
kinds of the symbol and the character string and the relationship of 
connection between the symbols; and the logical data are converted so as 
to be generated in such a predetermined text type as being capable of 
being delivered directly to the CASE tools or the like. Hence, the diagram 
recognizing system according to the present invention can offer the 
advantages that it can automatically recognize the diagram formed 
independently from the CASE tools, that is, the one formed manually or 
through the drawing software, particularly the one in which the symbols 
have a directivity in relationship of connection between the symbols, the 
positions of the symbols are not determined in advance, and each of the 
users may define the diagram in an arbitrary manner, and that the logic 
recognized can be delivered to the CASE tools or the like without the aid 
of any manual operation. 
Therefore, the diagram recognizing system according to the present 
invention enables the operations for entering the diagrams to be reduced 
in forming the source program through the CASE tools or the like, and the 
operations for inspecting the adaptability of the diagrams on the basis of 
the logical contents of the diagram, as well as editing the diagrams, 
storing them, printing them and converting them into data suitable for the 
step which follows, to be implemented with ease. Further, the diagram 
recognizing system according to the present invention offers the extremely 
useful advantage that it can remarkably improve the efficiency in the 
operations for automatically forming the source program. 
In addition, the recognizing rule is so adapted as to be entered from the 
image display unit 3, so that each user can define and modify the 
recognizing rule with great freedom. This offers the advantage that the 
optimal recognizing rule can be structured for each user. 
Further, in recognizing the diagram, the diagram recognizing system 
according to the present invention is arranged in such a manner that the 
data of the vector to be determined as a first candidate of the character 
string are separated; the recognition is allowed to start by setting an 
aggregate of the data of the vector that has the highest possibility of 
becoming the symbol as a first symbol; the processing is implemented one 
after another for choosing the open graphic being connected to the first 
symbol as the candidate for the connecting line, and recognizing a 
graphic, which exists on the side opposite to the candidate for the 
connecting line to be connected to the first symbol, as a symbol, and 
deciding said candidate for the connecting line as the connecting line. 
Then, each of the symbols is recognized so as to correspond to a candidate 
of the character string provided on the connected line. Hence, the present 
invention presents the advantage that error in recognition can be reduced 
and high recognition performance can be achieved, as compared with the 
instance where the symbol and the connecting line are recognized with the 
character string mixed up therewith. 
Furthermore, since the unrecognizable portion of the result of recognition 
can be corrected through the image display unit 3, recognition performance 
can be improved to a higher extent, and reliability upon the logical data 
to be generated into the output file 4 can also be improved. 
In addition, the data of the image of the diagram to be entered from the 
image input unit 1 are converted into the data of the vector by the data 
converting section 23, and the diagram is recognized on the basis of the 
data of the vector, so that a memory capacity of the storing section for 
storing the image of the diagram can be made smaller and the processing 
time can be shortened, as compared with the recognition of the image in a 
pixel unit. 
It is furthermore to be noted herein that, although the recognizing rule 
for recognizing the diagram is arranged to be entered through the image 
display unit 3, it can also be entered through the diagram input means. 
More specifically, as indicated by the broken line 10 in FIG. 1, the 
diagram may be entered from the image input unit 1 into the recognizing 
rule storing section 25a and the recognition of the diagram may be 
implemented in accordance with pattern matching techniques known to the 
art. Alternatively, the data of the image of the recognizing rule may be 
recognized by the diagram recognizing section 25b, the result of 
recognition may be stored as a recognizing rule, and the diagram may then 
be recognized with substantially the same operations as described 
hereinabove. In this case, however, it is required to set another 
recognizing rule for recognizing the data of the image for the recognizing 
rule. 
As shown in FIG. 15, which is a block diagram showing a diagram recognizing 
system according to the second embodiment of the present invention, the 
diagram recognizing system may be structured in such a manner that the 
diagram and the recognizing rule are entered from an input device 11 so 
arranged as to allow the drawing or the like to be entered through a pen 
as data of an image that is capable of being generated in a vector 
representation. This arrangement offers the advantages that the structure 
becomes simpler than the arrangement in the first embodiment of the 
present invention because neither the image data storing section 22 nor 
the data converting section 23 are required. Further, the steps for the 
processing for recognition can be reduced to those as shown in FIG. 16. 
It is to be noted herein that, as shown in FIG. 16, step 100A denotes the 
processing for storing the data of the image of the pen input device 11 in 
the vector data storing section 24; however, the processing at steps 300, 
400, and 500 are substantially the same as the corresponding steps as 
shown in FIG. 2. Hence, a description of the same steps will be omitted 
from the description that follows. 
In addition, the entity relationship diagram can be recognized when the 
recognizing rule is defined in the manner as shown in FIGS. 17 to 19, 
inclusive, although those as shown in FIGS. 7 to 9, inclusive, are 
demonstrated as examples of the recognizing rules. 
It can be noted herein that the recognizing rules as shown in FIGS. 17, 18 
and 19 are different from those as shown in FIGS. 7, 8 and 9, 
respectively, in that the recognizing rules are so arranged as to define a 
composite symbol, as indicated by reference numeral 1006b, as the kind of 
the symbol, as indicated by "WEAK ENTITY" and a connecting line with no 
indication of an arrow, as indicated by reference numeral 1016b. It is 
further to be noted that the reference symbols of the data in FIGS. 17, 18 
and 19 are provided with a suffix b in place of the suffix a of the 
reference symbols of the data in FIGS. 7, 8 and 9; however, the subject to 
be defined is the same. 
It is further to be noted that, for example, even if the user intends to 
draw a long line as a diagram, it may be recognized as an aggregate of 
shorter lines when it is converted into data of a vector due to an error 
or for other reasons. For instance, when a long line is drawn manually in 
such a manner as being curved to a slight extent, the situation as 
described hereinabove may happen. In this case, however, a technique for 
matching the symbol with a recognizing rule using an evaluation point with 
such an error taken into account may be utilized in order to improve the 
rate of recognition. This technique may be carried out in the processing 
for matching the symbol with the recognizing rule at steps 351a and 354a 
in FIG. 11 and at steps 351b and 354b in FIG. 14. 
FIG. 20 shows an example of a table 2010 for calculating evaluation points 
for the symbol recognizing rule 2000. The symbol recognizing rule 2000 
contains a column entitled "KIND OF SYMBOL", containing items such as 
"external agent", "process", and "file"; a column entitled 
"