Abstract:
The identity of an unknown character is determined by comparing matrices representing the unknown character with matrices representing known dictionary characters and determining the deviation between matrices for the unknown and the dictionary characters. By obtaining both matrices for the character and for the background surroundings of the character, it is possible to extract small differences between local portions of the character. The dictionary pattern of each character is automatically created by &#34;learning&#34; and also the features of the character at the time of judgment and the related result are totalized so that the dictionary pattern can be created from the complementary cumulative matrices obtained by such totalization. By virtue of this advantage, it is possible to asymptotically improve the recognition rate.

Description:
This application is a continuation of application Ser. No. 048,668, filed on May 11, 1987, now abandoned. 
    
    
     BACKGROUND OF THE INVENTION 
     The present invention relates to a character reading method whereby unknown characters are read by steps. These characters, which are defined as objects to be read, are inputted through a photoelectric conversion element such as a television camera. The dictionary patterns to which patterns which are being read are compared for deciding which character is being read are automatically created by learning the features thereof and the thus obtained dictionary is utilized for reading. 
     It is generally desirable that an optical character reader be capable of recognizing characters at the highest speeds possible and of course with extreme accuracy. Several pattern recognition systems for accomplishing high speed reading of characters have been disclosed in U.S. Pat. Nos. 4,556,985, 4,628,533 and 4,630,308 and assigned to the same assignees as the present application. The systems disclosed in these patents provide character recognition at high speeds, however, similar characters are sometimes misread, therefore a more accurate character reading method is desirable. 
     SUMMARY OF THE INVENTION 
     In accordance with the invention, a character reader is provided wherein the definition of a bit-matrix is expanded. Not only is the matrix which corresponds to the conventional character component of the pattern utilized but also the matrix which corresponds to the background component of the pattern as well is employed as a feature of the character reader. Generally speaking, size expansion is required because the size of the characters generated is smaller than that of the original characters. This expansion results in a deteriorating performance when discriminating between similar characters (for instance O and Q). In order to prevent this, the complementary bit-matrices are extracted, and the background component is expanded as well as the character component. This catches the features which would normally be lost. 
     Moreover, as part of an automatic learning mode, cumulative bit-matrices are obtained by measuring the complementary bit-matrices with respect to the character patterns. On the basis of the thus obtained cumulative matrices, the dictionary patterns are automatically created. In obtaining the cumulative bit-matrices, a binary-coded threshold value is varied within an allowable range, and a character pattern is measured. When reading the unknown character patterns with respect to the dictionary patterns, candidate character categories are gradually reduced by performing a classifying process and also, the time required for processing is shortened. 
     Furthermore, the complementary bit matrices are totalized for every category on the basis of the recognized results. To correct the dictionary patterns, these patterns are computed once again in accordance with the totalized cumulative matrices. Thus, the differences between the character patterns at the time of learning and at the time of reading can quantitatively be evaluated. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     An embodiment of the present invention will be described hereinafter, with references to the accompanying drawing, in which: 
     FIG. 1 is a block diagram of the embodiment of the present invention; 
     FIG. 1A is an explanatory view showing an example of images of characters; 
     FIG. 2 is a flow chart showing the learning process of one character; 
     FIG. 2A is an explanatory view showing complementary bit matrices; 
     FIG. 2B is an explanatory view showing cumulative matrices; 
     FIG. 3 is a flow chart showing the process of creating basic bit-matrices from the cumulative matrices; 
     FIG. 3A is an explanatory view showing basic bit-matrices; 
     FIG. 3B is an explanatory view showing critical matrices; 
     FIG. 4 is a flow chart showing the process of creating dictionary patterns; 
     FIG. 4A is an explanatory view showing a horizontal run-number and a vertical run-number of a character basic bit-matrix; 
     FIG. 4B is an explanatory view showing a horizontal run-number and a vertical run-number of a background basic bit-matrix; 
     FIG. 4C is an explanatory view showing the horizontal and vertical bit-matrices of a character component; 
     FIG. 4D is an explanatory view showing the horizontal and vertical bit-matrices of a background component; 
     FIG. 4E is an explanatory view showing shift regions of the horizontal and vertical bit-matrices; 
     FIG. 4F is a explanatory view showing character stain bit-matrices relative to the critical matrices; 
     FIG. 4G is an explanatory view showing background stain bit-matrices relative to the critical matrices; 
     FIG. 5 is an explanatory view illustrating the flow of the development of matrices during the dictionary creating processes; 
     FIG. 6 is a flow chart showing the reading process; 
     FIG. 6A is a flow chart showing the process of determining a normalization ratio; 
     FIG. 6B is a flow chart showing classifying process I; 
     FIG. 6C is a flow chart showing classifying process II; 
     FIG. 6D is an explanatory view illustrating an evaluation index; 
     FIG. 6E is a flow chart showing a recognition process; 
     FIG. 6F is a flow chart showing a totalization process. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     Illustrated in FIG. 1 is one embodiment of the present invention. In the Figure, the reference numeral 1 represents a set of object characters, 2 denotes an imaging device such as a television camera, 3 designates a variable binary-coding circuit, 4 represents a feature extraction circuit, 5 designates an image memory, 6 denotes a processing unit such as a microprocessor, 7 represents a threshold value generating circuit, 8 designates an auxiliary memory, 9 represents a dictionary pattern memory, and 10 denotes an input/output interface. 
     In the operation of the character reader shown in FIG. 1, a pattern representing a character is scanned by the camera 2. Time-series signals generated by the camera are successively converted into binary values with a certain threshold level and divided into pixels (picture elements) with prescribed signals by the binary-coding circuit. Segments of the binary images are extracted by the feature extraction circuit 4 and are written into the image memory 5 in a DMA mode. It is to be noted that the segment is a run of black picture elements on a horizontal scanning line. The segment information includes a segment length, a segment right end coordinate value, a main boundary length, an auxiliary boundary length and linkage information. The microprocessor 6 has access to the image memory 5 via a system bus BS. The processor can therefore perform connectivity analysis and character reading processing. The microprocessor 6 receives the images in response to a command issued from an external source through the input/output interface 10. The microprocessor then carries out the learning, reading totalizing and correcting processes. When receiving the images, the microprocessor 6 can specify a variable binary-coded threshold value (TH) to the variable binary-coding circuit 3, through the threshold value generating circuit 7. Thus, the microprocessor 6 fetches the image data while modifying the binary and coded threshold value and creates dictionary patterns. These patterns are then stored in the dictionary pattern memory 9. The auxiliary memory 8 is available as a temporary memory for multiple purposes. The patterns are scanned in response to an active level on S 1 , and subsequently they are transmitted to the outside in response to an active level on S 2 . The binary-coded threshold value TH is either received via the input/output interface or the value is automatically determined from an area histogram. 
     FIG. 1(A) shows an example of binary images of characters which are scanned by the television camera 2 depicted in FIG. 1. The character patterns are within an effective picture P. The coordinates of picture elements Pc (black picture elements) of the individual patterns are expressed by the orthogonal coordinate systems X and Y. The background of the character patterns is displayed as a background picture element P B  (a white picture element). The character patterns are complimentarily exhibited by the black picture elements serving as the character component and by the white picture elements serving as the background. 
     FIG. 2 shows the flow chart of a learning process wherein the microprocessor 6 creates the dictionary patterns from standard character patterns. The binary threshold value is set in the threshold value generating circuit 7 (see 1 of FIG. 2) and the image data of the character patterns which are the object to be learned are fetched (see 2 of FIG. 2). Next, the connectivity is analyzed (see 3 of FIG. 2), and the segments are labeled. The segments connected in such a step are marked with the same labels. An aggregation of segments with the same labels is simply referred to as a pattern. Obtained from each individual pattern are circumscribed frame coordinate values (Y T , Y B , X L , X R , an area (Ac) and a circumferential length (L B ) Also, the character pattern is cut out (see 4 of FIG. 2) in accordance with a width (WC) and a height (H C ). The area (A C ) of the character pattern is displayed as the total sum of the areas of patterns. If an area of the character pattern at the time of fetching an initial image of the specified character is A co , a variation quantity δ An  of the area A cn  of the n&#39;th character pattern can be given by: ##EQU1## If the area variation quantity (δ An ) is less than the upper limit (δ An ) a complementary bit-matrix of the character pattern is obtained. If not, the binary coded threshold value is reset, and the process of image fetching is executed once again. (See 5 of FIG. 2.) 
     FIG. 2A shows examples wherein a pair of complementary bit-matrices (B C , B B ) are created. For example, the width W C  of a character pattern P A  is equivalent to 25 pixels, and the height H c  is equivalent to 35 pixels. This figure shows a normal size of width and height where the width is 10 pixels and the height is 14 pixels. If black picture elements are present in the meshes, the value will be 1. If no black picture elements exists, the value will be 0. The thus created binary matrix is called a character bit-matrix B c . FIG. 2A(b) shows the character bit-matrix, B c , corresponding to FIG. 2A(a). On the other hand, if white picture elements are present in the meshes, the value will be 1. If no white picture elements exist, the value will be 0. The binary matrix created by this method is called a background bit-matrix B B . FIG. 2A(c) shows the background bit-matrix, B B , corresponding to FIG. 2A(a). It can be observed from the Figure that when the normal size (10 pixels×14 pixels) is smaller than the size (25 pixels×35 pixels) of the original character pattern, the black pixels and the white pixels are expanded in terms of appearance by executing the normalizing process in the character bit-matrix and the background bit-matrix. The character bit-matrix B c  and the background bit matrix B B  together are called the complementary bit matrices. The complementary bit matrices are obtained in step 6 of FIG. 2. 
     These complementary bit-matrices involve the binary matrices. A character cumulative matrix, C C , and a background cumulative matrix, C B , are obtained (see 7 of FIG. 2) by cumulating the matrix elements with respect to a character component (B C  (i,j)) and a background component (B B  (i,j)). To accomplish this, the learning process must be repeated several times. As an example, FIGS. 2B(a) and 2B(b) show the cumulative matrices when the learning process has been repeated 10 times. 
     A basic bit-matrix, B O , is obtained from the complementary cumulative matrices of the character patterns in accordance with the processing procedures shown in FIG. 3. A bit rate, R B , of a basic bit-matrix, B O , is expressed as: ##EQU2## A reference value N 1 , before being converted into the binary matrix, is incremented from a given value by ones so that the bit rate (R B ) of the basic bit-matrix is R 1  or less. The bit-matrix which will initially have a rate less than R 1  is defined as the basic bit-matrix. Hence, if the values of R 1  and N 1  are varied, even in the same cumulative matrix, the basic bit-matrix will also vary. The bit rate differs, depending on the character pattern, and the value of R 1  is therefore determined as the upper limit value of the bit rate with regard to all of the characters. The reference value N 1  is used to simulate the fluctuations in the linear-width of the character. These fluctuations are caused by variations in the binary coded threshold value at the time of learning. 
     FIG. 3A(a) shows an example of a basic bit-matrix, B oc , which is obtained from the character cumulative matrix, C c , depicted in FIG. 2B(a), with the conditions of N 1  =5 and R 1  =8. FIG. 3A(b) shows an example of a basic bit-matrix, B OB , which is obtained from the background cumulative matrix, C B , depicted in FIG. 2B(b), with the conditions of N 1  =7 and R 1  =0.8. The bit-matrix obtained when N 1  =1 and R 1  =1 is a basic bit-matrix having a maximum bit rate. This is referred to as a critical bit-matrix. Critical bit-matrices, B CC  and B CB , are shown in FIGS. 3B(a) and 3B(b). 
     FIG. 4 shows the processing procedure in which the dictionary patterns are obtained from the basic bit-matrices (B OC , B OB ) and from the critical bit-matrices (B CC , B CB ). 
     First, a horizontal run-number (N H ) of each matrix element is obtained from the basic bit-matrices. The horizontal run-number N H  is the number of elements, having a value of 1, which can be found in a row of the emphasized matrix elements (B OC  (i,j) or B OB  (i,j)). If the number of elements is 0, no range exists and the horizontal run-number N H  is also 0. Similarly, a vertical run-number (N V ) is the number of elements, having a value of 1, which can be found in a column. FIGS. 4A(a) and 4A(b) show the matrices of the horizontal run-number, N H , and of the vertical run-number, N V , of the character basic bit-matrix, B OC . The matrices of the horizontal run-number and of the vertical run-number of the background basic bit-matrix, B OB , are shown in FIGS. 4B(a) and 4B(b). 
     The above described process is performed in steps 1 and 2 of FIG. 4. When N H  and N V  of the emphasized basic bit-matrix are equal to or greater than N 2 , the emphasized matrix elements of the horizontal/vertical bit-matrices become 1 (see 3, 4, 5 and 6 of FIG. 4). When N H  is less than N 2 , N V  is equal to or greater than N 2 , and N H  is greater than N 3 , then B H  is equal to 1 (see 3, 7, 9 and 10 of FIG. 4). If N H  &lt;N 2 , N V  &lt;N 2 , N H  ≧N V  and N V  ≧N 3  then B V  =1 (see 3, 7, 8, 11 and 12 of FIG. 4). If N H  ≧N 2 , N V  &lt;N 2 , and N 3  &lt;N V   then B V  =1 (see 3, 4, 11 and 12 of FIG. 4). When N H  &lt;N 2 , N V  &lt;N 2 , N H  =N V , N 3  &lt;N V , then B H  (i,j)=B V  (i,j)=1 (see 3, 7, 8, 11, 12, 13 and 10 of FIG. 4). In other cases, the individual matrix elements of B H  and B V  remain zero-cleared. 
     The reference values N 2  and N 3  which determine both the horizontal bit-matrix B H  and the vertical bit-matrix B V  are determined from the linear width of the character patterns. N 2  is the value used for examining whether or not it exceeds the linear width. N 3  is the value used for eliminating noise or other interference. FIGS. 4C(a) and 4C(b) show examples of horizontal/vertical bit-matrices, B HC  and B VC , with respect to the character basic bit-matrix. For this example N 2  =5 and N 3  =1. Similarly, FIGS. 4D(a) and 4D(b) show horizontal/vertical bit-matrices B HB  and B VB  with respect to the background basic bit-matrix. 
     After obtaining the horizontal/vertical bit-matrices B H , B V ), a mask bit-matrix B M  is obtained. To obtain the mask bit-matrix a stain bit must be defined. Stain bit-matrices, B DCC  and B DBC , are calculated with respect to the critical bit-matrices, B CC  and B CB . In this case, to obtain the stain bit-matrices, shifting operations are executed in both, a horizontal direction (S H ), and in a vertical direction (S V ). Examples of this shift are shown in FIGS. 4E(a) and 4E(b). A horizontal shifting quantity S H  implies that the (i)&#39;th row of the bit-matrix is shifted up to the S H  (i), and a vertical shifting quantity S V  implies that the (j)&#39;th column of the bit-matrix is shifted up to S V  (j). The stain bit-matrices B DCC  and B DBC  with respect to the critical bit-matrices are defined in the following equations. ##EQU3## 
     FIG. 4F(a) shows an example of the stain bit-matrix B DCC  relative to the character critical bit-matrix shown in FIG. 3B(a). FIG. 4F(b) shows an example of the stain bit-matrix B DBC  relative to the background critical bit-matrix shown in FIG. 3B(b). Mask bit-matrices, B MCC  and B MBC , are obtained by inverting the individual elements of these stain bit-matrices. Such examples are shown in FIGS. 4G(a) and 4G(b). 
     The above described process is performed in steps 14, 15, 16 and 17 of FIG. 4 thereby completing the creation of a dictionary pattern. 
     FIG. 5 shows the flow of the development of matrices during the dictionary creating process. 
     To start with, a bit-matrix of the character to be learned is measured N L  times and divided into a character component B C  and a background component B B  (see 1 and 2 of FIG. 5). After the cumulation has been executed N L  -times, a character cumulative matrix C C  and a background cumulative matrix C B  are obtained (see 3 and 4 of FIG. 5). From the respective cumulative matrices are obtained basic bit-matrices, B OC  and B OB , and critical bit-matrices, B CC  and B CB  (see 5, 6, 7 and 8 of FIG. 5). The horizontal run-number and the vertical run-number are obtained from the basic bit-matrices (B CC , B OB ). From there, horizontal bit-matrices (B HC , B HB ) and vertical bit-matrices (B VC , B VB ) are obtained (see 9 and 10 of FIG. 5). Furthermore, from critical bit-matrices (B CC , B CB ), mask bit-matrices (B MC , B MB ) are obtained (see 11 and 12 of FIG. 5). A character pattern statistic is obtained from the character patterns which have been learned N L  -times (see 13 of FIG. 5). The character pattern statistic includes a mean width Wc, a mean height Hc, a mean area Ac, a normalization ratio λ W  X λ H , an area weight mean threshold value t A  and a threshold value median t M . 
     The character pattern statistic can be given by the following equations: ##EQU4## In the above equations, the normalized size is given as W N  X H N . Also, t c  (i) is the binary coded threshold value. Dictionary pattern data includes the horizontal, vertical and mask bit-matrices as well as the character pattern statistic. The parameters of the statistic have an initial binary coded threshold equal to t 0  and a critical threshold value equal to t 1 . t 2  is obtained when the area variation quantity is δ AO , and is used as a binary coding control parameter during the automatic reading. 
     The dictionary patterns are created with respect to the character categories which are recognized in the above described process. The resultant patterns are stored in the dictionary pattern memory 9 depicted in FIG. 1. 
     The microprocessor 6 executes a reading process in response to an active level on S 1 . FIG. 6 shows a flow chart of the reading process. 
     Data on the unknown character images is received (see 1 of FIG. 6) and the segment information is written into the image memory 5 where conductivity analysis is performed (see 2 of FIG. 6). Subsequently, the pattern information is acquired, and the character pattern is cut out in accordance with the width Wc and the height Hc of the character pattern (see 3 of FIG. 6). These processes are performed for every character pattern. Next, the normalization ratio is determined for the purpose of performing the normalizing process of the character pattern (see 4 of FIG. 6). 
     FIG. 6A shows a flow chart of a process in which the normalization ratio is determined. At first, a width, W 1  pixel, and a height, H 1  pixel, of the character pattern is obtained (see 1 of FIG. 6A). Then a vertical length ratio R 1  is calculated from the following equation (see 2 of FIG. 6A). ##EQU5## A vertical length ratio with a reference value Ro is given by Ho/Wo with respect to the reference values Wo for width and Ho for height. 
     In a great majority of cases, the character pattern is vertically lengthy and hence the vertical length ratio reference value Ro falls within a range of 1.0 to 2.0. The next step, (R 1  /R 0 )&gt;D 1 , determines whether the unknown character is vertically lengthy or horizontally lengthy in regard to a reference character pattern frame (Wo X Ho). A lower limit value of the vertical length ratio usually ranges from 0.8 to 1.2. If the unknown character is vertically lengthy, a height magnification of α H  =(H 1  /H 0 ) is obtained in the next step (see 4 of FIG. 6A). Thereafter, the width is estimated (see 5 of FIG. 6A). If [|W 2  -W 1  |≦D 2  ] is established in relation to the estimated width value of W 2  =α H  ·W 0  (see 6 of FIG. 6A), the normalization ratios, λW 1  and λH 1 , are determined from the width, W 1 , and the height, H 1 , of the unknown character pattern (See 7 of FIG. 6A). If the width estimation upper limit value, D 2 , is exceeded, the normalization ratios are determined from both an estimated width, W 2 , and from the height, H 1 , (see 8 of FIG. 6A). Similarly, when the unknown character pattern is horizontally long, a width magnification of α W  =(W 1  /W 0 ) is obtained from the width reference value Wo, (see 9 of FIG. 6A), and the height is then estimated (see 10 of FIG. 6A). If the estimated height value given by H 2  =α W  H 1  satisfies [|H 2  -H 1  |≦D 3  ] (see 11 of FIG. 6A), the normalization ratios are determined from W 1  and H 1 . If the height estimation upper limit value, D 3 , is exceeded, the normalization ratios are determined from W 1  and H 2  (see 12 of FIG. 6A). 
     After determining the normalization ratios (λ W1 , λ H1 , the complementary bit-matrices, B C  and B B , are obtained by adjusting the upper left side of the circumscribed frame of the unknown character pattern to the upper left side of the mesh which has a normalized size of W N  X H N  (see 5 of FIG. 6). A character stain bit-matrix B DC  (i, j) is given by formula (12) and a character stain quantity D MC  is given by formula (13) (see 6 of FIG. 6). ##EQU6## The character stain quantity, D MC , is obtained with respect to all of the character categories and classifying process I is executed on the basis of these results (see 7 of FIG. 6). 
     FIG. 6B shows a flow chart of classifying process I. First the character stain quantities, D MC , of all of the character categories are arranged in the order of increasing quantities. The next step is to determine sequentially, starting from the smaller quantity, whether or not they are lesser than or equal to the set value D MC1  (see 2 of FIG. 6B). When such quantities are lesser than or equal to the set value, they are stored as the first candidate characters (see 3 of FIG. 6B). If the upper limit set value, D MCI , is decreased, the number of candidate characters will be reduced and therefore the processing time will be shortened. The above described classifying process I is performed for every character (see 4 of FIG. 6B). 
     After classifying process I is completed, the character cut quantity is computed as below (see 8 of FIG. 6). 
     A vertical cut quantity bit-matrix, B CVC , of the character is obtained by the following equation: ##EQU7## A character horizontal cut quantity bit-matrix, B CHC , of the character is likewise obtained by the following equation: ##EQU8## 
     Furthermore, a cut bit-matrix B KC  of the character is obtained by the following equation: 
     
         B.sub.KC (i,j)=B.sub.CVC (i, j)+B.sub.CHC (i,j)            (16) 
    
     A character cut quantity D CC  is obtained by the following equation: ##EQU9## 
     This character cut quantity D CC  is obtained with respect to all of the character categories and classifying process II is executed on the basis of these results (see 9 of FIG. 6). 
     FIG. 6C shows a flow chart of classifying process II. In process II, dissimilar quantities, indicated by D C  =D MC  +D CC , of character are arranged in the order of increasing quantities (see 1 and 2 of FIG. 6C). When each of the dissimilar quantities, D C , is equal to or less than D C1  (see 3 of FIG. 6C), and is also equal to or less than N CH1  (see 4 of FIG. 6C), the character category and the dissimilarity quantity are, with these serving as the second candidate characters, stored in the memory (see 5 of FIG. 6C). The above described process is executed on all of the first candidate characters (see 6 of FIG. 6C). 
     So far, the stain quantity and the cut quantity of the character component have been described. The same processing is required to obtain the background component (see 10 of FIG. 6). ##EQU10## (The equation numbers 12-17 are marked with (&#39;) such as (14&#39;) because these equations correspond to equations 12-17 described earlier). FIG. 6D shows a flow chart in which the matrices are calculated. The resultant stain quantity and cut quantity are used as evaluation indices when reading the characters. 
     The next phase of the reading process is the recognition process (see 11 of FIG. 6). FIG. 6E shows a flow chart of this process. 
     With respect to the previously obtained second candidate characters, a background dissimilarity quantity D B  =(D MB  +D CB ) and a total dissimilarity quantity D T  =(D C  +D B ) are obtained (see 1 and 2 of FIG. 6E). Subsequently a character category K 1  is established such that the total dissimilarity quantity D T  is decreased to its smallest value. This smallest value is D T1  (see 3 of FIG. 6E). A character category K 2  is obtained with the total dissimilarity quantity D T  decreased to a minimum (D T2 ) (see 4 of FIG. 6E). As a result, when satisfying both D T1  ≦D T3  and |D T2  -D T1  |≧D T4  (see 5 and 6 of FIG. 6E), the reading result of the unknown character is K 1  (see 7 of FIG. 6E). When not satisfying the above equations, a rejection exists (see 8 of FIG. 6E). A reading number N C  and a rejection number N R  are counted in connection with the character category K 1 . The above mentioned recognition process is executed on all of the unknown characters and the results are transmitted on output line S 2  (see FIG. 1). 
     Finally the contents of judgment are totalized (see 12 of FIG. 6). FIG. 6F shows a flow chart of this process. 
     To be specific, D MC , D CC , D C , D MB , D CB , D B  and D T  are totalized with respect to the character K 1  (see 1 of FIG. 6F). Cumulation of the complementary bit-matrices is executed on the character K 1  (see 2 of FIG. 6F). The totalized data is used for modifying the dictionary patterns. Therefore it is possible to acquire still more reliable standard patterns by executing both the processing operation of the basic bit-matrix, which is shown in FIG. 2, and the creation of the dictionary patterns through the cumulative matrices of the totalized data. The dictionary patterns are modified in this way. Also, it is possible to determine whether or not the dictionary patterns are adequately modified by analyzing the rejection rate.