Pattern recognition by generating and using zonal features and anti-features

Abstract: A system for pattern recognition and a system for generating feature masks being used in the pattern recognition process is described which utilizes beside pieces covering a master and sample pattern also pieces not covering the area of the master and sample pattern. Thus, there are formed zonal features and anti-features having positive and negative feature factor values as well as incorporating different weighting factors. This system allows a fully automatic feature generation technique and allows training of new character classes in a very short time. For each meaning class the different shaped classes are generated by statistical means and the weighted zonal positive features are derived. Besides this in parallel there are weighted anti-features trained which increase the accuracy of classification substantially. The system of the present invention implements a classifier of high accuracy for recognition of for example characters with low print quality according to banking requirements.

Description: The present invention pertains to a system for pattern recognition and a system for generating feature masks being used in the pattern recognition process. The recognition of pattern, for example in the form of printed or written text by optical means is existing as long as digital image processing exists. Generally speaking, pattern recognition means the classification of a given unknown pattern, for example a text pattern, consisting of characters and numbers, into one of a number of standard classes, i. e. characters in the ASCII code table. This recognition process is usually done in different steps as explained in the following in general terms: The image of a document is scanned, for example in vertical scans when it passes under a read-head. As read-head e.g. a charged coupled device (CCD) array can be used. The raw image is captured at a resolution of, for example 240 PELS/inch (picture element/per inch). Each PEL in the raw image might be an eight bit quantization of the intensity of the picture element measured by the CCD array. In a next step a thresholding process is applied to the PELS of the raw image. This thresholding process might be a dynamic one or a fixed thresholding value can be used. In the thresholding process the intensity of each PEL is clipped to either black or white for generating a binary black/white image. The generated black/white video image is input to a segmentation or framing process in the next step. In the segmentation process characters inside a trackband are identified and the characters or generally speaking the patterns are registered into a frame. The next performed step is called feature extraction. The feature extraction process uses information about the character font or generally speaking information about the different forms and shapes of the specific set of patterns. These features are Boolean expressions of characters or pattern fragments. In the document readers IBM 3890/XP, 3891/XP and 3892/XP which are widely used and known for character recognition especially in the banking area, all features are designed manually to allow high discrimination capability between the character classes of the used font. For example, there are necessary 96 features to do recognition of the OCR A font. For the also well known OCR B font there are 102 features necessary. It is an extreme disadvantage of this manual mode that additional fonts require a new feature design development cycle, training runs and fine tuning of the features. Furthermore, there exists a dependency between the feature design and the feature extraction and this process is an extremely labor-intensive one, often requiring a team of engineers to complete the task for a given font. To save processing time in the feature extraction process, the number of bits tested in the input character image matrix must be kept to a minimum. This necessitates a careful trade-off between reliability of the feature extraction process and the speed of its execution. Also aided by batch-oriented computer programs, the task of picking which bits to test in the input character image matrix falls to the recognition system development engineer. As the practical feature design cannot be carried out in isolation from the task of designing the process to extract the features, this feature and feature extraction design process is a very labor-intensive and cumbersome one. As already mentioned, each feature is described by a "Boolean" expression and contains AND and OR terms. The output of the feature extraction process is a feature vector. Each feature is addressed by a bit in this feature vector f. The final step of the recognition process is called the linear decision process. A coefficient storage or weight table is applied to the feature vector f to calculate a class sum for each character class. The character class with the highest class sum is taken if the highest class sum diminished by the next highest class sum is greater than a certain confidence level. From IBM Technical Disclosure Bulletin, Vol. 34, No. 4A, September 1991, pp. 469-472, a method is known for automatic character recognition suitable for machine printed fixed-font characters. This known method intends to eliminate the highly time- and labor-intensive process for feature design and feature extraction. For that purpose it is proposed to input a labelled training set immediately to a neural network. The neural network training algorithms discovers salient features of character images within the training set and uses these self-generated features in the classification process. This known method makes use of a fully interconnected feed-forward artificial neural network, not disclosing in further details the underlying methodology. Thus, an application to a classifier not including an artificial neural network is not possible. In document D1, Y. Hongo: "Character Reading Method", United States Patent No. 4,887,303, filed: February 14, 1989, Date of Patent: December 12, 1989, a method is described that determines unknown character by comparing matrices representing the unknown character with matrices representing known dictionary characters and determining the deviation between matrices for the unknown and the dictionary characters. In order to extract small differences between local portions of the characters, matrices for the character and for the background surroundings of the character are obtained. Document US-A-5 204 914, published April 20, 1993, discloses emphasizing those pixels of a template that distinguish a trained character from a similar one (like O and Q). It does so by repeatedly changing the weights of the pixels until the correlation coefficients with all the other characters in the font fall below a threshold. It is the object of the present invention, to provide a method and a system for pattern recognition and for generating feature masks in an automatic way, thus avoiding the time-consuming manual feature design process and also providing a system that is able to train a classifier not necessitating a artificial neural network. This object as well as other objects are basically solved by applying the features laid down in the independent claims. Further advantageous embodiments of the present invention are laid down in the dependent subclaims. The advantages of these embodiments are either self-explaining or explained later-on in connection with the specific description. The feature generation technique in accordance with the present invention uses zonal features and anti-features, thus allowing a fully automated feature generation technique which allows also training of new character classes in a very short time of, for example few minutes. The system is suitable for concurrent learning through normal operation. For each meaning class the different shape classes are generated by statistical means and the weighted zonal positive features are derived. In parallel to that weighted anti-features are trained which increase the accuracy of classification substantially. The classification process is repeated iteratively with decreasing number of probable candidates following a decision scheme. Thus, the invention provides a classifier implementing high accuracy recognition, especially of code line font characters with low print quality. Thus, banking requirements are fulfilled also considering stroke brakes, voids and unexpecting touching strokes that are normal in the banking environment. In the following more detailed description the invention will be explained in connection with embodiments shown in the drawing in which: Fig. 1 shows an example for features design manually; Fig. 2 shows an example for the linear decision process used in connection with systems having manually designed feature extraction; Fig. 3 shows examples of ideal masks; Fig. 4 shows examples of feature masks; Fig. 5 shows a feature class example for the character "0" of the OCR A font; Fig. 6 shows the several features of the character "0" in different grey levels representing different weights of the respective zonal features; Fig. 7 shows the anti-features of the character "0", also in different grey values representing different weighting of the anti-features; Fig. 8 shows the zonal features in different grey levels representing different weights of the number "4"; Fig. 9 shows the anti-features in grey level representations of the number "4"; and Fig. 10 shows examples for a recognition. Fig. 1 shows an example for feature extraction on the basis of the character "3". This number is shown in a pixel grid. Indicated are feature # 1, the upper left tip, feature # 2, the lower vertical bar, and for feature # 3 is the lower right curve indicated by an arrow. Dependent on the used font these features 1 to 3 are different in accordance to the different fonts, even with the character "3". Those features 1 to 3 in accordance with the prior art as used in the IBM document readers 3890, 3891 and 3892, are designed manually as far as shape is concerned and also as far as the selection which feature to take is concerned. Also for a better understanding, the known linear decision process is described in connection with the feature vector including the features # 1, # 2 and # 3 as shown in Fig. 1. The feature vector has the binary form 1101 including the features numbers or measurements # 1, # 2 and # 3 (wherein a 0 in the feature vector means a miss and a 1 means a hit). Out of these feature vectors a product is built in the coefficient storage 21 which is associated to the different feature numbers in the columns and to the different classes 0, 1, 2 and 3 as indicated in box 22 in the different rows. In the box 23 the result of the class sum is shown. For example the sum for the class 0 is generated by adding the numbers -5, -20 and -50 of the first row as those feature or measurements where present in the feature vector whereas feature # 2 is missed and hence the class 80 coefficient. This is done for all classes and as shown the highest result 182 is given for the class 3 and the next highest sum 39 is given for the number "2". This is indicated in the candidate table 24. In the comparison step 25 the highest class sum 182 is compared with the next highest class sum 39 against a constant value. If the difference is greater than a constant representing a confidence level, then this class associated with this sum is taken. Thus, in the decision box 26 the decision is indicated that the sum 182 pertains to the # "3". So in the linear decision process that character class having the highest class sum is taken if the highest class sum diminished by the next highest class sum is greater than a constant confidence level. Generally speaking the basic idea of the present invention is the utilization of pieces of sample patterns which do not cover the areas of the master characters or pattern. This is done beside using pieces which cover a master and sample pattern or character respectively. By using pieces of the sample pattern which do not cover areas of the master character, the probability is decreased of an assignment to a master character which does not have matching character pieces in this area. A decisive factor and advantage of the present invention is that the features used for the classification are not designed manually as in the prior art described above but are the result of an automatic training process. Thus, the implementation of the feature from generation is much faster and less expensive then in the traditional process. Tests have shown that the improvement is in the range of 60 : 1. In the training process in accordance with the present invention not only the matching character pieces are weighted and provided with weighting factors but also not matching character pieces of the training sample pattern are provided with weights. The result of that are negative and positive feature vectors. This increases the information distance between the classes drastically and thus the recognition process is faster, more effective and much more reliable. As in accordance with the present invention matching and non-matching pieces of a mask and a sample pattern are considered in the recognition process, there has to be fulfilled one prerequisite which are normalized, well aligned character pattern. In the implementation this prerequisite of normalized well aligned character pattern can be achieved by the following steps: 1. Capturing the image of the text in a sufficient number of grey levels, for example 256. 2. Preprocessing the image by a) intelligent clipping with, for example a dynamic clipping algorithm, to to get a black/white image, b) cleaning of the background around the character area to remove dirt. 3. Extracting the text lines to be recognized. 4. Segmentation of the print line into characters or pattern respectively. 5. Character-normalization by for example: a) slant correction of each character, thus normalizing of the character angle, b) normalization of the stroke width, i. e. expanding or compressing by arithmetic image processing methods the width of the character strokes, c) normalization of the size, i. e. scaling the height and width of a character by means of arithmetic image processing methods, so that a standard size is always achieved. This standard size is optional but has to be fixed and equal for the training and recognition part. 6. Alignment of the character or pattern respectively, i. e. the digitized character or pattern have to be aligned in a two-dimensional recognition pattern matrix. The size of this matrix is optional but has to be fixed and equal for the training and recognition part. If the normalization process takes place, also capital handprint characters and/or numbers can be processed by this invention. For the generation of the feature masks a further prerequisite is the existence of a sufficient set of training samples of the character font or the set of pattern to be recognized. The shape of the characters of a font are represented by a training deck with the images of all characters of the font. This must include bad, normal and good samples, i. e. high as well as low quality characters and it has to be representative. The result of the training phase is the classification of a given font into character classes or more generally speaking the classification of a given set of pattern into the pattern classes. In accordance with a preferred embodiment of the present invention the implementation is performed in different steps. The step 1 can be called the generation of ideal masks K i . For each of the n character classes i of a font a binary ideal mask K i with i = 1, 2, ... n and with K ε {K 1 , K 2 , ... K n } is calculated. The basis for this calculation are the normalized binary character images of the representative training deck which consists of m i with i = 1, 2, ... n characters for each of the n character classes. The ideal mask K i is built by adding up all m i character masks of a character class i on a pixel basis. n i is the number of characters of the class i. The result of this adding up is a pattern matrix with values between 0 and m i for each pixel dependent on the frequency of pixels in a certain position in all the character pattern. To get a binary pattern the resulting matrix is clipped by a value C with 0 ≤ C ≤ m i . Thus, again a black/white binary pattern representation is generated. The value of C = a o x m i . The value of a o is in the range of 0.5 to 0.8 for practical cases. It is depending on the scanning device. All pixel values > C are set to 1, else to 0. In Fig. 3 there is shown an example of such an ideal mask for a given font with only three characters of the classes "3", "8" and "-". These ideal masks K3, K8 and K- are represented by pixel values 1 in the positions occupied by the ideal mask. They are shown to be within a normalized frame thus having size and form which is the basis for the later recognition phase. The ideal mask K3 for the character 3 is indicated by a 1 schematically at each position where this font requires to be one of the three. The same is true for the ideal mask K8 for the character 8 and for the ideal K- for the symbol "-". The next step in accordance with the present invention is the feature mask initialization process in which the feature classes f i are calculated. For this calculation of the feature classes f i each ideal mask K i of the class or mask respectively i in the font is multiplied by (n + 1), the number of characters in the font +1. From the result of this multiplication the sum of all n existing ideal masks is subtracted. The calculation is performed on a pixel basis in the character matrix and the result is the feature class f i for the character or pattern i. This follows the formula: In Fig. 4 there are shown examples of feature masks f3, f8 and f-. Besides these three features is shown the sum of the three existing ideal character masks K3, K8 and K-. If, for example, the feature mask for the character 3 has to be calculated, the pixel mask values in the ideal mask K3 shown in Fig. 3 is multiplied by four and from the result the sum shown in Fig. 4 on the right hand is subtracted. The result is shown as f3 in the left hand part of Fig. 4. The same scheme is applied for calculating the feature masks f8 and f- shown Fig. 4. As furthermore depicted in Fig. 4 in the feature mask f3 there exist two areas 41 in which the indicated values are -1. In all the other areas of the feature class f3 the values are 1 or either 2. The positive values can be seen to be zonal features and the negative values can be seen to be the anti-features in accordance with the present invention. Their weight, negative as indicated in the figure, is extremely helpful in distinguishing this feature mask f3, for example from the feature mask f8. There we have in the same position areas 42, as indicated by character 3 increased positive zonal features for the character 8 in this area 42. The contribution on the anti-features in accordance with the present invention can be seen very clearly in the feature mask f- where only the area 43 contains positive values, indicated by character. All other values in the form of the "8" are either -1 or -2 values, thus indicating that in these areas there exists strong anti-features in other areas like the area 44 or 45. During the step of feature class initialization there has to be calculated also a normalization factor R. This normalization factor R is the total of all pixel values of a matrix which is the result of the multiplication of f i x K i with v columns and u rows. This is given by the formula: The values for the different normalization factors associated to the feature classes f3, f8 and f- are indicated as R 3 = 30, R 8 = 42 and = 2 in Fig. 4 beneath the associated feature masks. In Fig. 5 there is shown an example for a feature mask for the character 0 of the OCR A font. Within the lines there are shown different positive weight factors. Outside the 0 and within the 0 there are shown areas with 0 weight factors and negative weight factors. There are zones having high weight factors, high positive as well as high negative weight factors. The example shown in Fig. 5 shows a real life example of a feature mask. It was the result of a training run with a life data training deck. The shown weight factors, indicated by the numbers in the matrix are the numbers with which the pixels will be multiplied in the recognition phase. In Fig. 6 there is shown a slightly different illustration of what is shown in Fig. 5. The different zones having different positive weight factors of a 0 are shown by different grey levels. As darker the grey level is, as higher the positive weight factor is. In Fig. 7 there is indicated the weight factors of the anti-feature of the 0. Also here the importance of the weight factor is indicated by the grey level. It can be seen that as higher the negative weight factor is, as darker is the representation in Fig. 7. For demonstration purposes in Fig. 8 the same scheme is used to show the different positive weight factors in the feature mask of the character 4. There also the darkest areas indicate the most important or highest respectively zonal weight factors. In Fig. 9 the negative values are indicated in accordance with the same scheme, representing the anti-features. Until now the most important aspect of the invention has been described having the advantage that the invention provides an automatic feature generation process as a result of the described new learning or training processes respectively combined with intelligent features with negative and positive values increasing the information distance. These intelligent features will support zonal features and anti-features. Finally the recognition process should be briefly discussed. During the recognition phase sample character image pattern, as result of a scanning and normalization process, will be classified. That means that every character image will be assigned to a mask or class in a font of standard characters, i. e. OCR A. A prerequisite for this method is that the normalization process of the sample pattern after scanning is equal to the normalization which took place in the training phase. This is clear as a pixel by pixel comparison and multiplication has to be performed and the areas to be compared have to be congruent. During recognition each sample character matrix S k , with k being the sample sequence number, is multiplied with all feature masks or classes f i of the mask or class i of the font, divided by the normalization factor R i . The result is the determination value D i . This follows the formula: D i = S k × f i R i . The maximum value of D i is 1, but is normally lower than 1 and even negative. Only perfect sample characters show a determination value D of approximately 1. The decision which character class or mask is closest to the sample character image depends now on the value of D. The class with the maximum D i is taken as the most likelihood character or pattern in the font. That means S is recognized as class or character i if: i → S if D i = max. In Fig. 10 there are shown examples for the recognition of sample character matrixes S1 in Fig. 10A, S2 in Fig. 10B and S3 in Fig. 10C. The samples shown in Fig. 10 correspond to the characters depicted in Fig. 3 and Fig. 4. In the right hand box of each figure the different determination values D3, D8 and D- are indicated in relation to the different samples. It is easy recognizable where the determination value has its maximum and thus to which character in the font the input sample pertains.

Claims: A method for pattern recognition, comprising the step of: comparing a pattern (S1, S2, S3) with feature masks (f3, f8, f-), each of said feature masks (f3, f8, f-) representing a specific character ("3", "8", "-"), characterized in that each of said feature masks (f3, f8, f-) comprises zonal features (42, 43), representing the character itself, and anti-features (41, 44, 45), representing its background, and information about more than one character, namely information about all characters ("3", "8", "-") in the set of characters to be recognised, is utilized for the generation of said zonal features (42, 43) and anti-features (41, 44, 45) of each of said feature masks (f3, f8, f-). The method for pattern recognition according to claim 1, wherein said feature masks (f3, f8, f-) are generated comprising the steps of: a) adding up, on a pixel basis, m i patterns, with i = 1, 2, ..., n, for each of the n character classes obtained from normalized binary pattern images of a representative training character set including bad, normal and good samples of said character, resulting in a sum pattern mask for each of said n characters with values between 0 and m i for each pixel, representing a respective character i, said sum pattern mask being subjected to a thresholding process using a value C to generate a black/white binary pattern representation, whereby C is between 0 and m i , setting all pixel values > C to 1, else to 0, resulting for each i of said n characters in an ideal character mask K i (i = 1, 2, ..., n), b) calculating for each i of said n characters one feature mask f i (i = 1, 2, ..., n) by multiplying, on a pixel basis, said ideal character mask K i (i = 1, 2, ..., n) by (n + 1), and subtracting, on a pixel basis, the sum of all n existing ideal character masks K i (i = 1, 2, ..., n) (K3, K8, K-), said calculation following the formula: The method for pattern recognition according to claim 2, wherein said generation of feature masks (f3, f8, f-) is performed automatically, particularly is performed concurrently with normal pattern recognition operation. The method for pattern recognition according to claim 2 or 3, further comprising the step of: calculating a normalization factor R i (i = 1, 2, ..., n) for each i of said n characters, said normalization factor R i (R3, R8, R-) being the sum of all pixel values of a matrix, said matrix being the result of a multiplication, on a pixel basis, of f i x K i with v columns and u rows, said calculation following the formula: The method for pattern recognition according to claim 4, further comprising the step of: multiplying, on a pixel basis, each sample pattern image matrix S k (S1, S2, 53) with all feature masks f i (f3, f8, f-) divided by said corresponding normalization factor R i (R3, R8, R-), resulting in a determination value D i (i = 1, 2, ..., n) for each i of said n characters corresponding to said sample pattern image matrix S k (S1, S2, S3), wherein said sample pattern image matrix S k (S1, S2, S3) is normalized after scanning in equal manner as said binary pattern images of said representative training character set during training and generating phase, and k is a sample sequence number, said multiplication following the formula: D i = S k × f i R i . The method for pattern recognition according to claim 5, wherein said determination value D i (D3, D8, D-) has an absolute maximum value of "1", said absolute maximum value "1" being associated to a perfect sample pattern, whereas normally said determination value D i being lower than "1" or even negative. The method for pattern recognition according to claim 5 or 6, wherein deciding which character is closest to said sample pattern image matrix S k (S1, S2, S3) is made dependent on said value of D i (D3, D8, D-), whereby character i, corresponding to feature mask f i having a relative maximum value D i,(max, rel) within i = 1, 2, ..., n, is taken as the most likely character, i. e., S k (S1, S2, S3) is recognized as character i ("3", "8", "-"), respectively if: i → S k if D i = S k x f i /R i has a relative maximum D i,(max, rel) within i = 1, 2, ..., n. The method for pattern recognition according to any one of the preceding claims, wherein said patterns are characters and/or numbers and said patterns are part of a font and said font is preferably a standarized font and/or an image pattern as logos, pictograms and textures. The method for pattern recognition according to any one of the preceding claims, wherein said patterns are normalized capital handprint characters and/or numbers. A system for pattern recognition, comprising: means for comparing a pattern (S1, S2, S3) with feature masks (f3, f8, f-), each of said feature masks (f3, f8, f-) representing a specific character ("3", "8", "-"), characterized in that each of said feature masks (f3, f8, f-) comprises zonal features (42, 43), representing the character itself, and anti-features (41, 44, 45), representing its background, and said system for pattern recognition further comprises means for generating said zonal features (42, 43) and anti-features (41, 44, 45) of said feature masks (f3, f8, f-), which utilize information about more than one character, namely information about all characters ("3", "8", "-") in the set of characters to be recognised, for the generation of each of said feature masks (f3, f8, f-). The system for pattern recognition according to claim 10, wherein said means for generating said feature masks (f3, f8, f-) comprise: a) means for adding up, on a pixel basis, m i patterns, with i = 1, 2, ..., n, for each of the n character classes obtained from normalized binary pattern images of a representative training character set including bad, normal and good samples of said character, resulting in a sum pattern mask for each of said n characters with values between 0 and m i for each pixel, representing a respective character i, said sum pattern mask being subjected to a thresholding process using a value C to generate a black/white binary pattern representation, whereby C is between 0 and m i , setting all pixel values > C to 1, else to 0, resulting for each i of said n characters in an ideal character mask K i (i = 1, 2, ..., n), b) means for calculating for each i of said n characters one feature mask f i (i = 1, 2, ..., n), including: means for multiplying, on a pixel basis, said ideal character mask K i (i = 1, 2, ..., n) by (n + 1), and means for subtracting, on a pixel basis, the sum of all n existing ideal character masks K i (i = 1, 2, ..., n) (K3, K8, K-), said calculation following the formula: The system for pattern recognition according to claim 11, wherein said generation of feature masks (f3, f8, f-) is performed automatically, particularly is performed concurrently with normal pattern recognition operation. The system for pattern recognition according to claim 11 or 12, further comprising: means for calculating a normalization factor R i (i = 1, 2, ..., n) for each i of said n characters, said normalization factor R i (R3, R8, R-) being the sum of all pixel values of a matrix, said matrix being the result of a multiplication, on a pixel basis, of f i x K i with v columns and u rows, said calculation following the formula: The system for pattern recognition according to claim 13, further comprising: means for multiplying, on a pixel basis, each sample pattern image matrix S k (S1, 52, S3) with all feature masks f i (f3, f8, f-) divided by said corresponding normalization factor R i (R3, R8, R-), resulting in a determination value D i (i = 1, 2, ..., n) for each i of said n characters corresponding to said sample pattern image matrix S k (S1, S2, S3), wherein said sample pattern image matrix S k (S1, S2, S3) is normalized after scanning in equal manner as said binary pattern images of said representative training character set during training and generating phase, and k is a sample sequence number, said multiplication following the formula: D i = S k × f i R i . The system for pattern recognition according to claim 14, wherein said determination value D i (D3, D8, D-) has an absolute maximum value of "1", said absolute maximum value "1" being associated to a perfect sample pattern, whereas normally said determination value D i being lower than "1" or even negative. The system for pattern recognition according to claim 14 or 15, having means for deciding which character is closest to said sample pattern image matrix S k (S1, S2, S3), said decision being made dependent on said value of D i (D3, D8, D-), whereby character i, corresponding to feature mask f i having a relative maximum value D i,(max, rel) within i = 1, 2, ..., n, is taken as the most likelihood character, i. e., S k (S1, S2, S3) is recognized as character i ("3", "8", "-"), respectively if: i → S k if D i = S k x f i /R i has a relative maximum D i,(max, rel) within i = 1, 2, ..., n, The system for pattern recognition according to any one of claims 10 to 16, wherein said patterns are characters and/or numbers and said patterns are part of a font and said font is preferably a standardized font and/or an image pattern as logos, pictograms and textures. The system for pattern recognition according to any one of claims 10 to 17, wherein said patterns are normalized capital handprint characters and/or numbers. The system for pattern recognition according to any one of claims 10 to 18, wherein a neural network is used for generating ideal character masks K i (K3, K8, K-) as well as for calculating said feature masks f i (f3, f8, f-).

Inventors: MAIER UDO, RUPPERT WERNER, MAIER, UDO, RUPPERT. WERNER
Applicants: IBM, INTERNATIONAL BUSINESS MACHINES CORPORATION
Application year: 1993
Main classification: G06K9
IPC codes: G06K9, G06T7