Improved segmentation process for machine reading of handwritten information

Abstract: An improved machine process for recognizing the transition point between letters and digits in handwritten text is disclosed. The script, digits and/or letters, is mapped onto a pixel matrix. The entropy for each column for a range of assumed slant angles is analyzed to determine the angle at which image entropy is a minimum. By reverse-shifting the pixel array an amount equal to this angle, the slant component of the script is verticalized and the segmentation step which follows is more accurately accomplished.

Description: Field of the Invention This invention relates to optical character recognition. More specifically, the invention relates to improved processes for recognizing letters and digits in handwritten text by improving the step of segmenting the connected letters/digits into individual components. The U.S. Postal Service has rights in this invention pursuant to Task Order Number 104230-90-C-2456. Background of the Invention One of the promising applications of automated script reading technology is the machine reading of script-written names and addresses in postal systems. To separate handwritten text such as street address and zip-code information into discrete digits and letters, the handwritten strokes or script must be separated. In the art of machine reading of script, this step is called "segmentation." Typically, individual handwriting exhibits a characteristic global slant. By eliminating slant in a specimen of script, it becomes easier to machine-slice the script into its letter or number components. One approach to eliminating the slant factor, is to manipulate the individual handwriting strokes to maximize the verticality of the strokes. The characteristic slant angle thus is essentially removed; and the algorithms for operating on the still-connected script to separate it into discrete letters and numbers can then be deployed more successfully. Obtaining a relatively reliable estimate of the global characteristic slant angle in a given specimen of script has proven difficult in practice. One reason is the wide individual variability of slant and style in handwriting. Slant angles encountered in script span a range of many degrees. Further, there is an uncertainty of knowing whether the slant component in a given element of script is due to the inherent shape of the letter/number, or to the writer's penmanship. Another reason is that some letters and numbers, even if consciously written without slant, have shapes that inherently contain slanted segments. Further, there is the fundamental question of how to define "characteristic slant angle" of script in terms that can be simply translated to manipulatable data elements. In IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 11, No. 1, January 1989, pages 68-83 there is an article entitled "Off-Line Cursive Word Recognition" by R.M. Bozinovic. In that article a slant correction algorithm for cursive script is suggested. Each word is considered and portions considered irrelevant (long horizontal runs and small width horizontal strips) are blanked out. The remainder of the word is divided into individual windows by vertical lines. The upper and lower halves of each window are considered and the centres of gravity of the two halves connected to define an average slope for the word. The defined slope is used to modify the coordinates of the individual pixels making up the word to reduce the effect of the slant. In EP-A-301,384 a method of slant correction is described which comprises considering each letter and scanning that letter at different angles. The base line shadow resulting from each selected angle of scan is noted. The scanning angle at which the base line shadow is a minimum is deemed to be the slant angle. In US-A-3,831,146 an arrangement is described for determining the optimum angle to scan characters for subsequent character recognition. A scan of a character generates video data which is stored in a two-dimensional matrix. Portions of the video data in selected areas of the matrix are fed to a measurement logic set from which the angularity of the characters can be determined. For each instance when the logic requirements are satisfied for a particular angle a count is made. The angle giving the highest count is deemed to be the optimum scanning angle. Objects of the Invention Accordingly, one object of the invention is to improve the accuracy of recognition of handwritten digits and letters. A further object of the invention is to improve the process of removal of the slant from handwritten digits and letters imparted by the writer's characteristic style of penmanship. A further object of the invention is to improve processes for reading of isolated handwritten zip-code digits. A more specific object of the invention is to provide a machine process that increases the probability of correctly identifying the characteristic slant angle present in letters and digits in handwritten script, thereby to correctly identify the border between advancement letters or digits. Summary of the Invention According to the invention in the machine-reading of handwritten script, a method for normalizing the characteristic slant angle in the script, CHARATERIZED BY the steps of: generating a pixel image of the original handwriting; measuring the number of dark pixels falling each pixel column, and the total number of dark pixels in the image; calculating the ratio of dark pixels in each column to total dark pixels in said image; calculating the entropy values of the said columns of said original image and storing said values ; successively shifting the individual rows of pixels in the total image in incremental amounts to vary the slant angle from that exhibited in the writer's characteristic script, and calculating and storing the successive entropy values ; determining the minimum value of the stored entropy values ; measuring the extent of pixel shift between the pixel positions of said original image and the pixel positions of the image that resulted in the minimum entropy column value ; converting the last-named measure to an angular value; and shifting the pixels rows of the image which yielded the minimum entropy column value by the amount of said angular value. Also according to the invention an apparatus for normalizing the characteristic slant angle in handwritten script CHARACTERIZED BY: means for generating a pixel image of the original script; means for measuring the number of dark pixels falling in each pixel column, and the total number of dark pixels in the image; means for calculating the ratio of dark pixels in each column to total dark pixels in said image; means for calculating the entropy values of said columns of said original image, and storing said values ; means for successively shifting the individual rows of pixels in the total image in incremental amounts to vary the slant angle from that found in the handwritten script; means for calculating and storing the respective successive entropy values ; means for determining the minimum entropy value of the plurality of stored entropy values ; means for measuring the extent of pixel shift between the pixel positions of said original image and the pixel positions of the image that resulted in the minimum entropy column value ; means for converting the last-named measure to an angular value; and means for shifting the pixel rows of the image having the minimum entropy column value by the amount of said angular value. In the invention, handwritten digits and letters are mapped onto a pixel matrix of a predetermined X-Y granularity, and the entropy for each column is analyzed for a range of assumed slant angles. The angle at which the entropy is found to be a minimum is, on average, the angle at which the distances or gaps between adjacent letters/digits is maximized. With this information determined, the letter/digit pixel mapping is altered, by shifting each pixel row in a direction opposite to the slant, thereby to remove, or normalize the global slant. After global de-slanting, if no appreciable vertical overlap from one digit to the next is present, then segmentation can be accomplished by a machine dissecting of the script object into vertical slices. Brief Description of the Drawing FIGS. 1 and 2 are replicas of pixel screens illustrating the placement of dark pixels defining script at different slant angles; FIG. 3 is a graph showing entropy as a function of slant angle; FIGS. 4 and 5 are similar to FIGS. 1 and 2, illustrating pixel representation of a number; FIG. 6 is a graph plotting the same functions as the graph of FIG. 2, but for a discrete number; and FIG. 7 is a high-level functional block diagram of apparatus for practicing the invention. Detailed Description of an Illustrative Embodiment By way of background, it will be recalled that C.E. Shannon, in his paper "A Mathematical Theory of Communications" Bell System Technical Journal, Vol.27, No.3, taught that a discrete information source can be represented by a Markov process. Further, the quantity of information produced in such a process can be measured by entropy, H: where p i is the probability of occurrence of the ith symbol. The N symbols are assumed to be uncorrelated. In bit representation, the logarithm in the above equation becomes 2-based. It can be shown that H is maximum when p 1 =p 2 =....=p N =1/N. Also, H is minimum when only one type of event is possible (P i = δ ik ). Specifically, if n bits are used to represent the states or types of events, then: H min =0 The pixel data is manipulated in accordance with the invention in general in the following steps. First, a pixel image of the original handwriting is captured by imaging cameras. This original image reflects the writer's characteristic penmanship style, including any slant component. For this original pixel image, the number of dark pixels falling in each pixel column, and the total number of dark pixels in the image, are counted. Next, the ratio of columnar dark pixels to total dark pixels is determined for each pixel column. Now, in accordance with the invention, the entropy of the column projections is computed, in accordance with the relationship stated in Equation 1. The resultant value is the entropy of the pixel image at the particular characteristic slant angle of the writer, without yet applying any variations to the slant. In accordance with a further aspect of the invention, the individual rows of pixels in the total stack are each shifted a first time, with the objective of varying the slant angle from that exhibited in the writer's characteristic script. The varying may be accomplished, for example, by moving each individual row an amount that is lineally proportional to its distance from a stationary reference row. Thus, for example, the bottom-most pixel row is kept stationary, the top-most row is shifted to the right by an amount equal to the length of a pixel side, and the intermediary rows shifted proportional amounts. The effect of the shifting is to present a revised total pixel image in which the slant component has been varied from the nominal or original. The preceding step is repeated a plurality of times, in which for each time, the top row (to use the same example) is shifted successively greater distances, while keeping the bottom row stationary, and moving the intermediary rows proportionately. With each shift, the number of dark pixels falling in each pixel column are counted; and the ratio of columnar dark pixels to total dark pixels is determined for each pixel column. The entropy of the individual column projections is computed, in accordance with the relationship stated in Equation 1. The shifting of pixel rows is effected in increments both to the right and to the left. As a result, the slant angle is varied over a range of, for example, 10 degrees from the normal in a clockwise direction and 40 degrees from the normal in a counterclockwise direction. The unique value of the entropy associated with the pixel image after each shift, is then evaluated to determine minimum entropy value in accordance with Equation 3. The slant angle associated with this minimum entropy value, is the predominant slant angle present in the writing, and therefore determines the amount by which the original pixel array should be shifted in order to remove the slant component of the writing. Finally, the pixel rows of the original image are shifted to the left or right proportionally by that angle, or linear shift amount, that reduces the characteristic slant to zero. The shift direction is opposite to the slant in order to remove the global slant. At this stage, the handwriting is essentially made vertical no matter what slant angle it contained originally, and is ready to be segmented. The following examples will further illustrate the de-slanting process of the invention. In the first example, as seen in FIG. 1, the handwritten word "York," comprises part of an address on an envelope. The elongate strokes show a decided slant, in a clockwise direction from the vertical. The envelop, denoted 1 and shown in FIG. 7, is moved on a conveyer means (not shown) into position under a scanning video camera 10. A processor 11, which for example can include an AT&T PC6300, receives the video signal raster from camera 10. In conventional fashion, processor 11 inputs to its memory 12, and also to a monitor 13 if desired, an image of the word "York." The pixel representation is as shown in FIG. 1; but it is understood that the information from which the dark and light pixels are created, is contained in memory 12. Advantageously, pixel density can be 300 per inch, or any other useful density of choice. In accordance with the invention, the original pixel image information in memory 12 is analyzed. The number of dark pixels falling in each pixel column and the total number of dark pixels are calculated; and the ratio of dark pixels to total dark pixels is determined for each vertical column. Entropy of the column projections is computed in processor 11 in accordance with Equation 1. The information comprising the individual pixel rows is now successively shifted in increments to vary the slant angle from the slant angle present in the original script. Following each shift, the entropy of the individual column projections is computed, as per the relationship of Equation 1. FIG. 3 shows the range of slant angles varying from -35 degrees to +10 degrees over which the entropy value is calculated. In this example, therefore, over forty measures of entropy are developed, at increments of one degree of slant sample angle. The minimum entropy value in accordance with Equation 3, is seen to occur at a slant angle seen in FIG. 3 to be approximately -12.5 degrees. Thus, the predominant slant angle present in the word "York" is 12.5 degrees. Processor 11 is controlled to recognize the minimum entropy value and the corresponding slant angle; and to apply that angle to the original pixel grid of FIG. 1 by shifting the respective pixel row information to the right with the result shown in FIG. 2. An inspection of FIG. 2 reveals that the word "York" now is comprised of strokes which to the eye are substantially vertical, and which according to the invention are now more readily segmentable into individual letters. In example 2, illustrated in FIGS. 4, 5 and 6, the same steps as described with respect to example 1, are practiced. The choice of the boundary angles in this example, is from -52 degrees to +30 degrees. For simplicity, the positive-going data was cut off at -20 degrees in FIG. 6. The two examples illustrate the fact that boundary angle end-range values will vary. Obviously, a range should be chosen that is sufficiently wide to be reasonably certain of including the angle at which the entropy is a minimum. The data suggests that a range of, say, from -10 to +10 degrees will miss a lot. At the same time, the range not be so wide as to needlessly lengthen the run time of the computation. From the preceding, it will be apparent to persons skilled in the art that calculating the total pixel count is not essential. What determines the shape of the functions illustrated in FIGS. 3 and 6 is strictly the total number of dark pixels in each column. To rigorously follow the logic of Equation 1, however, the total dark pixel count was used.

Claims: In the machine-reading of handwritten script, a method for normalizing the characteristic slant angle in the script, CHARACTERIZED BY the steps of: generating a pixel image of the original handwriting; measuring the number of dark pixels falling in each pixel column, and the total number of dark pixels in the image; calculating the ratio of dark pixels in each column to total dark pixels in said image; calculating the entropy values of the said columns of said original image and storing said values; successively shifting the individual rows of pixels in the total image in incremental amounts to vary the slant angle from that exhibited in the writer's characteristic script, and calculating and storing the successive entropy values; determining the minimum value of the stored entropy values; measuring the extent of pixel shift between the pixel positions of said original image and the pixel positions of the image that resulted in the minimum entropy column value; converting the last-named measure to an angular value; and shifting the pixels rows of the image which yielded the minimum entropy column value by the amount of said angular value. The method in accordance with claim 1, wherein said successive shifting comprises moving each individual pixel row an amount linearly proportional to its distance from a selected stationary pixel reference row. The method in accordance with claim 2, wherein said successive shifting comprises keeping the reference pixel row stationary, shifting the pixel row furthest from the reference row sideways in incremental steps substantially equal to the length of a pixel side, and shifting the intermediary rows by amounts proportional to their respective distances from the reference row. The method of claims 1 or 2, FURTHER CHARACTERIZED BY making said incremental pixel row step shifts within a substantial lateral range both in a right and left direction with respect to the pixel configuration of said original image. The method of any one of the preceding claims characterized by the additional step of segmenting the image into discrete letters or digits. Apparatus for normalizing the characteristic slant angle in handwritten script CHARACTERIZED BY: means (10) for generating a pixel image of the original script; means (11) for measuring the number of dark pixels falling in each pixel column, and the total number of dark pixels in the image; means (11) for calculating the ratio of dark pixels in each column to total dark pixels in said image; means (11) for calculating the entropy values of said columns of said original image, and storing said values; means for successively shifting the individual rows of pixels in the total image in incremental amounts to vary the slant angle from that found in the handwritten script; means (11) for calculating and storing the respective successive entropy values; means for determining the minimum entropy value of the plurality of stored entropy values; means (11) for measuring the extent of pixel shift between the pixel positions of said original image and the pixel positions of the image that resulted in the minimum entropy column value; means (11) for converting the last-named measure to an angular value; and means for shifting the pixel rows of the image having the minimum entropy column value by the amount of said angular value. Apparatus in accordance with claim 6, wherein said generating means is a video camera (10). Apparatus in accordance with claim 7, wherein said successive shifting means comprises means for moving each individual pixel row an amount linearly proportional to its distance from a selected stationary pixel reference row. Apparatus as claimed in claim 8 wherein said successive shifting means comprises means for shifting the pixel row furthest from the reference row sideways in incremental steps substantially equal to the length of a pixel side and for shifting intermediary rows by amounts proportional to their respective distances from the reference row. Apparatus as claimed in any one of claims 6 to 8 wherein said successive shifting means comprises means for shifting the individual rows of pixels both to the left and to the right.

Inventors: KIANG RICHARD KIASHING, KIANG, RICHARD KIASHING
Applicants: NCR INT INC, NCR INTERNATIONAL INC.
Application year: 1991
Main classification: G06K9
IPC codes: G06K9