Image processing apparatus and method

Abstract: An image containing characters and lines, which is inputted by a scanner or the like, is binarized and stored in a memory. A thickening process is repeatedly applied on groups of significant bits stored in the memory, until a predetermined condition is fulfilled. After this process, the thickened image is thinned at least a greater number of times than the number of times the thickening process is repeated in order to eliminate lines. As a result, characters are determined to exist at areas where groups composed of remaining significant dots exist, then characters are separated from the input image based on the areas of groups composed of remaining significant dots.

Description: BACKGROUND OF THE INVENTION The present invention relates to an image processing apparatus and method and, more particularly, to an image processing apparatus and method capable of separating characters and lines in an image. Assume that an original image having characters and lines, such as maps and drawings, read by a scanner or the like is processed, for instance, edited and/or searched for characters, by a computer. Conventionally, the original image is binarized, labeled, and thinned, and so on, and numbers and length of groups of the labeled pixel are determined, then labeled pixel groups having relatively large numbers of pixels (having long length) are recognized as lines, and labeled pixel groups of relatively short lengths are recognized as characters. Thus both lines and characters are extracted at the same time. In the conventional method as described above, extraction as well as recognition of characters and lines are performed in parallel. Therefore, there is a problem in that it takes quite a long preparation time for the process. SUMMARY OF THE INVENTION The present invention has been made in consideration of the above situation, and has as a concern to provide an image processing apparatus and method which enable separation of characters and lines efficiently when an image having lines and characters is inputted. US Patent Specification No US-A-5202933 discloses a method and apparatus for differentiating and extracting text and line graphics in an image. The method described uses morphological operations in the horizontal and vertical directions to coalesce group pixels so that groups of character pixels can be separated from lines. According to the present invention there is provided an image processing apparatus as set out in claim 1, and an image processing method as set out in claim 12. Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principle of the apparatus and method according to the invention. Fig. 1 is a block diagram of an image processing apparatus according to a first embodiment of the invention; Fig. 2 is a flowchart explaining a process according to the first embodiment of the invention; Fig. 3 shows transition of an image in a thickening process according to the first embodiment of the invention; Fig. 4 shows transition of the image in a thinning process according to the first embodiment of the invention; Fig. 5 is a flowchart explaining a process according to a second embodiment of the invention; Fig. 6 is a flowchart explaining a process according to a third embodiment of the invention; Fig. 7 shows an example of a table for setting a number of times a thickening process should be repeated; and Fig. 8 is a block diagram illustrating a configuration of an image processing apparatus according to another embodiment of the invention. DETAILED DESCRIPTION OF THE EMBODIMENTS [First Embodiment] Preferred embodiments of the present invention will be described in detail in accordance with the accompanying drawings. Fig. 1 illustrates a construction of an image processing apparatus according to a first embodiment, and Fig. 2 shows a processing sequence performed by CPU 3 in Fig. 1, the program which controls the process is stored in a program memory 6 in Fig. 1. In Fig. 1, reference numeral 1 denotes a scanner as an image input device for reading an original image, reference numeral 2 denotes RAM for temporarily storing various kinds of data, such as image data, which is transmitted from the scanner 1, and reference numeral 3 denotes CPU for controlling the entire operation of the apparatus, where the CPU 3 controls a process shown in Fig. 2 based on a control program stored in the program memory 6. Further, reference numeral 4 denotes an address bus for designating an address to access data stored in the RAM 2; 5, a data bus for inputting and outputting the data of the address designated by the address bus 4 and for transmitting data from the scanner 1 to the RAM 2 or the CPU 3; 7, VRAM for storing data to be displayed on a display 8. The operation of the CPU 3 according to the first embodiment will be described with reference to Fig. 2. First, at step S1, image data is read by the scanner 1, and stored in the RAM 2. Next, at step S2, the image data is binarized. At step S3, thickening process is applied on an arbitrary pixel f ij of the image data binarized at step S2, according to the following function (1), where "1" indicates a significant dot, namely a print dot. This process makes widths of lines thicker as well as combining character strings into lumps. Transition of lines and characters in this process is shown in Fig. 3. In this embodiment, lines are assumed to have width of one dot, length of 75 dots, and distances between lines are 4 dots. After the thickening process is applied once, the width of the lines becomes 3 dots (gains 2 dots), and a total number of significant dots added to the three lines is 468 = ((75 + 2) x 2 + (1 x 2))x3 After the thickening process is repeated, a number of newly added significant dots to the already thickened lines is 492 = ((75 + 4)x 2 + (3 x 2))x3. Note that after performing the thickening process twice, the three lines become one lump as shown in the bottom of Fig. 3. Further, if another thickening process is applied, then a number of newly added significant dots is 192 = (75 + 6) x 2 + (15 x 2 ) . Thus the number of newly added significant dots decreases compare to the two previous thickening process. Therefore, the thickening process is terminated two times before when the number of dots added during the current thickening process is less than the number of dots added during the previous thickening process. After repeating the thickening process the required number of times, the obtained image data is outputted at step S3. Note that in a case where there is only one line existing, since a number of newly added significant dots never decreases after each thickening process, the process should be repeated a predetermined number of times. At step S4, a thinning process is performed on a arbitrary dot, f ij , in the image data outputted at step S3, based on the following equation (2). where if f ij before the thinning process is "1" and all the eight neighboring pixels are "1", then f ij remains "1". By performing this operation, widths of lines can be thinned. Transition of the image is shown in Fig. 4. Contrary to the process at step S3, the number of deleted significant dots is counted at step S4. After repeating the thinning process three times, only the significant dots of lumped characters are left, thus, if the process is repeated one more time, smaller number of significant dots are deleted comparing to the number of deleted significant dots after the previous process. Thus, the thinning process is repeated until the number of newly erased significant dots decreases sharply. As a result of above, by letting the numbers of times to repeat the thickening process and the thinning process be m and n, respectively, the lines totally disappear when m < n. Note that n is decided based on m, for example, m = n + α (α = 1, 2,...). In addition, an area of joined significant dots which represent a character string after the thickening and thinning processes is smaller than a corresponding character string area in the original image. Thus, the thickening process is repeated (n - m) times at step S5. Next, a lump of significant dots is extracted from the image data outputted at step S5, then a rectangular area surrounding the lump is determined at step S6. More specifically, logical "AND" operation between the image data obtained after the thickening process at step S5 and the image binarized soon after inputted is performed. At step S7, a rectangular area which corresponds to the determined rectangular area is extracted from the original image data, then characters are extracted from the extracted rectangular area. In order to separate characters in the extracted area, first, numbers of print dots are counted in the vertical and horizontal directions, and histograms are made based on the counted numbers. Then, areas having local minimum numbers in the histograms are determined as spaces between characters, and areas having global and local maximums are determined as characters. It should be noted that, in the first embodiment, the input original image data is stored in the RAM 2 first, then binarized. However, the original image can be binarized during input by comparing image data from the scanner 1 with a predetermined threshold value, so that the RAM 2 stores the binarized image. Further, according to the aforesaid example, lumps of significant dots which representing characters are left by repeating the thickening process and thinning process. In order to complete the process according to the first embodiment, there is required additional memory space in the RAM 2 for storing unprocessed original image data, or the original image have to be read again. Thus, the original image can be temporarily stored in a low priced hard disk, or the like, then after necessary information on the character area in the RAM 2 is extracted by performing the aforesaid processes, the original image can be written in the RAM 2 for separating characters and recognizing them. This alternative method to store the original image in the hard disk can be applied in the embodiments. According to the first embodiment as described above, character areas can be extracted in high speed from original image data including both characters and lines in mixture by performing simple processes. Accordingly, by using a character recognizing apparatus to which the aforesaid configuration is employed, a preparation time before starting extracting character images is shortened, therefore character recognition is performed in high speed from an operator's point of view. [Second Embodiment] A second embodiment is described with reference to a flowchart in Fig. 5. Note that a program based on the flowchart in Fig. 5 is stored in the program memory 6 in Fig. 1 in order to perform a process according to the second embodiment,. First, at step S11, image data is read by the scanner 1, and stored in the RAM 2. Then, at step S12, the input image data is binarized. At step S13, a thickening process is applied to an arbitrary dot, f ij , of the binarized image data outputted at step S12 based on the following equation (3) (binarized data of a dot position, g ij , is determined). The thickening process is performed in only the horizontal direction, thus widths of vertical lines becomes thicker than the vertical lines in the original image data as well as characters stretching in the horizontal direction can be joined. The number of times to repeat the thickening process is determined in the same manner as described in the first embodiment. Next at step S14, a thinning process is applied to an arbitrary dot, f ij , of the image data outputted at step S13 based on the following equation (4). where if f ij before the thinning process is "1" and both the right and left pixels are "1", then f ij remains "1". The number of times to repeat the thinning process is determined in the same manner as described in the first embodiment. If the numbers of times to repeat the thickening process based on the equation (3), and the thinning process based on the equation (4), are m1 and n1, respectively, when m1 < n1, vertical lines disappear. Size of joined significant dots representing a character string in the image after the thinning process is smaller than an area representing characters in the original image data. Therefore, the thickening process based on the equation (3) is applied to the processed image (n1 - m1) times, at step 15. Then, at step S16, a thickening process is performed on an arbitrary dot, f ij , of the image data outputted at step S15, based on the following equation (5). Note that the number of times to repeat the thickening process is determined in the same manner as at step S13. The thickening process is performed in the vertical direction, and widths of horizontal lines becomes thicker than the horizontal lines in the original image data as well as characters stretching in the vertical direction can be joined. Next at step S17, a thinning process is applied to an arbitrary dot, f ij , of the image data outputted at step S16 based on the following equation (6). where if f ij before the thinning process is "1" and both the above and below pixel are "1", then f ij remains "1". The number of times to repeat the thinning process is determined in the same manner as described at step S14. If the numbers of times to repeat the thickening process based on the equation (5) and the thinning process based on the equation (6), are m2 and n2, respectively, when m2 < n2, horizontal lines disappear. Size of jointed significant dots representing characters in the image data after the thickening and thinning processes is smaller than an area representing characters in the original image data. Therefore, the thickening process based on the equation (5) is applied to the processed image (n2 - m2) times, at step 18. Next, at step S19, joined significant dots in the image data outputted at step S18 are extracted, and a rectangular area surrounding the jointed dots is determined at step S19. At step S20, a rectangular area which corresponds to the detected rectangular area is extracted from the original image data, then characters are extracted from the extracted rectangular area. A method of separating the characters is the same as the one described in the first embodiment. According to the second embodiment as described above, by performing the thickening and thinning processes in the vertical and horizontal directions, areas representing characters are extracted faster comparing to the first embodiment, since the addition and elimination of significant dots are performed in the vertical and horizontal directions only. [Third Embodiment] A third embodiment will be explained with reference to Fig. 6. In the embodiment, a program which controls a following procedure is also stored in the program memory 6 as in the second embodiment. In the third embodiment, a rectangular area including characters is extracted from the original image at steps S21 to S29 in the same method as described in the second embodiment (steps S21 to S29 in Fig. 6 correspond to steps S11 to S19 in Fig. 5, respectively). Next, at step S30, an identical and unique number is assigned to each significant pixels jointed to next significant pixel inside of the rectangular area (labeling). At step S31, characters are extracted by considering the relative positions of each labeled group of pixels. For example, in order to label a character image, "aj", "1" is assigned to pixels which construct "a", "2" is assigned to a pixel constructing the point of "j" (top part of "j"), and "3" is assigned to pixels constructing a lower part of "j" excluding the point. Then, rectangles surrounding each labeled pixel group and centers of each labeled pixel group are determined. As a result, the pixels making the labeled group 2 and the labeled group 3 are determined to be of the same character, because the distance between the centers of the labeled groups 2 and 3 is much smaller than the distance between the centers of the labeled group 1 and of the labeled group 2 or 3. Likewise, the pixels making the labeled group 1 are considered as being of the same character since the centers of the labeled group 1 and that of the labeled groups 2 and 3 are wide apart. Thus, each character is extracted. More specifically, a threshold value is set, and pixels of a labeled group is determined that they compose a single character if distances between centers of this labeled group and of other labeled groups are larger than the threshold. Note that in order to extract rectangular areas including characters from original image data, a process shown at steps S1 to S6 in the first embodiment can be utilized. Binarization is performed by considering to significant pixels, however, in a case where an original image, such as a map, includes colors, characters can be extracted by performing the same procedure as described above by considering to one of the colors in the original image. As an example, by further providing a control panel on the image processing apparatus in the embodiment, a color of interest is designated by using key input to the control panel. Then the original image data read from the scanner 1 is binarized by assigning "1" to the designated color data and assigning "0" to other colors. The binarized data is stored in the RAM 2. As a result, characters and lines of only the designated color are stored in the RAM 2. Therefore, by performing the same process as described in the above embodiments, only characters can be extracted from the image data stored in the RAM 2, thus colored characters in the original image, such as a map, can be recognized. Further, a map, for instance, may be printed with a plurality of colors, thus characters of different colors, such as blue characters and red characters, may be found. Therefore, it is more desirable to designate a plurality of colors at color designation. Then, image data of the designated colors is assigned as "1" and other colors are assigned as "0" to binarize the original image data which is scanned by the scanner 1, then store the binarized image data in the RAM 2. As a result, it is possible to separate characters of plurality of colors simultaneously. In addition, the image data stored in the RAM 2 is only characters and lines of the designated colors since the image data of non-designated colors is assigned as "0". Accordingly, the thickening process and the thinning process are simplified, thus the process is performed at high speed. According to the third embodiment as described above, the processing time can be reduced by erasing lines, extracting areas representing characters, then extracting characters. Further, the image processing apparatus is constructed with a single device in the above embodiments, however, the present invention can be applied to a system having an image reading unit and a processor, independently, for performing an aforesaid process. For instance, the system can be constructed with an image scanner and a personal computer. Furthermore, the present invention can be enabled by providing an external program to the apparatus or the system. Note that lines in the embodiments are one dot width, however, a number of dots representing a width of a line changes depending upon resolution of the scanner, the thickness of line, the font size, and so on. Thus the number of dots representing the width of a line can be designated by an operator when needed. By changing the number of dots of a line width, the number of times to repeat a thickening process and a thinning process alters in accordance with the designated number of dots. Further, Fig. 3 shows an example that three lines are placed close to each other. In a case where only one line exist, the timing when the thickening process should stop is not known, since a number of newly added dots after each thickening process keeps increasing. In order to overcome this problem, in a case where the number of newly added dots does not decrease after performing a thickening process predetermined times, the thickening process is set to stop. Therefore, it is desirable to construct the apparatus so as to set a maximum number of times to repeat the thickening process. Other reasons for setting the maximum number of times to repeat the thickening process are that timings when a character becomes a lump of significant dots and/or when characters join and form a lump, for example, differ depending on reading resolution, font size of an original image, and so on. As for reading resolution, an operator does not have to set it each time once the resolution is stored in an apparatus. Whereas, regarding sizes of characters and widths of lines, they should be designated when necessary from a control panel provided on the apparatus. The size of characters and width of lines can be chosen from three different sizes or widths, for instance. As another method to set the maximum number of times to repeat a thickening and thinning processes, a table composed of reading resolution, designated widths of lines, and maximum numbers of times to repeat a thickening process depending on font sizes, as seen in Fig. 7, is stored in the program memory 6, then the maximum number of times to repeat a thickening process is obtained by the CPU 3 referring the table. Further, as shown in Fig. 8, a apparatus can be connected to a device, such as a keyboard 9, capable of inputting information, so that the thickness of characters and widths of lines can be inputted. Furthermore, in a case where color data needs to be processed, color or colors can be designated by using the keyboard 9. Further, instead of using keyboard, above described conditions to be selected can be displayed on the display 8, and selected by pointing to the proper icons, for instance, by using a pointing device, such as a mouse. Further, an original image is directly inputted from the scanner 1 in the embodiments, however, an image containing characters and lines can be read from a floppy disk, or the like, or can be received via a communication line. According to the apparatus and method of the present invention as described above, when an image containing characters and lines is inputted, characters can be efficiently separated from the image. The apparatus and method of the present invention can be applied to a system constituted by a plurality of devices, or to an apparatus comprising a single device. Furthermore, the apparatus and method of the invention is applicable also to a case where the object of the invention is attained by supplying a program to a system or apparatus. The present invention is not limited to the above embodiments and various changes and modifications can be made within the scope of the present invention as set out in the appended claims.

Claims: Image processing apparatus adapted to receive image data representing an image containing characters and lines and to separate the characters from the lines, the apparatus comprising processing means (3) for thickening groups of significant dots in the image by the addition of further dots and for thinning groups of dots so thickened; and determining means (3) for determining areas representing characters from the thickened and thinned groups of dots, and characterised in that said processing means are adapted to repeatedly execute a thickening operation until the number of dots added at a n-th thickening operation is less than the number of dots added at the (n-1)th thickening operation, where "n" is the number of repetitions of the thickening operation, and to obtain an image at the (n-m)th thickening operation, wherein "m" is an integer greater than "1", said processing means being further adapted to thin the thickened image so obtained by said processing means by repeatedly removing dots for at least a number of times greater than n-m, the determining means being adapted to determine that an area of significant dots remaining after the thinning process is a character region area. Apparatus according to claim 1, wherein the processing means are adapted to thicken the image which has been previously thickened and thinned on the basis of the number of times the thickening process and the thinning process had been repeated; and extracting means for extracting characters from the input image based on the area of significant dots obtained after this second thickening process. Apparatus according to claim 1 or claim 2, wherein the thickening process at each thickening step is carried out in accordance with the following function for each significant pixel f ij Apparatus according to claim 3, wherein the thickening process at each thinning step is carried out in accordance with the following function Apparatus according to any one of claims 2, 3 or 4, wherein said extracting means comprises logic operating means for performing a logical operation between the final thickened image and the input image. Apparatus according to claim 5, further comprising recognition means for recognizing characters extracted by said extracting means. Apparatus according to any preceding claim, wherein the processing means operate the thickening and thinning processes in the vertical and horizontal directions of the image. Apparatus according to any preceding claim, wherein the processing means is adapted to repeat the first mentioned thickening process only a predetermined number of times if the number of dots added does not decrease. Apparatus according to claim 8, wherein said predetermined value can be changed. Apparatus according to any preceding claim, further comprising reading means (1) for reading an original image to generate the image data. Apparatus according to claim 10, further comprising setting means for setting a color or colors of the input image to be read by said reading means. An image processing method comprising: receiving image data representing an image containing characters and lines and separating the characters from the lines, the method comprising the steps of thickening groups of significant dots in the image by the addition of further dots and thinning groups of dots so thickened; determining areas representing characters from the thicked and thinned groups of dots; said method being characterised by the steps of repeatedly executing the thinning generation until the number of dots added at an (n)-th thickening operation is less than the number of dots added at the (n-1)th thickening operation, where "n" is the number of repetitions of the thickening operation; obtaining an image at the (n-m)th thickening operation, wherein "m" is a integer greater than 1; thinning the thickened image so obtained by repeatedly removing dots for at least a number of times greater than n-m; and determining that an area of significant dots remaining after the thinning process is a character region area. A method according to claim 12, including thickening the image which has been previously thickened and thinned on the basis of the number of times the thickening process and the thinning process had been repeated; and extracting characters from the input image based on the area of significant dots obtained after this second thickening process. A method according to claim 12 or claim 13, wherein the thickening process at each thickening step is carried out in accordance with the following function for each significant pixel f ij A method according to claim 14, wherein the thickening process at each thinning step is carried out in accordance with the following function A method according to any one of claims 12 to 15, wherein said extracting means comprises logic operating means for performing a logical operation between the final thickened image and the input image. A method according to any one of claims 12 to 16, further comprising recognizing characters extracted by said extracting means. A method according to any one of claims 12 to 17, wherein including carrying out the thickening and thinning processes in the vertical and horizontal directions of the image. A method according to any one of claims 12 to 18, including repeating the first mentioned thickening process only a predetermined number of times if the number of dots added does not decrease. A method according to any one of claims 12 to 19, further comprising reading an original image to generate the image data. A method according to claim 20, further comprising setting means for setting a color or colors of the input image to be read. A storage medium storing processor implementable instructions for controlling a processor to carry out the method of any one of claims 12 to 21. An electrical signal carrying processor implementable instructions for controlling a processor to carry out the method of any one of claims 12 to 21.

Inventors: HARADA TAKASHI, MATSUBAYASHI KAZUHIRO, MORI SHIGEKI, NAGASAKI KATSUHIKO, SHIMADA KAZUTOSHI, SUNAKAWA SHINICHI, TATSUMI EISAKU, HARADA, TAKASHI, MATSUBAYASHI, KAZUHIRO, MORI, SHIGEKI, NAGASAKI, KATSUHIKO, SHIMADA, KAZUTOSHI, SUNAKAWA, SHINICHI, TATSUMI, EISAKU, HARADA, TAKASHI, C/O CANON KABUSHIKI KAISHA, MATSUBAYASHI, KAZUHIRO, C/O CANON KABUSHIKI KAISHA, MORI, SHIGEKI, C/O CANON KABUSHIKI KAISHA, NAGASAKI, KATSUHIKO, C/O CANON KABUSHIKI KAISHA, SHIMADA, KAZUTOSHI, C/O CANON KABUSHIKI KAISHA, SUNAKAWA, SHINICHI, C/O CANON KABUSHIKI KAISHA, TATSUMI, EISAKU, C/O CANON KABUSHIKI KAISHA
Applicants: CANON KK, CANON KABUSHIKI KAISHA
Application year: 1994
Main classification: G06K9
IPC codes: H04N1, G06K9