Image processing method and apparatus

Abstract: An image recognition method and apparatus in which a character region is segmented from input image information and the segmented character regions are recognized. Segmenting and recognition of character regions are performed in parallel with the input of image information. In response to to an interruption in input of image information, the recognition process is halted until the segmenting process is completed. Thereafter, recognition processing may be performed on any desired ones of the segmented character regions.

Description: This invention relates to an image processing method and apparatus for segmenting and recognizing character information of an input image. A conventional image processing apparatus which can perform character recognition processing is configured as shown in FIG. 12. In FIG. 12, A scanner unit 41 converts a character image of a subject to be read into an analog electric signal. A preprocessing unit 42 performs binary-coding processing of analog signals and removal of noise. A character segmenting unit 43 separates respective characters in a character string. A characteristic extraction unit 44 extracts plural characteristics peculiar to a single character according to a predetermined algorithm and generates characteristic vectors. A recognition dictionary 45 stores statistics (average, variance and the like) of characteristic vectors for each character. A collation unit 46 collates characteristic vectors obtained from an input character image with the recognition dictionary and selects the most suitable candidate. A word dictionary 47 stores results of translated words. A dictionary retrieval unit 48 refers a recognized character string to the word dictionary and takes out the corresponding translated word. A display unit 49 displays the translated word. Processing of the above-described recognition unit is performed according to the following two methods. In the first method, segmenting and recognition processing is started after determining that the entirety of an input image has been stored in an image buffer. In the second method, segmenting and recognition processing is started for each portions of the images as soon as the portion is stored in an image buffer. In the first method, since segmenting and recognition processing is started after the entirety of an input image has been stored in an image buffer, processing time required for the entire process becomes long. In the second method, recognition processing is performed for the entire input character string. However, if it is desired to recognize and translate only a certain target word, processing for portions of the character string other than the target word becomes unnecessary processing. In general, in an apparatus which inputs a character string while translating it, words immediately before and after the word to be translated are also read. This is because image reading and processing occur in parallel. Suppose, for example, that in trying to translate a word "great", words immediately before and after the word are read and a character string "a great many" is stored in the image buffer. In such a case, although recognition processing is necessary only for the word "great", segmenting and recognition processing is performed for the entire character string "a great many", since the processing is sequentially performed for respective words. While segmenting processing must be performed for all the words, recognition processing is originally needed only for the word "great". As a result, additional time is wasted for recognition processing for the words "a" and "many". It is known from JP55078379 to provide an apparatus in which input characters are segmented and recognised character by character. It is also known from JP1041069 to provide such apparatus as a portable dictionary for translating a word selected by an operator from a displayed character string after recognition. According to the present invention there is disclosed an image processing apparatus for recognizing a word in a line of text comprising: input means operable to scan a part of a line of text around a word to be recognized; segmenting means operable to segment individual character images from the input image; word position detecting means operable to determine the presence of individual words within a string of character images segmented from said segmenting means on the basis of the presence of empty space in the part of the line that was scanned, and further operable to select one of the words thus determined according to predetermined criteria and to provide position information for said selected word; recognition means using said position information for said selected word to limit character recognition to the string of character images output forming said selected word; and first controlling means operable to control operation of said segmenting means and said recognition means such that a sequence of segmentation of one character image followed by recognition of said one character image is repeated for subsequent individual character images; detection means (4) operable to detect an interruption of said input of the image; and second controlling means operable to interrupt, in response to said detection of the interruption, said first controlling means and to thereafter control operation of said segmenting means and said recognition means such that segmentation is repeated without performing recognition until a word is detected by said word position detecting means. Recognition of character information segmented from input image information is thereby sequentially performed while the image information is input, and segmenting processing of characters is preferentially performed when the input of the image information is interrupted, whereby characters other than those belonging to the detected word need not be recognized, and processing time is shortened. In response to an interruption in inputting image information, recognition is interrupted and instead segmentation is performed. Recognition remains interrupted at least until when segmentation is completed. Thereafter, recognition may be performed on desired ones of the segmented character regions, Alternatively, recognition remains interrupted at least until when a signal to activate recognition is input. Thereafter, recognition may be performed on desired ones of the segmented character regions. Objectives and advantages in addition to those discussed above shall be apparent to those skilled in the art from the description of the preferred embodiment of the invention which follows. In the description, reference is made to the accompanying drawings, which form a part hereof, and which illustrates an example of the invention. Such example, however, is not exhaustive of the various embodiments of the invention, and therefore reference is made to the appended claims for determining the scope of the invention. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram showing the configuration of an embodiment of the present invention; FIGS. 2A, 2B and 2C are views showing an external appearance of an information processing apparatus according to the embodiment shown in FIG. 1; FIG. 3 is a schematic diagram illustrating a method of storing data in an image buffer; FIG. 4 is a flowchart showing the entire processing; FIG. 5 is a flowchart showing character recognition processing; FIG. 6 is a flowchart of recognition processing RECOGX; FIG. 7 is a flowchart of segmenting processing CLIP_ONE; FIG. 8 is a flowchart of recognition processing RECOG_ONE for each character; FIG. 9 is a flowchart of word extracting processing GET_WORD; FIG. 10 is a diagram illustrating character segmenting processing by contour-line tracking; FIG. 11 shows an example of extraction of a word; and FIG. 12 is an example of the configuration of a conventional recognition-type electronic dictionary. DESCRIPTION OF THE PREFERRED EMBODIMENT FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention. FIG. 1 shows the configuration of a recognition-type electronic dictionary which performs character recognition by manually reading an English document, and displaying a Japanese word corresponding to an English word by performing dictionary retrieval using the result of the character recognition. In FIG. 1, a scanner 1 reads image information of an object to be read while converting the information into an analog signal. A binary-coding circuit 2 converts the analog signal output from the scanner 1 into a binary digital signal. An encoder unit 3 rotates in synchronization with a reading operation by manual scanning, detects the rotation by a photo-interrupter, and outputs a signal synchronized with the manual scanning. A reed switch 4 monitors a pushing state of a roller for providing the encoder 3 with rotation, and detects the end of a reading operation. A central processing unit (CPU) 10 comprises a large-scale integrated circuit (LSI) including an interruption input port, an interruption control circuit, a clock pulse generator, an operation decoder, registers, an arithmetic logic unit (ALU), an input port, an output port, and an image input shift register. A random access memory (RAM) 11 includes readable/writable storage units allocated for respective addresses. The storage unit has a memory function of storing data, a flag function of storing the result of a determination, a counting function of storing a state as a count value, a register function of temporarily storing data, and the like. A read-only memory (ROM) 12 stores microprograms to be sequentially executed by the CPU 10, a recognition dictionary, an English-Japanese dictionary, and encoded constants used in various kinds of determinations. A liquid-crystal display driver (LCD driver) 13 receives output data and generates a signal for displaying the data on a liquid-crystal display (LCD) 14. An external bus line 16 includes an external-address bus and an external-data bus. Addressing of the ROM 12 and the RAM 11, exchange of data, and the like are performed via the external bus line 16. FIGS. 2A, 2B and 2C are views showing the external appearance of an information processing apparatus of the present embodiment. In FIG. 2A, there are shown an upper case 21 and a lower case 22. FIG. 2B shows input keys 23, a liquid-crystal display unit 24, a roller 25 for transmitting rotation to the encoder, a reading unit 26, and a recess 28 for a thum for increasing holdability.. FIG. 2C shows a marker 27 for reading. First, an explanation will be provided of an image buffer arranged in the RAM with reference to FIG. 3. The scanner used in the present embodiment comprises a one-dimensional line sensor in which image data are received in units of a line. In the schematic diagram of the image buffer shown in FIG. 3, reference numeral 31 schematically indicates the image buffer arranged in the RAM, where the x direction and the y direction are termed a column and a row, respectively. Received image data for one column are sequentially stored from the left to the right starting with the 0-th column. Since image data are already binary-coded, "1" and "0" on the image buffer correspond to a black pixel (picture element) and a white pixel on the original image, respectively. FIG. 4 is a flowchart showing the entire processing of the present embodiment. In step S401, character recognition is performed. In step S402, dictionary retrieval is performed using the obtained character string. In step S403, the result of translation is displayed. The character recognition in step S401 will be explained according to a flowchart shown in FIG. 5. In step S501, both a counter UCNT representing to which column in the image buffer data have been stored and a flag READEND representing the state of the scanner are set to 0 as the initial values, and interruption of the encoder is authorized. Step S502 is a subroutine RECOGX for performing character recognition processing. Three kinds of interruptions will occur during this processing. First, when a reading operation is manually started, the roller rotates to generate encoder interruption. If the encoder interruption occurs, an LED (light-emitting diode) is turned on in step S503, and a timer is set in step S504. When a time for storing data for one column in the scanner has lapsed after the timer was set, timer interruption is generated, and the data for one column is transferred from the scanner to the image buffer. In step S506, the value of the UCNT is incremented. In step S507, the LED is turned off in order to save power consumption. When the reading operation has been terminated, read-off interruption is generated by halting the pushing of the roller. This interrption sets the READEND to "1" in step S508. In step S509, interruption is prohibited. In the present invention, character segmenting is performed by contour-line tracking. That is, as shown in FIG. 10, pixels are checked from a1 of the image buffer in the rightward direction, and contour-line tracking is started from a point where a black pixel is first hit. A circumscribing rectangle for one character can be determined from the minimum values and the maximum values in the x and y directions of a contour line obtained by making one round. For the second character, pixels are checked from position a2 situated at the right end of the circumscribing rectangle of an immediately preceding character in the x direction and at the center of the rectangle in the y direction in the rightward direction and the same processing is continued. Finally, the coordinates at the rightmost end reached during the contour-line tracking are set in a counter PCOL. The initial value of the PCOL is 0. Next, processing in the RECOGX will be explained according to a flowchart shown in FIG. 6. If the value of the PCOL is not smaller than the value of the UCNT in step S601, data sufficient for character segmenting processing are not transferred to the image buffer. Hence, segmenting processing cannot be performed, and the process proceeds to step S605. If the READEND = 1 (that is, the reading operation has been terminated in step S605), the process proceeds to step S606. If the READEND does not equal 1 in step S605, the process proceeds to R1. In step S606, a subroutine GET_WORD is evoked for taking out a word to be translated from the read character string. In the GET_WORD subroutine, the first position and the last position of the word to be translated are calculated for all characters C(i) (i = 1, 2, ..., N) and set these positions to STC and ENDC, respectively. In step S607, N1 is set in a counter i for loop control. In a subroutine RECOG_ONE, a flag indicating a recognized character is raised every time a character is recognized. In step S608, the flag is checked. If an i-th character C(i) has already been recognized, the process proceeds to step S610, and the next character is checked. If the i-th character is not yet recognized, the subroutine RECOG_ONE is evoked to perform recognition for this character. Referring again to step S601, if the value of the PCOL is smaller than the value of the UCNT, meaning that there is sufficient data for character segmenting in the image buffer, the process proceeds to step S602, where a subroutine CLIP_ONE is evoked for performng contour-line tracking for one character. If the READEND = 1, that is, if the reading operation has been terminated after the end of the CLIP_ONE, in step S603, the process proceeds to R2. If the READEND is not 1 in step S603, the RECOG_ONE is evoked for performing recognition for one character in step S604. Next, an explanation will be provided for processing of the subroutine CLIP_ONE for performing segmenting processing with reference to the flowchart shown in FIG. 7. In step S701, the initial value is set in a counter (x, y) representing a target pixel. The initial value of x is the value of the PCOL, and the initial value of y is the average of the coordinate y 1 of the upper end of the immediately preceding character and the coordinate y 2 of the lower end of the character. For the first character, the central position between the upper and lower ends of the image buffer is used for the value of y. In step S702, it is determined whether or not the coordinates (x, y) represent a black pixel. If the result of the determination is affirmative, the process proceeds to step S703, where contour-line tracking is started. The contour-line tracking is terminated if the tracking returns to the original position after making one round (success of segmenting processing), or PCOL ≧ UCNT (stoppage of segmenting processing). If the segmenting processing has succeeded, the process returns from step S704. In step S705, the state of the READEND is checked. If the READEND is 1, the process returns to the start point. If the READEND is 0, the process returns to step S703, where the contour-line tracking is continued. If the result of the determination in step S702 is negative, the process proceeds to step S706, where the value of x is incremented. In step S707, the value x is compared with the value of the UCNT. If the value x is smaller than the value of the UCNT, the process returns to step S702, where the next pixel is checked. If the value x is not smaller than the value of the UCNT, the process proceeds to step S708. If the READEND is 1 in step S708, the process returns to the start point. If the READEND is not 1, the process returns to step S707, where it is awaited until the value of the UCNT is incremented. FIG. 8 is a flowchart of the subroutine RECOG_ONE for performing recognition for one character. In step S801, characteristics are extracted according to a predetermined algorithm, and characteristic vectors are obtained. In order to prevent variation in characteristics due to the size of the character, the characteristic vectors are normalized. Subsequently, in step S803, the distance to the input vector is calculated using the average and variance of the character vectors for each character which have been calculated previously. In step S804, the distances obtained for respective characters are compared with one another, and a character which has a minimum distance is selected as a candidate for the recognized character. In step S805, a flag indicating that recognition processing has been terminated for this character is raised. Next, an explanation will be provided for word selection processing in the subroutine GET WORD with reference to the flowchart shown in FIG. 9. FIG. 11 illustrates a segmented character string. As a result of character segmenting processing by contour-line tracking, circumscribing rectangles Ci (i = 1, 2, ..., N) and intervals bi (i = 1, 2, ..., N-1) (hereinafter termed blanks) between adjacent characters are obtained. First, in step S901, the average b of blanks is calculated. At that time, calculation is performed with removing the maximum value bmax1 and the second largest value bmax2 of the bi's. b = ( i-1 N bi - bmax1 - bmax2)/(N - 2), where it is assumed that b = min(b1, b2) when N =2. Subsequently, in step S902, a threshold value bt for determining an interval (hereinafter termed a space) between adjacent words is calculated according to the following expression: bt = α · b + β , where α and β are predetemined threshold values. In step S903, the values of bi's are sequentially checked, and a blank in which bi > bt is detemined as a space. If the number NS of spaces is 0 in step S904, the process proceeds to step S908. If the number NS is not 0, the process proceeds to step S905. In step S908, the value 1 is set to a counter STC representing the leading position of the word to be translated, and sets the number N of the total segmented characters in a counter ENDC representing the last position of the word. When a space is present, if the position of the first space is next to the k-th character, the value k is set in a counter SP representing the position of the space. In step S905, the length L 1 of the character string immediately preceding the SP is compared with the length L 2 of the character string immediately after the Sp (see FIG. 11). If the L 1 is longer than the L 2 , SP + 1 and N are set in the STC and the ENDC, respectively, in step S906. If the L 1 is not longer than the L 2 , 1 and SP are set in the STC and the ENDC, respectively, in step S907. Although, in the above-described embodiment, an English sentence is read and translated into Japanese, any other language may, of course, be read, provided that spaces are present between words. The present invention does not depend on a recognition algorithm and an algorithm for extracting a word, but can, of course, be applied to any other methods. Although, in the above-described embodiment, read-off interruption (that is, the end of a reading operation) is detected opun the release of a pushing state of the roller, a read button may be mounted on the main body of the apparatus, a read key may be depressed in a reading operation, and read-off interruption may be generated when the read key is released. The present invention may, of course, be applied to a system comprising a plurality of apparatuses, an apparatus comprising only one unit, or a case in which a system or an apparatus is supplied with a program.

Claims: An image processing apparatus for recognizing a word in a line of text comprising: input means (1; 26) operable to scan a part of a line of text around a word to be recognized; segmenting means (S602, S606) operable to segment individual character images (C(i)) from the input image; word position detecting means operable to determine the presence of individual words within a string of character images segmented from said segmenting means on the basis of the presence of empty space in the part of the line that was scanned, and further operable to select one of the words thus determined according to predetermined criteria and to provide position information for said selected word; recognition means using said position information for said selected word to limit character recognition to the string of character images output forming said selected word; and first controlling means (S601, S602, S604) operable to control operation of said segmenting means and said recognition means such that a sequence of segmentation of one character image followed by recognition of said one character image is repeated for subsequent individual character images; detection means (4) operable to detect an interruption of said input of the image; and second controlling means (S603) operable to interrupt, in response to said detection of the interruption, said first controlling means and to thereafter control operation of said segmenting means (S602) and said recognition means (S604) such that segmentation is repeated without performing recognition until a word is detected by said word position detecting means (S606). Apparatus as claimed in claim 1 wherein the second controlling means is operable to control operation of said recognition means such that, after a word has been detected by said word position detecting means, operation of said recognition means (S609) is resumed to perform recognition of characters in said word. Apparatus as claimed in claim 2 comprising translation means (12, S402) operable after completion of recognition of characters in said detected word to translate said detected word into a translated word of another language. Apparatus as claimed in claim 3 further comprising display means (14, S403) operable to display the translated word. Apparatus as claimed in any preceding claim wherein the input means comprises a scanner (1). Apparatus as claimed in claim 5 wherein said scanner comprises a roller (25) which rotates during scanning and wherein said detection means is operable to monitor rotation of said roller and to detect an interruption of said input of the image when said rotation ends. Apparatus as claimed in any preceding claim wherein said apparatus is integrally formed as a hand-holdable device. A method of image processing for recognizing a word in a line of text comprising the steps of inputting an image to an image processing apparatus by scanning a part of a line of text around a word to be recognized; segmenting (S602, S606) individual character images (C(i)) from the input image; determining the presence of individual words within the string of character images segmented in said segmenting step on the basis of the presence of empty space in the part of the line that was scanned, selecting one of the words thus determined according to predetermined criteria and providing position information for selected word; performing recognition processing (S602, S606) using said position information for said selected word to limit character recognition to the string of character images output forming said selected word; and controlling said segmenting processing and said recognition processing such that a sequence of segmentation of one character image followed by recognition of said one character image is repeated for subsequent individual character images; detecting (S603) an interruption of said input of the image; and, in response to said detection of interruption, controlling said segmenting processing and recognition processing such that segmentation is repeated (S606) without performing recognition processing until a word is detected (S606). A method as claimed in claim 8 including the step of controlling the recognition processing such that, after a word has been detected, the recognition processing (S609) is resumed to perform recognition processing of characters in said word. A method as claimed in claim 9 including the step of translating (S402) said detected word after completion of recognition processing of characters in said detected word into a translated word of another language. A method as claimed in claim 10 including the step of displaying (S403) the translated word. A method as claimed in any of claims 8 to 11 wherein the step of inputting the image is effected by operation of a scanner (1). A method as claimed in claim 12 wherein said scanner comprises a roller which rotates during scanning and wherein said detection step comprises monitoring rotation of the roller to detect an interruption of said input of the image when said rotation ends. A method as claimed in any of claims 8 to 13 wherein said apparatus is integrally formed as a hand-holdable device and wherein said apparatus is hand-held in use.

Inventors: NIKI TORU, NIKI, TORU, NIKI, TORU, C/O CANON KABUSHIKI KAISHA
Applicants: CANON KK, CANON KABUSHIKI KAISHA
Application year: 1992
Main classification: G06K9
IPC codes: G06K9, G06F17