Method and apparatus for determining the frequency of words in a document without document image decoding

Abstract: A method and apparatus for determining word frequency from a document without first converting the document to character codes. The method includes morphological image processing to determine word unit characteristics for placement into equivalence classes utilizing non-content based information. Word shape representations are preferably determined and compared to define equivalent word units.

Description: The invention relates to improvements in methods and apparatuses for document image processing, and more particularly to improvements for recognizing and determining the frequency of words or images in a document without first decoding the words or images or referring to an external lexical reference. In computer based electronic document processing, an attribute of the document(s) being processed which the operator often desires to know is the frequency with which some or all of the words occur. For example, Salton & McGill, Introduction to Modern Information Retrieval , Chapter 2, pp. 30, 36, McGraw-Hill, Inc., 1983, indicates that in information retrieval contexts, the frequency of use of a given term may correlate with the importance of that term relative to the information content of the document. Word frequency information can thus be useful for automatic document summarization and/or annotation. Word frequency information can also be used in locating, indexing, filing, sorting, or retrieving documents. Another use for knowledge of word frequency is in text editing. For example, one text processing device has been proposed for preventing the frequent use of the same words in a text by categorizing and displaying frequently occurring words of the document. A list of selected words and the number of occurrences of each word is formulated for a given text location in a portion of the text, and the designated word and its location is displayed on a CRT. Heretofore, though, such word frequency determinations have been performed on electronic texts in which the contents have been converted to a machine readable form, such as by decoding using some form of optical character recognition (OCR) in which bit mapped word unit images, or in some cases a number of characters within the word unit images, are deciphered and converted to coded representations of the images based on reference to an external character library. The decoded words or character strings are then compared with dictionary terms in an associated lexicon. Disadvantages of such optical character recognition techniques are that the intermediate optical character recognition step introduces a greater possibility of computational error and requires substantial time for processing, slowing the overall word unit identification process. European Patent Application No. 0-402-064 to Sakai et al. describes a text processing device in a computer system for counting the occurrence of words in a text and displaying a list of repetitive words on a CRT. The list includes the selected words together with their number of occurrences and their locations in the text. In a case where word repetition is undesirable, an operator may substitute synonyms or otherwise alter the text by using search, display, and editing actions. European Patent Application EP-A-0-364-179 to Hawley describes a method and apparatus for extracting key words from text stored in a machine-readable format. The frequency of occurrence of each word in a file, as compared to the frequency of occurrence of other words in the file, is calculated. If the calculated frequency exceeds by a predetermined threshold the frequency of occurrence of that same word in a reference domain appropriate to the file, then the word is selected as a key word for that file. European Patent Application EP-A-0-364-180 to Hawley describes a method and apparatus for automatically indexing and retrieving files in a large computer file system. Key words are automatically extracted from files to be indexed and used as the entries in an index file. Each file having one of the index entries as a key word is associated in the index with that key word. If a file is to be retrieved, and its content, but not its name or location, is known, its key words are entered and its identifying information will be displayed (along with that of other files having that key word), facilitating its retrieval. FIRST INTERNATIONAL CONFERENCE ON DOCUMENT ANALYSIS AND RECOGNITION ICDAR 91, 30 September-2 October 1991, San Malo, FR, pages 963-971; D.S.BLOOMBERG: "Multiresolution morphological approach to document image analysis" discloses morphological operations for performing document image analysis. For example, techniques are disclosed for identifying italic and bold words in text using word-level extraction at reduced resolution. PROCEEDINGS 7TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION, Vol. 2, 30 July-2 August 1984, Montreal, CA, pages 1256-1258; R.G. CASEY et al.: "Unsupervised construction of decision networks for pattern classification" discloses a classifier for encoding documents using pattern matching. Classification of character patterns by pattern matching is disclosed, after using a decision network as preclassifier to select a prototype class. According to one aspect of the invention, there is provided a method for determining the frequency of word image units in an electronic document comprising one or more images, characterised by (A) segmenting the one or more images into word image units without decoding image content; (B) determining at least one shape characteristic for each of a set of the word image units in the one or more images without decoding word image units; (C) identifying equivalence classes of the set of word image units by clustering word image units with similar shape characteristics; and (D) determining the number of word image units in each equivalence class. According to another aspect of the invention, there is provided an apparatus for determining the frequency of word image units in an electronic document comprising one or more images, characterised by segmentation means for segmenting the one or more images into word image units without decoding image content; determination means for determining at least one shape characteristic for each of a set of the word image units in the one or more images without decoding word image units; identification means for identifying equivalence classes of the set of word image units by clustering word image units with similar shape characteristics; and determining means for determining the number of word image units in each equivalence class. One object of the invention is to provide a method and apparatus for determining the frequency of occurrence of word image units in a document based solely on the visual characteristics of the scanned document, and without reliance on an external lexical reference. Other objects of the invention include striving to provide a method and apparatus of the type described without any requirement that the words or word image units themselves be determined or decoded, without first converting the document to optical character or ASCII codes, and that can be used to assist in key word recognition. In accordance with one example of the invention, a method and apparatus are provided for determining word image unit frequency in a document without first decoding the word image units in the document, or converting the document to optical character codes. The invention utilizes non-content image unit recognition, based on image shape properties of the image unit, such as length, height or other characteristics. Also, the invention is not limited to systems utilizing document scanning. Rather, other systems such as bitmap workstations (i.e., a workstation with a bitmap display) or a system using both bitmapping or scanning would work equally well for the implementation of the methods and apparatus described herein. In accordance with an embodiment of the method of the invention, the document is first input and segmented into image units. At least one significant image shape characteristic of the image unit is determined, and equivalence classes of the image units are identified into which image units having similar image shape characteristics are clustered. The number of image units in an equivalence class determines the frequency of occurrence of the image unit. The image units may be word units in a textual document, and a word unit is preferably evaluated by deriving a word shape representation of the word unit, which is either at least one, one-dimensional signal characterizing the shape of the word unit; or an image function defining a boundary enclosing the word unit, which image function has been augmented so that an edge function representing edges of the character string detected within the boundary is defined over its entire domain by a single independent variable within the closed boundary, without individually detecting and/or identifying the character or characters making up the word unit. More particularly, a method and apparatus are provided for determining the frequency of word image units in a document directly from the stored bit mapped image of the document, without decoding the word image units, such as by converting the word image units in the document image to character code representations, such as ASCII or other coded text. The technique, therefore, is essentially language independent, and, in fact, graphic patterns, coded and nonsense words, can easily be included and processed, and the possible introduction of unnecessary errors due to intermediate interpretation processes such as optical character recognition (OCR) can be eliminated. The method also can take advantage of the naturally segmentable nature of the word unit images used throughout printed text. The equivalence classes are determined by comparing selected image shape characteristics or combinations of characteristics, or the derived representations of the image unit shapes, with each other. The image shape characteristics can include image unit length, width, font, typeface, cross-sectional characteristics, number of ascenders, number of descenders, or the like. The image units in each equivalence class are linked together, and mapped to enable the frequency of each to be determined. The method may comprise performing data driven processing in a data processing system which comprises execution processing means for performing functions by executing program instructions in a predetermined manner and memory means containing a plurality of processing program modules is presented. The method includes identifying word units in the text images, and determining the least one image shape characteristic of the word units. The word units with similar image shape characteristics are then clustered, and the clustered word units are quantified. The apparatus can be a programmed digital computer. The invention will be described further, by way of example, with reference to the accompanying drawings, in which: Figure 1 shows a flow chart of a method according to a preferred embodiment of the invention for determining word image unit frequencies in a document without first converting the document to character codes, and Figure 2 shows an apparatus according to a preferred embodiment of the invention for determining word image unit frequencies in a document without first decoding the word image units or converting the word image units in the document to character codes. A preferred embodiment of the method of the invention is illustrated in the flow chart of Figure 1, and apparatus for performing the method of Figure 1, is shown in Figure 2. For the sake of clarity, the invention will be described with reference to the processing of a single document. However, it will be appreciated that the invention is applicable to the processing of a corpus of documents containing a plurality of documents. With reference first to Figure 2, the method is performed on an electronic image of an original document 5, which may include lines of text 7, titles, drawings, figures 8, or the like, contained in one or more sheets or pages of paper 10 or other tangible form. The electronic document image to be processed is created by any conventional manner, for example, by input means, such as an optical scanner 12 and sensor 13 as shown, a copier machine scanner, a Braille reading machine scanner, a bitmap workstation, an electronic beam scanner or the like. Such input means are well known in the art, and thus are not described in detail herein. An output derived from, for example, a scanner sensor 13 is digitized to produce bit mapped image data representing the document image for each page of the document, which data is stored, for example, in a memory 15 of a special or general purpose digital computer 16. The digital computer 16 can be of the type that performs data driven processing in a data processing system which comprises execution processing means for performing functions by executing program instructions in a predetermined manner, such computers now being well known in the art. The output from the computer 16 is delivered to an output device, such as, for example, a memory or other form of storage unit, or an output display 17 as illustrated, which may be, for instance, a photocopier, CRT display, printer, facsimile machine, or the like. With reference now to Figure 1, the first phase of the image processing technique of the invention involves a low level document image analysis in which the document image for each page is segmented into undecoded information containing image units (step 20) using conventional image analysis techniques; or, in the case of text documents, using a bounding box method. Another method for finding word boxes is to close the image with a horizontal structural element (SE) that joins characters but not words, followed by an operation that labels the bounding boxes of the connected image components (which in this case are words). The process can be greatly accelerated by using one or more threshold reductions (with threshold value 1), that have the effect both of reducing the image and of closing the spacing between the characters. The threshold reduction(s) are typically followed by a closing with a small horizontal SE. The connected component labeling operation is also done at the reduced scale, and the results are scaled up to full size. The disadvantage of operating at reduced scale is that the word bounding boxes are only approximate; however, for many applications the accuracy is sufficient. The described method works fairly well for arbitrary text fonts, but in extreme cases, such as large fixed width fonts that have large inter-character separation or small variable width fonts that have small inter-word separation, mistakes can occur. The most robust method chooses a SE for closing based on a measurement of specific image characteristics. This requires adding the following two steps: (1) Order the image components in the original or reduced (but not closed) image in line order, left to right and top to bottom. (2) Build a histogram of the horizontal inter- component spacing. This histogram should naturally divide into the small inter-character spacing and the larger inter-word spacings. Then use the valley between these peaks to determine the size of SE to use for closing the image to merge characters but not join words. After the bounding boxes or word boxes have been determined, locations of and spatial relationships between the image units on a page are determined (step 25). For example, an English language document image can be segmented into word image units based on the relative difference in spacing between characters within a word and the spacing between words. Sentence and paragraph boundaries can be similarly ascertained. Additional region segmentation image analysis can be performed to generate a physical document structure description that divides page images into labelled regions corresponding to auxiliary document elements like figures, tables, footnotes and the like. Figure regions can be distinguished from text regions based on the relative lack of image units arranged in a line within the region, for example. Using this segmentation, knowledge of how the documents being processed are arranged (e.g., left-to- right, top-to-bottom), and, optionally, other inputted information such as document style, a "reading order" sequence for word images can also be generated. The term "image unit" is thus used herein to denote an identifiable segment of an image such as a number, character, glyph, symbol, word, phrase or other unit that can be reliably extracted. Advantageously, for purposes of document review and evaluation, the document image is segmented into sets of signs, symbols or other elements, such as words, which together form a single unit of understanding. Such single units of understanding are generally characterized in an image as being separated by a spacing greater than that which separates the elements forming a unit, or by some predetermined graphical emphasis, such as, for example, a surrounding box image or other graphical separator, which distinguishes one or more image units from other image units in the document image. Such image units representing single units of understanding will be referred to hereinafter as "word units." Advantageously, a discrimination step 30 is next performed to identify the image units which have insufficient information content to be useful in evaluating the subject matter content of the document being processed. One preferred method for detecting function or stop words is a morphological method. In this method, first a length and height of each image unit in the image is determined, and short image units are identified as image units of no more than a predetermined number of characters, preferably three characters or less in length. Next, image units which are not short image units are deleted from the image. Then, the image is blurred or smeared in a horizontal direction, although the image units are not smeared together. This can be accomplished for example by CLOSING the image with a horizontal structuring element having, e.g., a length of 5 pixels. The length of the horizontal structuring element used to blur the x-height characters in the image is dependent upon the width of the character type being used. Furthermore, other configurations of structuring elements may be used in the CLOSING operation to obtain the same smearing effect. However, the most efficient and effective way to smear characters of x-height is to use a horizontal structuring element as described above. A UNION of erosions is then taken of the image using a set of ascender matching structuring elements and a set of descender matching structuring elements. The UNION provides optional noise elimination filtering, and the UNION will provide a seed from which to fill short image unit masks in a subsequent seed filling operation such as a fillClip operation. The UNION acts on all image units remaining in the image (i.e., only short image units in this case) and since the UNION of erosions was taken using a set of ascender matching structuring elements and a set of descender matching structuring elements, the image units that will be filled are those containing ascender and/or descender characters, i.e., function words. The function words are then identified as those image units which are filled short image unit masks. Next, in step 40, selected image units, e.g., the image units not discriminated in step 30, are evaluated, without decoding the image units being classified or reference to decoded image data, based on an evaluation of predetermined image shape characteristics of the image units. The evaluation entails a determination (step 41) of the image shape characteristics and a comparison (step 42) of the determined image shape characteristics for each image unit with the determined image shape characteristics of the other image units. In one preferred method, at least one, one-dimensional signal characterizing the shape of a word unit is derived; or an image function is derived defining a boundary enclosing the word unit, and the image function is augmented so that an edge function representing edges of the character string detected within the boundary is defined over its entire domain by a single independent variable within the closed boundary, without individually detecting and/or identifying the character or characters making up the word unit. The derived image unit shape representations of each selected image unit are compared, as noted above (step 41), with the derived image unit shape representations of the other selected image units for the purpose of identifying equivalence classes of image units (step 50), such that each equivalence class contains most or all of the instances of a given word in the document. The equivalence classes are thus formed by clustering the image units in the document based on the similarity of image unit classifiers, without actually decoding the contents of the image units, such as by conversion of the word images to character codes or other higher-level interpretation. Any of a number of different methods of comparison can be used. One technique that can be used, for example, is by correlating the raster images of the extracted image units using decision networks, such technique being described for characters in a Research Report entitled "Unsupervised Construction of Decision networks for Pattern Classification" by Casey et al., IBM Research Report, 1984, herein incorporated in its entirety. Depending on the particular application, and the relative importance of processing speed versus accuracy, for example, comparisons of different degrees of precision can be performed. For example, useful comparisons can be based on length, width or some other measurement dimension of the image unit (or derived image unit shape representation e.g., the largest figure in a document image); font, typeface, cross-section (a cross-section being a sequence of pixels of similar state in an image unit); the number of ascenders; the number of descenders; the average pixel density; the length of a top line contour, including peaks and troughs; the length of a base contour, including peaks and troughs; and combinations of such classifiers. In instances in which multiple page documents are processed, each page is processed and the data held in the memory 15 (see Figure 1), as described above. The entirety of the data can then be processed. One way in which the image units can be conveniently compared and classified into equivalence classes is by comparing each image unit or image unit shape representation when it is formed with previously processed image units/shape representations, and if a match is obtained, the associated image unit is identified with the matching equivalence class. This can be done, for example, by providing a signal indicating a match and incrementing a counter or a register associated with the matching equivalence class. If the present image unit does not match with any previously processed image unit, then a new equivalence class is created for the present image unit. Alternatively, as shown (step 50) the image units in each equivalence class can be linked together, and mapped to an equivalence class label that is determined for each equivalence class. The number of entries for each equivalence class can then be merely counted. Thus, after the entire document image, or a portion of interest, has been processed, a number of equivalence classes will have been identified, each having an associated number indicating the number of times a image unit was identified having similar shape characteristics, or classifiers, thus determining the image unit frequency. A salient feature provided by the technique of the invention is the processing, identification, comparison, or manipulation of image units without an accompanying requirement that the content of the image units be decoded, even for output. More particularly, image units are determined, processed and delivered for output without decoding, so that in essence, the actual content of the image units is never required to be determined. Thus, for example, in such applications as copier machines or electronic printers that can print or reproduce images directly from one document to another without regard to ASCII or other encoding/decoding requirements, image units can be identified, and processed using one or more morphological characteristic or property of the image unit. In the comparison process described, for instance, each image unit, of undetermined content, in the area of the document image of interest is compared with other image units in the document also of undetermined content. Selected image units, still of undetermined content, can then be optically or electronically delivered for output, for example, to an image reproducing apparatus of a copier machine, an electronic memory, a visual display, or the like, for example in producing a list of significant "words", or image units in order of frequency of appearance in the document image. The technique described above can be used to determine the significance of the image units of a document, based upon the criterion of frequency of occurrence of a particular image unit. Thus, for example, the number of times an image unit appears in its respective equivalence class can be used to construct a hierarchy of words, such hierarchy being useful for many purposes, such as for example, generating document summaries and annotations. It is noted, however, that the classifiers are determined without actually decoding the content of the image unit; only the selected classifiers of the image unit itself are used. The method can be applied, of course, to documents of multiple page length in a similar manner to that described above. Although the invention has been described and illustrated with a certain degree of particularity, it is understood that the present disclosure has been made only by way of example, and that numerous changes in the combination and arrangement of parts can be resorted to by those skilled in the art without departing from the scope of the invention, as hereinafter claimed.

Claims: A method for determining the frequency of word image units in an electronic document comprising one or more images, characterised by (A) segmenting the one or more images into word image units without decoding image content; (B) determining at least one shape characteristic for each of a set of the word image units in the one or more images without decoding word image units; (C) identifying equivalence classes of the set of word image units by clustering word image units with similar shape characteristics; and (D) determining the number of word image units in each equivalence class. A method as claimed in claim 1 in which (C) comprises correlating word image unit shape characteristics using a decision network. A method as claimed in claim 1 or claim 2 in which (C) comprises comparing word image unit shape characteristics. A method as claimed in any preceding claim in which (A) comprises deriving a boundary box enclosing a word image unit without individually detecting and/or identifying the character or characters making up the word image unit. A method as claimed in any preceding claim in which (D) comprises steps of: linking word image units in each equivalence class together; and counting the number of linked word image units for each equivalence class. A method as claimed in claim 5 in which said step of linking word image units together comprises: determining an equivalence class label for each word image unit; and mapping each word image unit to the determined equivalence class label. A method as claimed in any preceding claim in which said shape characteristic includes at least one of an image unit dimension, number of ascender elements, number of descender elements, and a contour characteristic of said word image unit. A method as claimed in any preceding claim in which (A), (B), (C), and (D) are performed by operating a programmed digital computer. An apparatus for determining the frequency of word image units in an electronic document comprising one or more images, characterised by segmentation means (16, 20) for segmenting the one or more images into word image units without decoding image content; determination means (16, 41) for determining at least one shape characteristic for each of a set of the word image units in the one or more images without decoding word image units; identification means (16, 50) for identifying equivalence classes of the set of word image units by clustering word image units with similar shape characteristics; and determining means (16) for determining the number of word image units in each equivalence class. An apparatus as claimed in claim 9 in which the determining means (16) determines a frequency for each equivalence class, the apparatus further comprising an output device (17) for producing an output in response to the frequencies of the equivalence classes.

Inventors: CASS TODD A, HALVORSEN PER-KRISTIAN, HUTTENLOCHER DANIEL P, KAPLAN RONALD M, RAO RAMANA B, WITHGOTT M MARGARET, CASS, TODD A., HALVORSEN, PER-KRISTIAN, HUTTENLOCHER, DANIEL P., KAPLAN, RONALD M., RAO, RAMANA B., WITHGOTT, M. MARGARET
Applicants: XEROX CORP, XEROX CORPORATION
Application year: 1992
Main classification: G06K9
IPC codes: G06T1, G06K9, G06T11, G06F17