Certifiable optical character recognition

Abstract: A certificate (27), calculated on the text of a document (24) to be reproduced, is included in the document in a form which is machine-readable. Upon optical character recognition scanning (at 16), or other digitizing reproduction, the certificate (27) is also scanned and stored for comparison. A new certificate is calculated on the contents of the reproduced document (25) and compared to the scanned certificate (27). If the two certificates are different, it is known that at least one error exists in the reproduced document (25). More than one certificate can be provided for a document in order to facilitate error location on the document and error correction therein.

Description: This invention relates to the use of automatically generated document markers. More particularly, it relates to the use of markers of encoded information incorporated into each page of a document for providing a means for not only document identification and document structure recognition, but also error detection and error correction when the marked documents are reproduced using optical character recognition technology. The identification of products using computer readable bar codes, wherein digital data is recorded directly on paper, provides for item identification given a fixed set of values using simple numeric encoding and scanning technologies. Identification of computer generated and stored documents is another technology which has been developed using binary encoding to identify and provide for retrieval of stored documents. Most document-generating software programs provide not only identification and/or retrieval information for the document. but also include encoded information for provision to an associated printer specifying, for example, such details as spacing, margins and related layout information. Once the document has been printed on paper, however, that information no longer accompanies the document, other than as discerned by the user. If it is desired to reproduce the document using an optical character recognition (OCR) system, there is no automatic means by which to communicate the layout information through the scanner and to the receiving computer. A desirable extension of the identification technology would be, therefore, the provision of a means for generating a paper version of a document which can be recognized, reproduced and proof-read by a computer by optically scanning a marker incorporated in or on the paper document in conjunction with the OCR text scanning of the document. Document or product identification systems which have been employed in the past include bar code markers and scanners which have found use in a wide range of arenas. With respect to paper documents, special marks or patterns in the paper have been used to provide information to a related piece of equipment, for example the job control sheet for image processing as taught by Hikawa in U.S. Patent No. 5,051,779. Similarly, identifying marks have been incorporated into forms as described in U.S. Patent No. 5,060,980 of Johnson, et al. The Johnson, et al system provides for the editing of forms which are already resident in the computer. A paper copy of the form is edited by the user and then scanned to provide insertions to the fields of the duplicate form that is stored electronically in the computer. Still another recently patented system is described in U. S. Patent 5,091,966 of Bloomberg, et al, which teaches the decoding of glyph shape codes, which codes are digitally encoded data on paper. The identifying codes can be read by the computer and thereby facilitate computer handling of the document, such as identifying, retrieving and transmitting the document. The systems described in the art do not incorporate text error detection or correction schemes. Further, the systems require that the associated computer have a copy of the document of interest in its memory prior to the input of information via the scanning. The systems cannot be applied to documents which are being created in the scanning computer by OCR. Optical character recognition systems, as illustrated schematically in Figure 1, generally include a digitizing scanner, 16, and associated "scanning" computer, 18, for scanning a printed page, 14, which was generated by an originating computer, 12, and output by a printer, 13. The scanner, 16, extracts the text to be saved, as electronic document 15, in a standard electronic format, such as ASCII. What is desirable is to additionally incorporate information about the text for error detection and about the layout thereof, which information can be optically scanned or otherwise automatically input. Due to the inherent limitations in both the scanning process and the ability of an optical character recognition system to effect accurate character recognition, errors are introduced into the output, including not only character misinterpretation errors but also layout-dependent errors. Post-processing, specifically error detection, must then be performed, primarily by human proof-reading of the reproduced document. Errors in layout are ordinarily not automatically rectifiable by the computer; but, rather, require extensive, user-intensive editing or possibly re-creation of the document. The human post-processing is expensive not only in terms of actual costs but also in the time needed to complete the processed document. Optimally, solutions will provide not only a means for detecting errors but also a means for correcting the errors. Further, an ideal solution should facilitate identification of the document and define the appropriate layout structure for the document. Error detection systems which have been employed in the computer document creation technology (e.g., word processing) include techniques based on dictionary lookup and/or attempts to use semantic, or context, information extracted from the document in order to identify and correct errors. Many of these systems require that entries in the document which do not correlate to an entry in the lexicon will be reviewed by a "human post-processor". The automated error correction version of a dictionary-based system will, upon identification, spontaneously correct entries which do not correlate to dictionary entries. One can readily envision a circumstance wherein automatic correction is not desirable, such as in the case of a proper name, an intentional misspelling or a newly coined term. The presumption in the use of dictionary-comparison versions of such systems is that each entry in the entire document be compared to a data-base dictionary of terms. The cost of comparison of each entry of a document to a given lexicon is quite high. The use of semantic information extracted from the document is further proposed in the art in order to facilitate the identification and automatic correction of errors which cannot be readily identified as misspellings of available dictionary terms or which "resemble" more than one available dictionary entry. Such a system will recognize and correct the term "ofthe" to "of the" when a dictionary lookup would simply reject the term. Similarly, a bank of commonly-occurring errors for the hardware or software being used, and for the font or fonts being scanned, has been proposed for use with the context, or semantic, information in order to identify and automatically correct common errors, such as "rn" being incorrectly identified as "m", or the letter "1" being incorrectly identified as the number "1". To detect errors without requiring an entry-by-entry lookup, particularly for documents which are transmitted over extended networks, systems have made use of parity bits transmitted with the data. Once the transmission has been effected, a bit count is done on the "new" document. If the calculated bit matches the transmitted parity bit, then an error-free transmission is assumed. Such systems, and extensions of the parity and check bit concept, as taught in U.S. Patent No. 3,068,854 of Chandran, et al, are useful for detecting errors in digitally encoded information. Further extensions of the parity bit concept, such as balanced weight error correcting codes, to detect and provide correction of more than a one-bit error are also found in the art, such as in U.S. Patent No. 4,965,883 of Kirby. Parity and check bit systems developed for use with binary coded information are capable of ascertaining the presence of errors with reasonable accuracy given the low probability of the error bit of an erroneously-received quantity of data matching the check bit of the transmitted material. Since the bits are calculated on binary-encoded data, they are most effective for detecting one-bit errors; except as modified in the weighted balancing and random checking instances. Generally speaking, however, the check and parity bit systems tend to be data-independent methods for assuring error-free transmission of computer-to-computer transfers. The check and parity bit systems are not, therefore, considered thorough checking systems but merely first screening techniques which are limited to digital-to-digital communications and not applicable to analog-to-digital conversions such as optical character recognition. A further prior art system, providing a 16-bit check sequence which is data-dependent and calculated on the contents of the data field, is found in U.S. Patent No. 4,964,127 of Calvignac, et al. Once again, the system is applied to data which is transmitted along a data path, presumably in digital format. In the field of optical character recognition (OCR), there is a similar need to provide the means for detecting and correcting errors in data which has been reproduced from optical scanning, bit mapping and computer encoding. Both dictionary lookup and common-error reference have been proposed for use in the OCR context. However, as with the document creation needs of the past, the entry-by-entry checking is both costly and inefficient. Moreover, in addition to the printed words, the document layout is a critical feature in OCR. The use of current parity bit check systems in an optically-scanned, bit-mapped system is only nominally effective for error detection, relatively ineffective for error location and totally ineffective for detection and correction of improper layout. Apparatus for identifying and correcting "unrecognizable" characters in OCR machines is taught in U.S. Patent No. 4,974,260 of Rudak. In that system, the characters which are not recognized, in the electronic dictionary lookup operation, are selectively displayed for an operator to effect interpretation and correction. More fully automated OCR error detection and correction is desirable, but not currently available. There is further known from US-A-3,200,372, a method according to the preamble of Claim 1. This method, however relates to the correction of errors in machine readable media, such as punched cards. The method is capable of correcting only a single error per line of text and only a certain class of error, that is to say an uncertainty error. It is accordingly, an object of the invention to provide an improved error correction method that is also able to correct a larger class of error and/or more than one error per line of text. This object is achieved by the features of the characterising clause of Claim 1. The invention will now be described in greater detail with reference to the accompanying drawings wherein: Figure 1 schematically illustrates the prior art OCR method of scanning and reproducing a document. Figure 2 schematically illustrates the OCR method for reproducing a document with markers having certificates to provide error detection. Figure 3 illustrates a document generated in accordance with the present invention. Figure 4 illustrates a complex document containing diagrams, text blocks and photographs. Figures 5A through 5F illustrate one scheme for encoding the layout of the complex document illustrated in Figure 4. In accordance with the present invention, markers are created for paper documents which may contain data-dependent error detection information, hereinafter referred to as "certificates", and document layout information for communication to a "scanning" computer and use by the scanning computer upon reproduction of the document using OCR technology. When using the certificate component of a marker to assist in the computer generation of an original document, the certificate generator 21 of the originating computer, 22 as illustrated in Figure 2, calculates one or many data-dependent certificates, 27, with an appropriate algorithm, an example of which is detailed below. A certificate is a succinct key encoding information about the contents of the page, produced at print time, and recognizable to the OCR software, 29, associated with the scanning computer, 28. Any document generated on a computer can have a certificate appended to or associated with each text block or page. The process of generating the certificate requires no human intervention, and only a small added computational cost. As illustrated in Figure 3, the document 34, as generated as a printed page or in another medium, is comprised of an area 35, formatted primarily for human use, and an area 37 formatted for machine use to assist the machine in its "understanding" of the so-called "human" area. The human area is the analog portion of the document and the machine area is the digital portion of the document. The distinction is used to designate the use made of the portions of the document rather than the specific embodiments. The two portions can, and preferably would be, printed using the same technology. As an example, the "digital" portion can be printed using a special font, bar code or other symbology which may or may not be "readable" to the human user, but which is chosen to facilitate computer readability. The certificate is intended to provide information to the OCR software so that it becomes possible to produce a perfectly transcribed digital copy of the original printed page. The certificate that is computed and printed on the page contains information about the contents of that page. The originating computer, 22 of Figure 2, includes certifying software, 21, referred to as the certificate generator. Once the document has been created, or in the process of the creation thereof, the certifying software calculates one or more certificates based upon the information in and on the document. The generated certificate, 27, is produced as a machine readable part of the hardcopy, 24, of the document which is output by the printer, 13. As in the prior art, the hard copy to be reproduced is scanned using a digitizing scanner, 16, which is associated with a second computer, 28, equipped with the OCR software. The original certificate, 27, is scanned and saved by the scanning computer. After the OCR document 25 has been created, the OCR software uses the same algorithm, as that used by the original printing software, to calculate a certificate for the extracted text. The illustration provides the "certificate verifier", 26, as a separate part of the computer 28 and the "certificate generator", 21, as part of computer 22. As would be evident to one having skill in the art, the certificate generator or certificate verifier is incorporated into the overall system and need not be a separate program. In addition, the calculating of certificates for original or scanned data is not necessarily a separate process step, but may be conducted concurrently with the creating and/or saving of the data. Once calculated, the new certificate is compared to the original certificate scanned from the printed page. If the two match, the translation is deemed complete and presumed to be error free with high probability. If there is a mismatch, the OCR software can detect and correct a small number of errors, given its own or one or more of the known error correction schemes used manually by current document creation or OCR systems, as discussed above. Upon completion of the error correction, the certificate verifier then recalculates a certificate for the corrected text and again compares it to the original certificate to establish when the output is correct. The certificate can be computed in any of a number of ways. For example, the "C" subroutine shown below computes a simple check-sum on a line-by-line basis as follows: The ASCII value of each non-space character is exclusive-or'd with a running 8-bit check-sum. This check-sum is then bit-rotated one position to the left, and the process is repeated with the next character in sequence. In this case, the line "This is a test." would receive the certificate "03" (expressed in hexadecimal notation), which would be printed on the page in question. If, in the process of scanning, the OCR software misread the line as "Thus is a test.", the Calculated certificate would be "73". Hence, the OCR software would detect its error by comparing the two certificates (one newly computed on the reproduced text and one originally computed, printed and read from the printed certificate) and determining that they do not match. in using this sample system, the probability that two random lines of text would hash to the same certificate is 1 in 256. The eight-bit check-sum is only one example of a certificate computation system. Those skilled in the art will readily recognize alternative computation methods which fall within the scope of the present invention. one having skill will additionally recognize that the size of the check-sum and the computation method can be varied to increase or decrease the probability of error detection as required by the specific application. As discussed in the Background section, the error correction methods which are available for incorporation into an OCR system include dictionary lookup search strategies, semantic or context information codes and common error recognition codes, among others. Further to that end, the precise mathematical nature of the certificate generation algorithm lends itself to precise identification of the error and ready correction thereof without a need to invoke auxiliary error correction programs. Certificates can improve OCR recognition rates and provide a reliable method by which users can ascertain whether or not each scanned page is error free. As noted above, use of an error detection and correction system without knowing if intentional "errors" exist in a document can actually cause errors to be introduced into the text. When using a certificate system of error detection and correction. however, this can be avoided. In the instance of an intentional misspelling, for example, the certificate system would not indicate that an error had been made, and would not therefore erroneously correct the intentional misspelling. One important consideration, particularly in the case of documents with complicated structures, is determining the canonical parsing order for computing the certificate. Obviously, the software that calculates the original certificate and the OCR software must both use the same order. One linearization formula may follow a left-to-right, top-to-bottom order in the same way that English text is normally read by humans. Another approach would be to decompose the page as a series of text blocks, each a separate entity in the calculation. Any blocks containing graphics or other non-text information must be handled differently than standard text. In the case of diagrams, recognition that a collection of "dots" corresponds to a perfect circle is a difficult task for image processing software. If, however, the certificate generator encodes the information that a given diagram contains 3 circles and a triangle, this information may greatly speed processing time and increase accuracy. Beyond encoding the existence of the diagram components, the precise locations and sizes of the basic geometric elements in the diagram could be encoded (e.g., circle radius 0.3 cm; x-coordinate 1.3 cm, y-coordinate 3.8 cm, etc.). It is further possible to adapt a certificate scheme to recognize mathematical equations or other special typeset structures. It is also desirable to incorporate the document structure information in a document marker. A 15.2 x 17.8 cm (6" x 7") document having a complicated layout structure is illustrated in Figure 4. The document, 44, contains text blocks A, B, D, E and G at 45, a photograph in block C at 48, and a diagram in block F at 46. In order to identify the document layout to the scanning system, one layout identification system which can be utilized is based upon a plane-slicing model, as is illustrated in Figures 5A through 5F. Other models can be utilized as appropriate. The plane-slicing model example presumes that the layout of a document, no matter how complicated, can be described by some number of cuts. The plane slicing can be represented recursively as a binary tree, provided the canonical ordering for the subplanes, represented by the leaves, has been defined. The slices or planes are first identified, and characterized as specific horizontal and vertical components, H and V, for example. Each slice is identified as a part of a tree structure. This tree structure is then encoded as a linear string. A recursive syntax is used to yield, for the illustrated document, the following: (H1''CRT A (V2''(H3''CRT B (H5''PHT C CRT D ))(V4''CRT E (H3''FIG F CRT G ))); wherein each precise location is defined in inches for example, vertically or horizontally oriented; CRT represents the certificate calculated for the designated text block; PHT represents the photograph; and, FIG de8ignates the diagram. This short string then completely describes the basic layout of the document. within the string can be embedded additional information such as a description of the subplane contents (e.g., text, figure, equation, photo), precise x,y coordinates of the subplane on the page, and of course the error detection certificates. To remain unobtrusive to the human reader, it is possible to "hide" the markers in, for example, a logo. A 19 x 19mm (¾" x ¾") logo can encode over 1,000 bits of information. Other embodiments may include using invisible inks or hiding the markers in the format of the document itself. It is not necessary that the information be provided apart from the human readable portion of the document, for example in a blank margin. What is necessary is that the information be provided in such a manner that the computer can readily ascertain its location and easily read the encoded information; and that it not interfere with the human readable portion in such a manner as to render it unreadable. Since the markers are being described in terms of OCR use, it has been assumed thus far that the medium for reading the marker is a digitising scanner. With the development of other input media, the encoding schemes will require adaptation to accommodate the relevant system. As with all of the preceding discussion, such modifications as would occur to one having skill in the art may be made without departing from the scope of the appended claims wherein:

Claims: A method for ensuring the accuracy of data sequences electronically reproduced from original data sequences comprising: first deriving one or more data-dependent first digital values from said original data; second deriving one or more data-dependent second digital values from said reproduced data; comparing said first and second digital values and identifying and correcting errors and error locations within said reproduced data sequences based on the difference between said first and second values; characterised in that each of said second digital values correspond to a separate one of said first digital values and that said method further comprises effecting one or more methods of automatic correction of said identified errors on said reproduced data, the number and capacity of the methods being dependent on the size and kind of the digital values and/or on additional external information. A method according to claim 1, for electronically reproducing original data sequences of computer-pre-processed printed documents comprising the further steps of: printing said document including said one or more first data-dependent digital values derived from said original data; and creating an electronic document having reproduced data sequences by optically scanning said printed document and said first digital values and saving said electronic document and said first digital values; wherein said one or more second data-dependent digital values are derived from said reproduced data of said electronic document. A method according to claim 2, wherein the step of printing said document includes creating a first computer version of said document in a first computer; deriving at least one first digital value as a said data dependent first digital value for said first version of said document, and creating a printed version of said document by outputting said first version and said at least one first digital value from a printer associated with said first computer; wherein the step of creating said electronic document comprises: creating a second computer version of said document comprising reproduced data by optically scanning said printed version and said at least one first digital value with a scanner associated with a second computer having software for recognizing said optical characters and said digital value; and wherein the step of deriving said one or more second data-dependent digital values comprises deriving at least one second digital value for said second computer version of said document from said reproduced data. A method according to claim 3, wherein said effecting automatic error correction produces a third computer version of said document. A method according to any one of claims 1 to 4 wherein said deriving comprises calculating said data-dependent digital values. A method according to any one of claims 1 to 5 wherein said automatic correction produces error-corrected reproduced data and further comprising the steps of: deriving one or more data-dependent third digital values from said error-corrected reproduced data; and comparing said first and third digital values. A method according to claim 2 or 3 wherein said deriving of first and second digital values comprises applying the same algorithm to the data of each of said computer-generated and electronic documents. A method according to any one of claims 2, 3 or 7 wherein said creating and said deriving of said second digital values are conducted simultaneously. A method according to claim 2 or 3 wherein said printing of said one or more first values comprises providing said one or more first values in a non-textual machine-readable symbology. A method according to any one of claims 1 to 10, wherein said data sequences comprise lines of original data. A method according to claim 2 or 3, or any one of claims 4 to 10 as appended thereto, wherein said printed document comprises a plurality of blocks comprising at least multiple text blocks of multiple lines of original data and wherein said deriving of said at least one first digital value includes providing at least one first digital characterization of the spatial relationship of said plurality of blocks in said document. A method according to claim 2 or 3, or any one of claims 4 to 11 as appended thereto, for electronically reproducing a computer-pre-processed printed document having a layout comprising a plurality of original blocks comprising at least multiple text blocks of multiple lines of characters comprising the steps of: deriving at least one first layout-dependent value from the spatial relationship of said plurality of original blocks on said document; printing said multiple text blocks of multiple lines and said at least one first layout-dependent value; creating an electronic document with layout comprising a plurality of reproduced blocks comprising at least multiple text blocks of multiple lines of reproduced characters by optically scanning said multiple lines and said at least one first layout-dependent value; deriving at least one second layout-dependent value from the spatial relationship of said plurality of reproduced blocks on said electronic document; comparing said at least one first layout-dependent value to said at least one second layout-dependent value; and detecting and locating layout errors based upon the difference between said at least one first and second layout-dependent values. A method according to claim 12 further comprising the steps of: deriving at least one first character-dependent value from said characters or said document; and printing said at least one first character-dependent value upon printing of said multiple lines and said at least one first layout-dependent value. A method according to claim 13 further comprising the steps of: deriving at least one second character-dependent value from said reproduced characters of said electronic document; comparing said at least one first character-dependent value to said at least one second character-dependent value and identifying character errors and error locations in said lines based upon the differences between said first and second character-dependent values; and effecting one or more methods of automatic correction of said identified character errors on said reproduced characters. A method according to claim 1 for ensuring the accuracy of a second document comprising reproduced data sequences reproduced by optically scanning a first document, comprising original data sequences and at least one encoded, machine-readable, first data-dependent digital value assigned to and scannable on the face of said first document, comprising the steps of: decoding said at least one encoded, machine-readable, first data-dependent digital value; deriving at least one second data-dependent digital value for said second document from said reproduced data sequences; comparing said at least one first data-dependent digital value to said at least one second data-dependent digital value and identifying errors and error locations within said reproduced data sequences based upon the differences between said first and said second data-dependent digital values; and effecting automatic error correction on said reproduced data sequences when said first and said second data-dependent digital values are different.

Inventors: KORTH HENRY F, LIPTON RICHARD J, LOPRESTI DANIEL P, SANDBERG JONATHAN, KORTH, HENRY F., LIPTON,RICHARD J., LOPRESTI, DANIEL P., SANDBERG, JONATHAN, Lipton,Richard J. c/o Amster,Rothstein &amp; Ebenstein
Applicants: MATSUSHITA ELECTRIC IND CO LTD, MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD.
Application year: 1993
Main classification: G06K9
IPC codes: G06K9, G06T1