PageX : An Integrated Document Processing and Management Software for Digital Libraries

For digital libraries it is very important to design and to implement powerful engines to convert information on paper to electronic format. In this paper a PageX software is proposed as the integration of such engines, which include a set of intelligent document processing functions, a set of compact document management strategies, and a set of advanced accessories. With this software, a paper document will first be input as an optical image, which may be a mixture of graphics and text and may be skewed. The image will then be analyzed and decomposed into a series of component blocks, and encoded and stored in a structured and compact format. Well-developed accessory functions, including block editing and page annotation, page reconstruction and virtual editing, page matching and registration, document retrieval, etc., are provided to support advanced applications. With these carefully designed functions and strategies, PageX minimizes manual operations to a minimal degree. Index Terms – Digital library, document processing, document database, page segmentation I. OVERVIEW OF PAGEX SYSTEM igital libraries are some of the most typical applications of multimedia data storage and retrieval [1-3]. Page document oriented software [4] is an important aspect of digital library research. It has been emphasized that high-accuracy Optical character recognition (OCR) is greatly required in document auto-reading systems. However, OCR does not perform a unique role in document processing. Actually, due to a series of unavoidable factors (such as a mixture of text and graphics in documents, shadow and distortion in document image acquiring, etc.), very possible the scanned documents can only be treated as raw images, where no significant text information can be extracted from such page images. Clearly, such a case is unbearable for electronic document transmission and storage. Is there any solution to deal with a general document on paper and to automatically convert its scanned image to electronic format and to offer advanced properties of editing and retrieval? Is there any method to decrease the manual work in document processing to a trivial level? This paper introduces PageX as one of our efforts for tackling such problems. Fig.1 shows the main paradigm of PageX, which consists of three major modules and a TWAIN compatible image acquiring module. The first major module (engine set I) of PageX is intelligent document processing, which automatically analyzes page information and decomposes a page image into a Component block list (CBL). This module has little manual operation. The second module (engine set II) of PageX is compact document management, which encodes all the document components (blocks and other document properties) and offers a flexible data structure for transmission and storage. With the second module, CBL is further organized as compact Electronic document (e-Doc). The third module (engine set III) of PageX is a set of advanced accessory functions for documentoriented applications, such as the editing and output (saving as other formats, printing, dynamically linking and outputting to database, tc.). Retrieval engines are also included in this part. These three parts are explained in subsequent sections. It is interesting to compare PageX with some other document processing research in digital libraries. For example, the document processing system developed by the group of Document image decoding (DID) * at Xerox aims at developing Advanced structured documents (ASD) that provide contented-based access to information extracted from the scanned page, but are not necessarily intended to be reconstruction of corresponding paper documents . This is in contrast with PageX, which attempts to provide both structured documents and good reconstruction of the original paper documents. Therefore PageX can avoid * http://elib.cs.berkeley.edu/kopec/ D improper or wrong processing, and can be further utilized in areas other than digital libraries, such as document communication in teleconferencing. Fig.1 The scheme of PageX II. INTELLIGENT DOCUMENT PROCESSING A set of intelligent functions for document analysis is included in PageX to analyze and extract the page information (Fig.1). These functions include automatic correction for skew images with unknown distortion angles, automatic extraction of foreground from complex background, automatic page blocking, automatic language separation and character segmentation, character recognition, etc. With these functions, an acquired (either scanned or captured) page image, which contains different languages of texts, various types of graphics, and other information (Fig.2(a) and Fig.3(a)), will be decomposed into a series of component blocks, which have different types. These blocks make up the CBL of the original document page. The main idea in designing these functions is to provide intelligent/automation techniques to decrease the manual operation as much as possible. Several most important functions are briefly introduced in the following subsections. A. Foreground Extraction and Binarization It is unusual for many document reading and processing software packages to include a powerful and intelligent binarization module. Actually, due to the uneven property of input images, a global threshold, either automatically or manually selected, generally leads to poor binarization, in which situation the resulted page image often contains much wrong information. What's worse, it is usually hard to get rid of such wrong information ( e.g. noisy blocks and irregularly shaped spots). Another problem is the original scanned page images have a strong likeness to comprise foreground texts and background pictures. A typical example is shown in Fig.2(a), where the foreground texts should be extracted from the background picture of a building. A self-adaptive binarization engine is implemented to extract the foreground and to set the background to be white. This engine will firstly analyze all subregions of the whole image and then produce a mesh of binarization threshold. This mesh can lead to better binarization outcomes than a global binarization threshold. After binarization, an engine for noise elimination is used to erase most point noise and texture background. The processed image, as shown in Fig.2(b), is seen in good quality.