Patent Publication Number: US-6704753-B1

Title: Method of storage management in document databases

Description:
FIELD OF THE INVENTION 
     The present invention describes a method of storage management in document databases. 
     BACKGROUND OF THE INVENTION 
     In banks and insurance companies, thousands of documents are handled daily. In the case of banks, transfers and account withdrawal documents are most common; with regard to insurance companies, it is principally policies or claims. To be able to access and search these documents, they are usually filed in databases. The period in which these documents have to be accessible in the databases is determined chiefly by the type of the documents. As well as purely operational requirements, statutory regulations also exist regarding the period for which documents are kept. Since the quantity of data (quantity of documents) e.g. in banks and insurance companies, is constantly increasing over time and since existing databases cannot manage these increasing quantities of data, organisation into logical storage areas has become established as a leading concept for document management. This concept specifies that the entire logical storage area available (which may also extend over several systems) is divided up into storage segments, each individual storage segment only being able to hold one type of document defined by the user. If the relevant storage segment is full, this segment is closed for the receipt of further documents and a new segment is created for this type of document (see FIG.  1 ). This concept is described in greater detail in the published PCT application WO 97/16794 of the applicant. 
     In theory, an unlimited number of documents can be filed by means of this concept. 
     Normally, the documents put in the segments are filed first on the fast hard disk (part of the logical storage area). Owing to their speed, hard disks of this kind are relatively expensive. Modern computer systems therefore feature storage management systems (SMS), such as DFSMS by IBM, for example, which relocate files which have not been used for a fairly long time to slower and less expensive storage media, e.g. data tapes. If one of the relocated documents is to be accessed again, this is ascertained by the SMS, and it makes sure that the data are transferred from the data tapes back to the fast hard disks (recall). A process of this kind takes a relatively long time. It may take a few seconds to several minutes. 
     The document management systems currently available on the market, e.g. VisualInfo or OnDemand (R/DARS), manage a restricted storage space, similar to the situation with the main storage space in a computer system. This means that all documents are first collected in a database (document pool). Once this segment is full, either completely or to a certain degree, the segment is relocated to another storage medium, e.g. magnetic tapes or CDs. 
     Information on which elements have been relocated and which segments they are in is managed by the pertinent document management system (DMS). If a document is sought in the DMS, this establishes whether it is in the segment which it can access directly. If not, it looks in its management information to see which relocated, packed segment the document is in and procures this document. This packed segment is then read into a temporary storage space and the document sought is procured. If another document is then sought, the temporary storage space is released and occupied by the new documents. A temporary storage space of this kind may be a database, as in the case of OnDemand, or a CD drive, as for VisualInfo. 
     Documents which have already been found are then always deleted from the temporary storage space if other documents which have been relocated are sought. 
     The disadvantage of these systems is that no special access features to the document are taken into account. The documents are normally stored in order to be able perhaps to find them again one day and reproduce them. However, in this regard certain documents have different access periods and access probabilities, e.g. bank transfers are searched for normally for a maximum of 6 weeks, as banks have a corresponding appeal period for customers. For other documents, such as loan extensions, which sometimes run for months and are being constantly added to, searches for these documents are conducted in these time spans. Since the recall of relocated documents takes a relatively long time compared with accessing documents which are still in a segment, it is a disadvantage for all documents to be treated in the same way on relocation. The disadvantage here is that all documents are treated in the same way and that the criterion for relocation is the size and availability of an existing, equally large space. 
     It is similarly the case for documents which, following relocation, have been recalled. Some documents only need to remain stored for a short period, while in the case of other documents longer periods are required in which they should be available on-line. This is due to the fact that they will in all probability be re-accessed in a certain time period. 
     In the previous example of the bank, it is usually the case that transfers are only recalled briefly in order to reproduce them so that they can be used as evidence in relation to customers. In contrast, processes normally take longer in the case of loan applications. One has in mind here loan extensions which, with corresponding negotiations over interest rates and possible counter-offers obtained by a customer from competitors, normally extend over several weeks. 
     SUMMARY OF THE INVENTION 
     The object of the present invention is to provide a method of document management which avoids the aforementioned disadvantages. 
     This invention is achieved by the features of claim 1. Further advantageous forms of execution of the invention are set out in the sub-claims. 
     The essential advantages of the invention lie in the fact that the documents are found faster. Time and money are saved hereby. Lower-cost storage means can be used due to the relocation of documents. The total quantity of expensive storage means is hereby reduced. The residence period of the document in a storage means can be determined individually for each document by the user. However, the residence period can also be determined by a program, which calculates the access frequency for documents or document types and establishes the residence period according to this. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     The present invention is explained in greater detail with reference to a preferred practical example, in which: 
     FIG. 1 shows a logical storage area with segment division, 
     FIG. 2 shows a state-time diagram of a segment according to FIG. 1, 
     FIG. 3 shows a flow diagram of a search in a document management system with storage management according to the present invention, 
     FIG. 4 shows a structure of an exemplary storage segment according to the present invention. 
    
    
     DETAILED DESCRIPTION OF THE DRAWINGS 
     FIG. 2 shows a state diagram, which represents the individual residence times of a document. 
     The idea of the present invention subsists in the fact that residence times are assigned to the individual documents. Two different residence times are preferably assigned to the document. The first residence time (residual time) is defined as the time span between storage of the document in the DMS and the time at which the document may be relocated for the first time. The second residence time (keep time) is defined as the time span between recall of the document to the DMS and the time at which it may be relocated again. A further configuration of the present invention subsists in the fact that the second residence time is only stipulated if the document is recalled to the first storage means. 
     The residual and keep times are assigned on entry of the document into the DMS. This assignment can be effected via several mechanisms, which are described here by way of example: 
     The times are assigned by the user who files the document in the DMS. This user can either assign these times entirely freely or receives suggested limited time spans set by the system between which he can choose. 
     The times are laid down by a manager on description of the document type in the DMS. These times are assigned to each document of this document type which newly enters the DMS. The length of the keep and residual times is laid down by a manager on the basis of empirical values or operational or statutory requirements. 
     Another option would be for the DMS to calculate statistics regarding access probabilities and access frequencies on the basis of actual accessing and then assign these to new documents or the document type. 
     The classification of keep and residual times can either be assigned directly in the document, remain as a classification according to the description of the document type in the DMS or be given externally to the storage medium in which the document is filed. On entry of the document into the DMS, its state is designated as “stored”. 
     The residual time will preferably commence for a document at the time of entry into the DMS (T 0 ). For administrative reasons, this starting time can however also be the start and end time of the segment (e.g. segment “B”, TF  2 -TF  3 , see FIG. 1) or a time calculated according to mathematical viewpoints (e.g. arithmetic means) between the start and end time of the relevant segment. 
     If a document is accessed during the residual time, the start time of the residual time can either remain unchanged or can be brought forward to the time of the last access. 
     The document is then relocated only on expiry of the residual time. This relocation can be carried out in accordance with several criteria or types of management. Each document is fetched individually from a segment of the logical storage area and relocated once the residual time (T 2 ) has expired. To do this, a program which performs this relocation must be started at this time. This can either take place in a timed manner on expiry of the residual time, or this program is started regularly (e.g. once a day) and checks whether the residual time has expired for documents. This program can either be part of the DMS or of an external program, for example a storage management program (SMS) such as already exists for larger computer systems (example: DFSMS with HSM for IBM OS/390). The documents can be relocated singly, all together or according to document type. On relocation of the documents, the document can then either be removed completely from the database and stored in another form, due to which the type of access to this document may change, or the files in which the documents are filed are relocated and the form and access method remain unchanged. 
     Files can be relocated singly, all together or according to various criteria (e.g. index data, content data). 
     If the documents are then relocated, their state is then designated as “relocated”. Relocation can also take place asynchronously to the DMS. In this case, the DMS can establish whether a document or a segment with documents has been relocated and enter this in its management information. This is effected by information from the SMS to the DMS or interrogation of the SMS by the DMS or by the DMS attempting to access these documents and in doing so receiving a resulting value which tells it that the documents are no longer present through direct access. 
     FIG. 3 shows a flow diagram of a search in a document management system with the inventive storage management. 
     If the DMS receives a search request for documents, it establishes with reference to its management information whether this document is still directly accessible or whether it has already been relocated. If it is still directly accessible, the document is sought and procured entirely normally. If a document is sought which is no longer directly accessible, the document first has to be procured. As described in the patent referred to, documents can be separated into profile information and document contents (see FIG.  4 ). Profile information is information which can be used to find documents again. Document contents on the other hand are the actual documents, e.g. the image of a transfer. In a search for a document which has been relocated, therefore, either the complete documents concerned can be recalled, or for the search only the profile information may be recalled. The document contents can then be procured once it is established that a document also has to be procured for a request. Since, as described above, documents are relocated mostly in segments, these segments have to be recalled. This can either be done synchronously with processing of the search request or asynchronously by writing a procurement request and inactivating the search request for its period of execution. In this case, it is also possible to directly call the program which is to undertake the recall, or this program can be started in a timed manner. 
     Since recall takes a relatively long time, at least several seconds, but mostly several minutes, an interim state can be assigned to the state of the documents or the segments with documents, e.g. “being recalled”. It is achieved thereby that other search requests relating to the same document or the same segments with documents do not have to write any further recall request, but can be inactivated immediately until the documents have been recalled. 
     Once the documents have been procured, the state of the documents concerned is changed from “relocated” or “being recalled” to “recalled”. The documents thus become directly accessible to the DMS once more. 
     The requests which were inactivated due to documents or segments with documents having been relocated can now be reactivated. This can be performed directly by the program which executed the recall request or is effected in a timed manner, so that the search request is activated at certain intervals and it is checked whether the recall request has been executed. 
     The keep time commences at the time of recall of the document. As already described, it can be assigned fixedly to the document on introduction of the document or is assigned to the document only at the time of recall. This can be carried out with reference to a fixedly defined or calculated value, as already described in the case of residual time. The document can now be searched for in the context of the search request or the document be resupplied. 
     If during the keep time this document or segment with the documents is accessed once again in the context of document search requests, the keep time can be started anew or runs out at the first calculated end time. 
     On the expiration of the keep time, the document is then relocated in the same process as already effected on expiration of the residual time.