Method and distributed database file system for implementing self-describing distributed file objects

A method and apparatus are provided for implementing self-describing file objects. A node group is created for defining multiple computer systems for storing data. A hash algorithm for applying to data records is identified. A partition distribution map for distributing data to each of the multiple computer systems utilizing a set of predetermined hash algorithm results and remote system information for each of the multiple computer systems are identified. A file object is created in each of the multiple computer systems. Each the file objects includes the hash algorithm, the partition distribution map, and the remote system information. A data record is inserted into one of the distributed file objects by receiving the data record, applying the hash algorithm to the received data record, comparing the hash algorithm result with the partition distribution map to identify the particular computer system for the data record, utilizing the system information to establish connection to that system. The file objects are fully self-describing, eliminating the need for additional objects to be addressed, opened, paged into memory or the like.

FIELD OF THE INVENTION 
The present invention relates to a distributed database file system and 
method for implementing self-describing file objects. 
DESCRIPTION OF THE PRIOR ART 
Several known distributed database, or parallel database implementations 
exist. In known implementations, external constructs must be accessed for 
operations including hashing, communications information and partitioning. 
When implementing a distributed database file, or any object that spans 
multiple systems, there is a need for each piece of the distributed object 
to have information about the other pieces. In the instance of a 
distributed database file, each system must have information about the 
other systems that the file is distributed across, as well as have 
information about how the data itself is partitioned. If a piece of the 
distributed file is accessed, and the user wants to retrieve all of the 
data in the entire distributed file, then the local system has to 
determine which remote systems to access in order to retrieve all the 
data. Furthermore, when data is inserted into the file, the file needs to 
know the partitioning scheme for the data, in order to determine where to 
store the newly inserted data. 
A need exists for an improved method for implementing a distributed 
database, or any cross-system file object. 
SUMMARY OF THE INVENTION 
An important object of the present invention is to provide an improved 
method and apparatus for implementing self-describing file objects. 
In brief, a method and apparatus are provided for implementing 
self-describing file objects. A node group is created for defining 
multiple computer systems for storing data. A hash algorithm for applying 
to data records is identified. A partition distribution map for 
distributing data to each of the multiple computer systems utilizing a set 
of predetermined hash algorithm results and remote system information for 
each of the multiple computer systems are identified. A file object is 
created in each of the multiple computer systems. Each the file objects 
includes the hash algorithm, the partition distribution map, and the 
remote system information. 
In accordance with a feature of the invention, a data record is inserted 
into one of the distributed file objects by receiving the data record, 
applying the hash algorithm to the received data record, comparing the 
hash algorithm result with the partition distribution map to identify the 
particular computer system for the data record, and utilizing the system 
information to establish connection to that system. The file objects are 
fully self-describing eliminating the need for additional objects to be 
addressed, opened, paged into memory or the like.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Having reference now to the drawings, in FIG. 1 there is shown a 
distributed computer or data processing system of the preferred embodiment 
generally designated by the reference character 10. Distributed computer 
system 10 includes multiple computer systems 12 labeled #1, #2, and #3, as 
shown in FIG. 1. Each computer system 12 includes a processor 14, a memory 
16 containing a distributed file object 18 of the preferred embodiment, a 
user input 20, a user interface 22 and a network adapter interface 24. The 
multiple computer systems 12 are connected to a network 26 via the network 
adapter interface 24. 
In accordance with the preferred embodiment, the database file or 
distributed file object 18 is arranged for storing data portions 
distributed over the multiple computer systems 12. Each distributed file 
object 18 contains enough information to locate all of the other 
distributed file objects 18. A method of the preferred embodiment for 
locating data uses a hashing algorithm and predetermined partitioning 
data. The distributed file objects 18 themselves are fully self-describing 
and eliminate the need for any additional descriptive or operational 
information. 
In the distributed computer system 10, each of the multiple computer 
systems 12 can be implemented with various different commercially 
available computers, such as an IBM PS/2 including a storage file system. 
Also other types of computer system, whether it be another microcomputer 
such as an Apple Macintosh, a minicomputer such as an IBM System/390, or a 
microcomputer connected to a larger computer system such as an IBM AS/400 
can be used for any of the multiple computer systems 12 and fall within 
the spirit and scope of this invention. It should be understood that the 
computer system 10 includes multiple computer systems 12 with the 
communication network 26 defined by an local area network, a wide area 
network, or other communication network between the multiple computer 
systems 12. 
Having reference to FIG. 2, each distributed file object 18 contains a hash 
algorithm 31 that is applied to new data to determine its hash value and a 
partitioning scheme or partitioning data 32 that correlates each of 
possible hash values to a specific one of the systems 12. The hash 
algorithm 31 is generated in the file object 18 at creation time of the 
file object 18 and can be customized for the specific attributes of the 
file's data types. The hash algorithm 31 yields a hash value, which is 
simply a number from a predefined set of possible numbers. The 
partitioning data 32 that is stored in each file object 18 is an array of 
values that maps each of the possible hash values to one of the systems 12 
that will hold a portion of the distributed file data 36. Each distributed 
file object 18 contains communications information 34 for each of the 
remote systems across which the distributed file object 18 is distributed. 
Each distributed file object 18 is complete entity. There is no dependency 
on related objects on the systems 12 that perhaps could get destroyed, 
corrupted or get out of synch. Everything needed is in the file. There is 
also the secondary benefit of performance. Once the file is addressed, 
there is no need for additional objects to be addressed, accessed, opened, 
paged into memory, or the like. Each of the distributed file objects 18 
contain data 36 that is allocated to the different computer systems 12 #1, 
#2, and #3 in accordance with the partition data 32. 
Referring now to FIG. 3, the sequential operations performed by processor 
14 of one of the computer systems 12 to create the distributed file object 
18 begin as indicated at a block 300 with creating a node group as 
indicated at a block 302. Remote system information is identified for each 
of the multiple computers systems 12 #1, #2, and #3, as shown in FIG. 1, 
as indicated at a block 304. A hash algorithm 31 to be used for the file 
object 18 and a partition distribution 32 for partitioning data among each 
of the multiple computers systems 12 are identified as indicated at a 
block 306. Various partitioning schemes can be used for partitioning data, 
for example, a default partitioning scheme can provide an even 
distribution of data among each of the computer systems. Alternatively, a 
user specified partitioning scheme can provide a selected percentage for 
particular ones of the computer systems 12, such as 50% of the data to be 
stored on computer system 12 #1 and 25% of the data on each of computer 
systems 12 #2 and #3. A file object is created and stored on the 
particular one of the computer systems 12 as indicated at a block 308. The 
file object contains the hash algorithm 31, partition distribution 32, and 
remote system information 34. A remote connection is established to 
another one of the multiple computer systems 12 and the file object is 
created on that particular system 12 as indicated at a block 310. Checking 
for other of the multiple computer systems 12 that have not been accessed 
is provided as indicated at a block 312. A next remote connection is 
established to another one of the multiple computer systems 12 and the 
file object is created on that particular system 12 until the file object 
has been created on all the computer systems 12. 
Referring to FIG. 4, the sequential operations performed by processor 14 of 
one of the computer systems 12 begin as indicated at a block 400 with 
opening the distributed file object 18 as indicated at a decision block 
402. A data record to be stored is received as indicated at a block 404. 
The hash algorithm 31 is applied to the received data as indicated at a 
block 406. Then the hash algorithm result is compared with the partition 
data 32 as indicated at a block 408. A remote system can be identified as 
indicated at a decision block 410. When a remote system is identified at 
block 410, then a connection to the identified system is established as 
indicated at a block 412. The data record 36 is inserted in the particular 
remote system as indicated at a block 414. Otherwise, when a remote system 
is not identified at block 410, then data record is inserted in the system 
that received the data record at block 414 to complete the operations as 
indicated at a block 416. 
The result of this implementation is that when the distributed file object 
18 is opened, and the user wants all the data, all of the remote 
distributed file objects 18 can immediately opened as well as the local 
distributed file object 18. Also, there is a very simple process involved, 
since the information is self-contained in the distributed file object 18. 
At open time, the open process knows that the file is a distributed object 
file 18, and can quickly find the communications information 34 needed to 
establish the remote connections. No other objects or external constructs 
need to be accessed in order to establish the remote connections. Another 
benefit results when data is added to the file object 18, or data is 
updated in the file object because the hash algorithm 31 applied to the 
new data is stored in each file object 18. Again, the entire process can 
be handled without accessing external programs, external catalogs, or any 
other objects that describe this hashing information. Also, the above 
process works in reverse, when querying the file for specific values. When 
the user is querying for specific data, the hash algorithm 31 is applied 
to the desired data, and the hash result is compared to the partitioning 
data 32 to immediately identify which system 12 #1, #2, or #3 contains 
that data. Note that this can only be done when the user has provided a 
search predicate that involves a test for equality. 
Referring now to FIG. 5, an article of manufacture or a computer program 
product 500 of the invention is illustrated. The computer program product 
500 includes a recording medium 502, such as, a floppy disk, a high 
capacity read only memory in the form of an optically read compact disk or 
CD-ROM, a tape, a transmission type media such as a digital or analog 
communications link, or a similar computer program product. Recording 
medium 502 stores program means 504, 506, 508, 510 on the medium 502 for 
carrying out the methods of the preferred embodiment in the system 10 of 
FIG. 1. 
A sequence of program instructions or a logical assembly of one or more 
interrelated modules defined by the recorded program means 504, 506, 508, 
510, direct the computer systems 12 for implementing self-describing file 
objects of the preferred embodiment. 
While the present invention has been described with reference to the 
details of the embodiments of the invention shown in the drawing, these 
details are not intended to limit the scope of the invention as claimed in 
the appended claims.