Abstract:
Versioned metadata allows multiple versions of metadata for a given object to exist concurrently on a system thereby supporting the parallel execution of operations which would otherwise be mutually exclusive. Metadata updates are permitted while other objects maintain access to another version of the metadata through a versioning infrastructure. The versioning infrastructure allows the creation of new versions of metadata, maintains obsolete versions in the system as long as they are being used by any object, and deletes the obsolete versions once they are no longer in use.

Description:
CROSS-REFERENCED AND RELATED APPLICATIONS  
       [0001]     This application claims the benefit of U.S. Provisional Application Ser. No. 60/571,361 filed on May 14, 2004. This application is related to co-pending U.S. Application Ser. No. ______ filed on even date herewith entitled “System for Allowing Object Metadata to be Shared Between Cursors for Concurrent Read Write Access,” which claims the benefit of U.S. Provisional Application Ser. No. 60/571,362 filed on May 14, 2004. These applications are hereby incorporated by reference in their entireties as if fully set forth herein.  
     
    
     BACKGROUND AND SUMMARY  
       [0002]     This invention related to computer systems, more particularly to managing versions of metadata.  
         [0003]     Versioned metadata allows for multiple versions of metadata for a given object to exist concurrently on a system. This is required to support the parallel execution of operations which would otherwise be mutually exclusive. For, example, a system cache can serve as a global repository of cached metadata for objects such as tables and indexes. When a request is received to access a resource, the database system converts the request through compilation into a program unit, or cursor. When a program unit accesses an object, a lock can be placed on the metadata for that object. This lock prevents any changes to the metadata so that the program unit can execute the desired operation without mid-execution changes from other program units that may cause the program unit to fail or may invalidate the result. This means that long running program units may lock the metadata for a long period of time, potentially blocking object changes that cause metadata updates even if those updates were known to have no affect on the executing program unit. This inherently limits scalability and performance as all activity on an object must be completed before that object, and consequently, before its metadata, can be modified. A solution is required which will allow programming units to execute while concurrently allowing object updates.  
         [0004]     One embodiment of managing versions of metadata includes linking a dependent object for a resource to the most recent metadata version of the resource, maintaining any metadata version while in use, and deleting any metadata version when no longer in use.  
         [0005]     In one system embodiment, the versioning infrastructure includes a search and create process, an aging-out process, and a dependency mechanism. In another embodiment the dependency mechanism is not included. The search and create process manages new and obsolete versions of metadata. The aging-out process maintains the obsolete metadata versions in the system while needed. The dependency mechanism allows a client or another object to declare interest in, or dependency on, a metadata object. In another embodiment, a viewing process allows the viewing of any and/or all active versions of a metadata. Further details are described below in the detailed description, drawings and claims.  
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0006]      FIG. 1  is a representation of the search and create process.  
         [0007]      FIG. 2  is a representation of the aging-out process.  
         [0008]      FIG. 3  is a representation of versioned metadata over time.  
         [0009]      FIG. 4  is a representation of the dependency mechanism over time.  
         [0010]      FIG. 5  is a representation of a system in which managing transient versions of metadata takes place.  
     
    
     DETAILED DESCRIPTION OF INVENTION  
       [0011]     A system, method, and computer program have been created which allow for the creation and management of versioned metadata. In the following description, for the purposes of explanation, one or more embodiments having specific combinations of elements are set forth in order to provide a thorough understanding of the solution. It will be apparent, however, to one skilled in the art that the embodiments may be practiced without these specific combination of elements.  
         [0012]     Versioned metadata allows an the creation of a new version of metadata as needed for execution of an intended operation that would otherwise be prohibited. For example, a client can issue a write request to modify an object. However, if the object is in use the metadata would be locked, preventing object edits. By allowing the creation of an additional version of that object&#39;s metadata, the write request can execute. In other words, by using versioned metadata clients are permitted to modify an object even if that object is in use by another client. Program units using the metadata of a modified object can maintain access to the previous version of metadata.  
         [0013]     One embodiment of managing versions of metadata includes linking a dependent object to the most recent metadata version of the resource, maintaining the metadata version while in use, and deleting the metadata version when no longer in use.  
         [0014]     In one system embodiment, the versioning infrastructure includes a search and create process, an aging-out process, and a dependency mechanism. In another embodiment the dependency mechanism is not included. The search and create process manages new and obsolete versions of metadata. The aging-out process maintains the obsolete metadata versions in the system while needed. The dependency mechanism allows a client or another object to declare interest in, or dependency on, a metadata object. In another embodiment, a viewing process allows the viewing of any and/or all active versions of a metadata.  
         [0015]     When an operation is requested for an object, the metadata for the object is accessed. The search and create process details how the appropriate metadata version is accessed.  FIG. 1  depicts a representation of the search and create process  100  according to one embodiment. For the purpose of this example, assume that two types of requests may be issued to access the metadata. A first type of request is to create a new version of the metadata (e.g., a write or update operation request). A second type of request will not create a new version of the metadata but uses the current version (e.g., a read operation request). The request may identify a resource, or object, of the system to be operated upon. A request to access a resource occurs at process action ( 102 ). It is determined at process action ( 104 ) if the request is a request for a new version of the metadata, i.e., a write request. If the request is for a new version, the current version of the metadata is marked obsolete ( 106 ), and a new version of metadata is created ( 108 ). If it is determined at ( 104 ) that the request is not a request for a new version of the metadata, i.e., a read request, then a search ( 110 ) is conducted for the most recent version of the metadata. It is determined at process action ( 112 ) if the retrieved metadata version has been marked obsolete. If it is determined at process action ( 112 ) that the retrieved metadata version has been marked obsolete, then a new version of metadata is created and the request is linked to the new version at process action ( 114 ). If it is determined at ( 112 ) that the retrieved metadata version has not been marked obsolete, then it can be assumed that the most recent version of the metadata has been found, and the process links the request ( 116 ) to the current metadata version.  
         [0016]     For example, suppose a request to create a new partition in a table was issued. Process action ( 104 ) determines that the request is a request for a new version of metadata. The existing version of the table metadata is marked as obsolete ( 106 ) and the new version of the table metadata based on the modified table is created ( 108 ).  
         [0017]     In another example, suppose a client request to select data in a table is issued. Process action ( 104 ) determines that the request is not a request for a new version of metadata. Process action ( 110 ) searches for and locates the most recent version of the table metadata. If the metadata version found is not obsolete ( 112 ), then it is returned to the client.  
         [0018]     In another scenario, the metadata version returned in process action ( 110 ) is obsolete. This situation might occur if a previous operation to create a new version had failed, but the operation to obsolete the old previous version has already been performed. Consequently, a new metadata version was never created but the previous version was already marked obsolete. In this case, process action ( 114 ) would then create a new metadata version and return it to the client.  
         [0019]     Using the search and create process  200 , a system can have multiple concurrent versions of metadata.  FIG. 3  is a representation of concurrent versions of metadata over time according to some embodiments. For purposes of this example, the metadata is represented by rectangular structures labeled MDx, where “x” represent the object that the metadata describes, such as metadata MDa at t 0   302 , MDb at t 0   304 , and metadata MDa′ at t 1   322 . Each metadata structure has a “0” or “1” in an inset rectangle in the lower left corner. The inset rectangle represents the current version flag. A “1” in the current version flag indicates the metadata has not been marked obsolete, i.e., is current. A “0” in the current version flag indicates that the metadata has been marked obsolete. The structures connected to metadata MDa by double arrow lines represent objects, linked to the metadata. In this example, another metadata is linked to metadata MDa, as are two cursors, the ovals C 2   306  and C 3   308 . Although cursors C 2  and C 3  are shown as ovals and metadata are shown as rectangles, cursors are an example of metadata and may be shown by the same type of drawing element. Cursors having dashed lines indicate the cursor has completed its operation and has become freeable. Cursors removed from the drawing over time indicate the cursor is freed. Cursors added to the drawing over time indicate a new client request was issued and the resulting cursor was linked to the metadata. Metadata shown with dashed lined indicates the metadata has become freeable. Metadata removed from the drawing over time indicates the metadata was no longer needed and has been freed.  
         [0020]     In some embodiments, linked objects are tracked using a dependency mechanism.  FIG. 4  is a representation of an embodiment of the dependency mechanism of versioned metadata over time. The dependency mechanism provides data storage devices containing information regarding the relationship between parent objects and child objects. The dependency mechanism storage devices are represented by rectangular structures containing data fields. Each parent object stores dependency information in a Dependency Reference and each child object stores dependency information in a Dependency Structure. For example, each parent Dependency Reference contains a pointer to the Dependency Structure of each child (e.g., cursor or metadata) with which it is linked. Each child Dependency Structure contains a pointer to the Dependency Reference with which it is linked. An object can be both a parent and a child and can therefore have both a Dependency Reference and a Dependency Structure. This will become more clear as the figures are explained further.  
         [0021]      FIG. 3  and  FIG. 4  will be discussed in the following paragraphs in relation to the creation and linking process shown in  FIG. 1 .  
         [0022]     At time t 0 , three clients have requested access to object “a”. For each request the search and create process  100  performs the following: searches in process action ( 110 ) and finds metadata version MDa  302 , determines that metadata MDa  302  is not obsolete in process action ( 112 ) but is the current most recent metadata version indicated by the “1” in the current version flag  310 , and links the object to the metadata in process action ( 116 ). Metadata MDb  304 , and cursors C 2   306 , and C 3   308  are shown linked to the current version of MDa in the representation of an embodiment shown in  FIG. 3 . At this point there are three child objects MDb  304 , C 2   306 , and C 3   308  acting upon object “a”, and accessing metadata MDa  302 . The three objects will have access to metadata MDa until they complete their operation. If another read request comes in at this time, its object would also be linked to metadata MDa  302  as a result of the search and create process  100 .  
         [0023]     In another embodiment, MDb  304  and cursor C 2   306  are currently accessing object MDa  302  when another read request to access object “a” such as cursor C 3   308  is received. Since it is a read request and does not modify the metadata, C 3   308  would share the current version of Mda. In this instance, cursor C 3   308  has not yet accessed object “a” but is in a queue waiting to obtain access as indicated by the dashed link  390  to MDa  302 . In other words, cursor C 3  has declared interest in metadata MDa using the dependency mechanism but is waiting to obtain an active link and begin executing. In yet another embodiment, nested linking is permitted. For example, MDb  304 , C 2   306 , and C 3   308  may have another object linked to them.  
         [0024]     Also at time t 0 , the Dependency Reference for parent MDa  402  contains pointers  481 ,  482 , and  483  to each of the child dependency structures: MDb Dependency Structure  404 , C 2  Dependency Structure  406 , and the C 3  Dependency Structure  408 , respectively. Each of MDb Dependency Structure  404 , C 2  Dependency Structure  406 , and C 3  Dependency Structure  408  contain a pointer,  484 ,  485 , and  486  respectively, to the MDa Dependency Reference  402  for parent object metadata MDa.  
         [0025]     At time t 1 , all three objects MDb  314 , C 2   316 , and C 3   318  remain linked to MDa  312 . The MDa Reference at t 1   412  and child Dependency Structures  414 ,  416 , and  418  remain unchanged. However, a client requested for a new version of metadata such as C 4   324 , was received in process action ( 104 ), i.e., a request to modify object “a”. As a result, metadata version MDa at time t 1   312  is marked obsolete in process action ( 106 ) and a new metadata version MDa′  322  is created in process action ( 108 ). Current version flag  320  contains a “0” showing that metadata version MDa  312  is obsolete and is no longer the current most recent metadata version. Metadata version MDa′  322  is shown at time t 1  in  FIG. 3  with a “1” in the current version flag  326 , indicating that metadata MDa′  322  is not obsolete but is the current most recent metadata version. Since metadata MDa′ is the most recent not obsolete metadata, cursor C 4   324  is linked to metadata MDa′  322 . Along with a new metadata and cursor, also created are new parent object MDa′ Dependency Reference  422  containing the pointer to the child C 4  Dependency Structure  424 , and child C 4  Dependency Structure  424  containing the pointer to the parent MDa′ Dependency Reference  422 .  
         [0026]     At this point there is a current metadata version MDa′  322 , and an obsolete metadata version MDa  312 . Objects MDb  314 , C 2   314  and C 3   318  will maintain access to the obsolete metadata  312  until each is finished executing, or no longer needs to access the metadata. Cursor C 4   324  will maintain access to the current metadata version  322  until C 4  is finished executing. No new client requests will obtain access to the obsolete metadata  312 . All client requests will be linked to metadata MDa′ 322  by the search and create process  100  (unless another request for a new metadata version is received). Multiple versions of metadata can be created in this environment, and each one would be maintained while needed.  
         [0027]     At time t 2 , objects MDb  334 , C 2   336  and C 3   338  are finished executing. They are unlocked, and no longer need access to MDa  332 , although they still maintain a dependency on it. When an object having no dependents is no longer in use, it is marked freeable. As such, each object linked to MDa is marked freeable as illustrated in  FIG. 3  by the dashed lines of the child objects MDb  334 , C 2   336 , and C 3   338 . Since Objects MDb, C 2  and C 3  still depend on metadata MDa  332 , they are described as freeable dependents and are subject to the aging-out process, process  200  shown in  FIG. 2 . Process  200  is described later. In addition, no new cursors are linked to version MDa  332  because MDa is not the current metadata version and it is marked obsolete.  
         [0028]      FIG. 3  at time t 2  also depicts that a client requested a resource in search and create process  100  that is not a request for a new version ( 104 ). The resulting child, cursor C 5   346  is linked to metadata version MDa′  342  in process action ( 116 ) because MDa′  342  is the most recent metadata version ( 110 ) that is not obsolete ( 112 ). Metadata version MDa′  342  shows cursor C 4   344  remains linked and new cursor C 5   346  is linked. As such, parent Mda′ Dependency Reference at t 2   442  contains pointers for the child cursor C 4  Dependency Structure  444 , and the new child cursor C 5  Dependency Structure  446 . Each of cursor C 4  Dependency Structure  444  and cursor C 5  Dependency Structure  446  contain a pointer for the parent object, MDa′ Dependency Reference  442 .  
         [0029]     At this point metadata Mda′  342 , is the most recent metadata version, is not obsolete, and has two active dependents, C 4   344  and C 5   346 . Metadata Mda  332  is obsolete and has no active dependents. There are three freeable dependents, MDb  334 , C 2   336 , and C 3   338  which will be handled by the aging-out process described further below.  
         [0030]     At time t 3 , version MDa′  362  is shown as the current metadata version with cursor C 5   364  linked. The child C 5  Dependency Structure  466  contains the pointer for the parent MDa′ Dependency Reference  462 . And the parent MDa′ Dependency Reference  462  contains the pointer for the child C 5  Dependency Structure  466 . Cursor C 4   364  is no longer executing and is marked freeable, shown by dashed lines in  FIG. 3 .  FIG. 4  also shows the pointer to MDa′ Dependency Reference in the C 4  Dependency Structure  464  and the pointer to the C 4  Dependency Structure in the MDa′ Dependency Reference  462 . C 4   364  and metadata MDa  352  have been marked freeable as indicated by the dashed structures. Children MDb, C 2 , and C 3  have been freed via the aging-out process  200  as indicated by the removal of them from the drawing. As such, the MDb, C 2 , and C 3  memory can be reallocated as needed.  
         [0031]     As mentioned previously, dependency structures are used to track the relationship between parent and child objects. A Dependency Reference is used by the parent to track each child object. A child Dependency Structure is used by each child to track each parent. The pointers in the dependency structures indicate interest or dependency. Each version of the metadata remains in the system while any client is interested in it or dependent on it.  FIG. 2  depicts the aging-out process  200  that is triggered upon a dependent becoming freeable. For example, aging-out process  200  would be triggered in time t 2  when objects MDb, C 2 , and C 3  become freeable dependents, and in time t 3  when cursor C 4  becomes a freeable dependent.  
         [0032]     At time t 2 , objects MDb, C 2  and C 3  become freeable dependents. In this example, assume they become freeable dependents in written order. The first one to become a freeable dependent, MDb, will trigger the metadata aging-out process  200  at process action ( 202 ). The process determines if the freeable dependent was the last dependent of the metadata version in process action ( 204 ). Since there are three objects with a dependency on MDa, the first two times through process  200 , process action ( 204 ) determines that objects MDb and C 2  are not the last dependents and the system will resume other activities ( 214 ). When cursor C 3 , the third and final dependent, is marked freeable and goes though the metadata aging-out process  200 , process action ( 204 ) determines that it is the last dependent. Process action ( 206 ) determines if the metadata is locked by any other object. If the metadata is locked, the system will resume other processing activities ( 214 ). In this example the metadata is not locked by another object, and so the process continues. Process action ( 208 ) determines if the metadata is obsolete. If it is not obsolete the metadata is marked “can be freed” in process action ( 212 ). Another object could request a resource and still be linked to a metadata that is marked “can be freed” because it is still the most recent metadata version that has not been marked obsolete. In this instance, metadata MDa is obsolete, and is freed in process action ( 210 ). Methods for handling freed metadata resources are performed by the resource allocation subsystem and are known to those of ordinary skill in the art.  
         [0033]     In another embodiment, a viewing mechanism provides a client a way to view of all metadata versions obsolete or not.  
         [0000]     System Architecture Overview  
         [0034]     The execution of the sequences of instructions required to practice the invention may be performed in some embodiments by a computer system  1400  as shown in  FIG. 5 . In an embodiment, execution of the sequences of instructions required to practice the invention is performed by a single computer system  1400 . According to other embodiments, two or more computer systems  1400  coupled by a communication link  1415  may perform the sequence of instructions in coordination with one another. In order to avoid needlessly obscuring the explanation, a description of only one computer system  1400  will be presented below; however, it should be understood that any number of computer systems  1400  may be employed.  
         [0035]     A computer system  1400  according to an embodiment will now be described with reference to  FIG. 5 , which is a block diagram of the functional components of a computer system  1400 . As used herein, the term computer system  1400  is broadly used to describe any computing device that can store and independently run one or more programs.  
         [0036]     Each computer system  1400  may include a communication interface  1414  coupled to the bus  1406 . The communication interface  1414  provides two-way communication between computer systems  1400 . The communication interface  1414  of a respective computer system  1400  transmits and receives electrical, electromagnetic or optical signals, that include data streams representing various types of signal information, e.g., instructions, messages and data. A communication link  1415  links one computer system  1400  with another computer system  1400 . For example, the communication link  1415  may be the internet in which case the communication interface  1414  may be a telephone line, a cable or a wireless modem, or the communication link  1415  may be a LAN, in which case the communication interface  1414  may be a LAN card, or the communication link  1415  may be a PSTN, in which case the communication interface  1414  may be an integrated services digital network (ISDN) card or a modem.  
         [0037]     A computer system  1400  may transmit and receive messages, data, and instructions, including program, i.e., application, code, through its respective communication link  1415  and communication interface  1414 . Received program code may be executed by the respective processor(s)  1407  as it is received, and/or stored in the storage device  1410 , or other associated non-volatile media, for later execution.  
         [0038]     In an embodiment, the computer system  1400  operates in conjunction with a data storage system  1431 , e.g., a data storage system  1431  that contains a database  1432  that is readily accessible by the computer system  1400 . The computer system  1400  communicates with the data storage system  1431  through a data interface  1433 . A data interface  1433 , which is coupled to the bus  1406 , transmits and receives electrical, electromagnetic or optical signals, that include data streams representing various types of signal information, e.g., instructions, messages and data. In embodiments of the invention, the functions of the data interface  1433  may be performed by the communication interface  1414 .  
         [0039]     Computer system  1400  includes a bus  1406  or other communication mechanism for communicating instructions, messages and data, collectively, information, and one or more processors  1407  coupled with the bus  1406  for processing information. Computer system  1400  also includes a main memory  1408 , such as a random access memory (RAM) or other dynamic storage device, coupled to the bus  1406  for storing dynamic data and instructions to be executed by the processor(s)  1407 . The main memory  1408  also may be used for storing temporary data, i.e., variables, or other intermediate information during execution of instructions by the processor(s)  1407 .  
         [0040]     The computer system  1400  may further include a read only memory (ROM)  1409  or other static storage device coupled to the bus  1406  for storing static data and instructions for the processor(s)  1407 . A storage device  1410 , such as a magnetic disk or optical disk, may also be provided and coupled to the bus  1406  for storing data and instructions for the processor(s)  1407 .  
         [0041]     A computer system  1400  may be coupled via the bus  1406  to a display device  1411 , such as, but not limited to, a cathode ray tube (CRT), for displaying information to a user. An input device  1412 , e.g., alphanumeric and other keys, is coupled to the bus  1406  for communicating information and command selections to the processor(s)  1407 .  
         [0042]     According to one embodiment of the invention, an individual computer system  1400  performs specific operations by their respective processor(s)  1407  executing one or more sequences of one or more instructions contained in the main memory  1408 . Such instructions may be read into the main memory  1408  from another computer-usable medium, such as the ROM  1409  or the storage device  1410 . Execution of the sequences of instructions contained in the main memory  1408  causes the processor(s)  1407  to perform the processes described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and/or software.  
         [0043]     The term “computer-usable medium,” as used herein, refers to any medium that provides information or is usable by the processor(s)  1407 . Such a medium may take many forms, including, but not limited to, non-volatile, volatile and transmission media. Non-volatile media, i.e., media that can retain information in the absence of power, includes the ROM  1409 , CD ROM, magnetic tape, and magnetic discs. Volatile media, i.e., media that can not retain information in the absence of power, includes the main memory  1408 . Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise the bus  1406 . Transmission media can also take the form of carrier waves; i.e., electromagnetic waves that can be modulated, as in frequency, amplitude or phase, to transmit information signals. Additionally, transmission media can take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.  
         [0044]     In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. For example, the reader is to understand that the specific ordering and combination of process actions shown in the process flow diagrams described herein is merely illustrative, and the invention can be performed using different or additional process actions, or a different combination or ordering of process actions. The specification and drawings are, accordingly, to be regarded in an illustrative rather than restrictive sense.