Multiphase topology-wide code modifications for peer-to-peer systems

Systems and methods that supply changes on-the-fly and without breaking execution of codes for schema changes in a peer-to-peer environment. The modification component supplies changes to peers or nodes in a topology by initially identifying dependencies (e.g., multiple level) that can be affected by the schema change, followed by performing the schema change to the codes or stored procedures. Accordingly, dependencies in an entire system can be refreshed, wherein multiple levels of dependencies can exist.

BACKGROUND

Advances in computer technology (e.g., microprocessor speed, memory capacity, data transfer bandwidth, software functionality, and the like) have generally contributed to increased computer application in various industries. Ever more powerful server systems, which are often configured as an array of servers, are commonly provided to service requests originating from external sources such as the World Wide Web, for example.

As the amount of available electronic data grows, it becomes more important to store such data in a manageable manner that facilitates user friendly and quick data searches and retrieval. Today, a common approach is to store electronic data in one or more databases. In general, a typical database can be referred to as an organized collection of information with data structured such that a computer program can quickly search and select desired pieces of data, for example. Commonly, data within a database is organized via one or more tables. Such tables are arranged as an array of rows and columns.

Also, the tables can comprise a set of records, wherein a record includes a set of fields. Records are commonly indexed as rows within a table and the record fields are typically indexed as columns, such that a row/column pair of indices can reference particular datum within a table. For example, a row can store a complete data record relating to a sales transaction, a person, or a project. Likewise, columns of the table can define discrete portions of the rows that have the same general data format, wherein the columns can define fields of the records.

Database applications allow the user to compare, sort, order, merge, separate and interconnect the data, so that useful information can be generated from the data. Capacity and versatility of databases have grown incredibly to allow virtually endless storage capacity utilizing databases. Such database systems can become complex to manage, wherein substantial investment of time of a skilled administrator is typically required.

For example, in a peer-to-peer replication topology based on transactional replication in the publisher-subscriber model, it becomes challenging to provide a low-impact methodology to make topology-wide configurations, including conflict detection configurations such as enabling and disabling, and schema operations such as column adding and column dropping.

SUMMARY

The following presents a simplified summary in order to provide a basic understanding of some aspects described herein. This summary is not an extensive overview of the claimed subject matter. It is intended to neither identify key or critical elements of the claimed subject matter nor delineate the scope thereof. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.

The subject innovation provides for database schema changes (e.g., dropping columns from definition tables) without breaking execution of codes or stored procedures that are associated with such schema changes, via a modification component in a distributed environment. Such a modification component can supply changes to peers or nodes in a topology by initially identifying dependencies (e.g., multiple level), which can be affected by the schema change, followed by performing the schema change to the codes or stored procedures. Accordingly, dependencies in entire system can be refreshed, wherein multiple levels of dependencies can exist. For example, stored procedures are modified not to depend on dropped columns and the columns are then dropped from a table definition. Hence, the subject innovation facilitates topology wide configurations including conflict detection configurations such as enabling/disabling and schema operations (e.g., column adding/dropping, conflict detection and the like.)

According to an exemplary methodology of the subject innovation, initially a column that is to be dropped can be designated. Subsequently, by verifying associated dependencies, stored procedures that refer to and/or depend on columns that are to be dropped are identified. A refreshing act can then be performed on such stored procedures, so that they no longer refer to columns that are to be dropped.

Such dependencies can be propagated throughout the topology, wherein every peer is notified of such dependency change and not referred to a column that has been dropped. As such and from a user's perspective, columns can be dropped from a table without manual intervention or without shutting down the system.

In a related aspect, when a user drops a table or a column, such request can be detected and the code generated therefrom is identified. Such code can then be propagated throughout the system to refresh dependencies, so that the code now remains independent of the changes. The codes can also be written to each peer's transactional language. Subsequently, such codes are carried in the proper order and applied to other peers in the system. Hence, changes can be made to the system on-the-fly, to mitigate a requirement of halting activities or execution of the code (e.g., without affecting execution of stored procedures) or shutting the system.

DETAILED DESCRIPTION

FIG. 1illustrates a block diagram of a system100that provides for database schema change (e.g., dropping columns from definition tables) without breaking execution of codes or stored procedures that are associated with such schema change, via modification component in a distributed environment. In general, the peer to peer network100is a network of nodes104,106,108(1 to n, n being an integer) (e.g., databases) that employ a same program or type of program to communicate and share data therebetween. For example, each computer or peer (e.g., node)104,106,108can be considered equal in terms of responsibilities and acts as a server to others in a network. In such topology, it can be assumed that one peer at a time is issuing a topology-wide configuration.

In one aspect, in one particular implementation of a peer to peer network, a list of nodes and data stored on the nodes104,106,108can be maintained, to facilitate nodes joining or disjoining the network at will. Upon joining, the nodes104,106,108provide a list of files they are capable of serving, wherein such list of files can be added to a list of the available files for all nodes. Upon disjoining by any node, the list of files are removed from the list of available files for all the nodes. Moreover, a requesting node (e.g., user) can request a file by sending a request. For example, the requesting node can further obtain the file via a direct connection to one of the nodes able to satisfy the request. As illustrated inFIG. 1, the modification component110can supply changes to peers or nodes in the topology130by initially identifying dependencies (e.g., multiple level) that can be affected by the schema change, followed by performing the schema change to the codes or stored procedures.

FIG. 2illustrates a further exemplary aspect of the subject innovation, wherein the data distribution system200includes a host computer202, a bulletin board204, an access layer206and a plurality of nodes208. The modification component210facilitates topology wide configurations including conflict detection configurations such as enabling/disabling and schema operations such as column adding/dropping, conflict detection and the like. The data distribution system200is a peer to peer network and thus, permits data content to be requested and transferred to and from the plurality of nodes208. The system200typically does not centrally maintain a list of available files and nodes storing those files, and instead employs the bulletin board204to maintain a list of requests or requested files. Furthermore, the system200is not required to track when the nodes enter and leave (e.g., join and un-join) the system200. Additionally, the system200employs the access layer206to track and enforce fairness such that the nodes208properly contribute to the system200(e.g., sending files) in order to benefit from the system (e.g., receiving files).

The plurality of nodes208are operative to make requests for data, fill or serve requests for data and register periodically at a check in frequency. The plurality of nodes208are typically software components operating or executing on a computer system or server system. The nodes208are able to communicate and transfer data with each other, the bulletin board204and the access layer206, generally via a network connection (e.g., Internet, wireless network, local area network, wide area network and the like) employing established protocols.

The host computer202controls and operates the bulletin board204and the access layer206. The host computer202can be accessed by a user or operator in order to modify control and operation of the bulletin board204and the access layer206. The user or operator can access the host computer by an input device (e.g., keyboard) of the host computer or remotely via a network connection.

The bulletin board204can maintain a list of requests that are in a predefined machine readable format. The requests in the list respectively include requested data (e.g., a file, an item or portion of a file) and a pointer to a requesting node (e.g., the node requesting the data). The pointer can simply be a network address, such as an IP address and port number, of the requesting node or other indicator such that the requesting node is identifiable and contactable by other nodes of the plurality of nodes208. Additionally, the requests can include other information, such as, priority, desired time out period, download bandwidth and the like. Thus, the bulletin board204can remove un-filled requests after a time out period (e.g., after an age of the request exceeds the time out period). The time out period is typically selected to remove un-filled requests that have a low likelihood of being filled or served.

The access layer206is a mechanism for the host computer202to regulate access to and/or ability of the nodes208to make additional requests based on behavior or performance (e.g., served data). The access layer206tracks performance or behavior of the nodes208by tracking requests made, requests filled, checking in and the like of the nodes208. The access layer206can prevent nodes from making requests for data if the nodes fail to meet a performance or behavior threshold. One approach for the access layer206to track performance is to compute a performance ratio of requests satisfied versus requests made over time.

As explained earlier, the modification component210can supply changes to peers or nodes in such a topology by initially identifying dependencies (e.g., multiple level), followed by performing the schema change to the codes or stored procedures. Accordingly, dependencies in entire system can be refreshed, wherein multiple levels of the dependencies can exist. For example, stored procedures, as described in detail infra, are modified not to depend on dropped columns and the columns are then dropped from a table definition. Hence, the subject innovation facilitates topology wide configurations including conflict detection configurations such as enabling/disabling and schema operations such as column adding/dropping, conflict detection and the like.

FIG. 3illustrates a further exemplary aspect of the subject innovation wherein a peer(s) or node(s) is in the form of a database with codes that are to be modified. Such codes316can further include a plurality of stored procedures, wherein such stored procedures304,306,308(1 through n, where n is an integer) are programs (or procedures) that are physically stored within the database associated with the data storage system330. Such programs are typically written in a proprietary database language and in response to a user request, run directly by an engine of the database associated with the data storage system330.

The stored procedures304,306,308can have direct access to the data that requires manipulation, and typically need only to send results back to the user, thus mitigating the overhead of communicating large amounts of data back and forth. For example, typical uses for stored procedures304,306,308can include data validation, which is integrated into the database structure (stored procedures used for this purpose are often called triggers), or encapsulating some large or complex processing (such as manipulating a large data set to produce a summarized result). Stored procedures304,306,308can also be employed when the database associated with the data storage system330is manipulated from many external programs.

When a user drops a table or a column, such a request can be detected by the modification component310and a code generated therefrom. Such code can then be propagated throughout the system to refresh dependencies, so that the code now remains independent of the changes. The codes can also be written to each peer transactional language. Subsequently, such codes are carried in the proper order and applied to other peers in the system. Hence, changes can be made to the system on-the-fly and such mitigates a requirement of halting activities or execution of the code (e.g., without affecting execution of stored procedures) or shutting the system, which can be cumbersome

Furthermore, the data storage system330can be a complex model based database structure, wherein an item, a sub-item, a property, and a relationship are defined to allow representation of information within a data storage system as instances of complex types. For example, the data storage system330can employ a set of basic building blocks for creating and managing rich, persisted objects and links between objects. An item can be defined as the smallest unit of consistency within the data storage system330, which can be independently secured, serialized, synchronized, copied, backup/restored, and the like. Such an item can include an instance of a type, wherein all items in the data storage system330can be stored in a single global extent of items. Furthermore, the data storage system330can be based upon at least one item and/or a container structure, and can act as a storage platform exposing rich metadata that is buried in files as items. The data storage system330can include an associated database (not shown), to support the above discussed functionality, wherein any suitable characteristics and/or attributes can be implemented. Furthermore, the data storage system330can employ a container hierarchical structure, wherein a container is an item that can contain at least one other item. Such a containment concept can be implemented via a container ID property inside the associated class, wherein the store can also be a container in the form of a physical organizational and manageability unit. In addition, the store represents a root container for a tree of containers within the hierarchical structure. As such, and from a user's perspective, columns can be dropped from a table without manual intervention or without shutting down the system.

FIGS. 4-7illustrate a particular aspect of the subject innovation that modifies codes in a database via a modification component.FIG. 4illustrates a system400with databases (e.g., peers) A, B, C, and each database includes tables associated therewith, wherein the peers share a same schema. Accordingly, API procedures can write API procedures to Ta and indicate associated dependencies. Changes to the data and the schema can propagate through database A, database B, and database C. Such illustrates a topology, with three peers: A, B and C, which have tables Ta, Tb and Tc, respectively, with the same schema. The proc Pxy (x, y=a, b or c, x <>y) is called by peer X (A, B or C) to replicate data from X to a destination peer Y (A, B or C). The definition of Pxy depends on the schema Ty. Thus the schema change to Ty may need to change Pxy first, otherwise, the execution of Pxy may break. For example, when a column is dropped from Ty, then Pxy is required to be changed beforehand. On the other hand, when a column is added into Ty, Pxy can remain unchanged without breaking its execution, assuming the column to be added allows null values.

Moreover, conflict detection configuration can incur schema changes. In addition, enabling conflict detection results in a hidden column being added to Ty, and Pxy can remain unchanged without breaking its execution—(however, Pxy is required to be changed in order to make conflict detection effective). Disabling conflict detection results in the hidden column being dropped from Ty, and Pxy should typically be changed beforehand in order not to break execution of Pxy, because the existing definition of Pxy depends on the existence of the hidden column. The modification component enables dropping a column from the table for all peers without breaking any execution of stored procedures, and without quiescing.

FIG. 5illustrates dependencies between stored procedures and the tables. As illustrated inFIG. 5, two arrows510,512directed to Ta indicate that stored procedures Pba and Pca depend on Ta. Before a column is dropped from Ta, Pba and Pca must typically be changed to remove the dependency on the column to be dropped. If not, concurrent execution of Pba and Pca will break. Such is illustrated as dependency lines510,512from Pba and Pca to Ta, and similarly514,516for others.

As illustrated inFIG. 6, when an attempt is made to change Pba, there exists a possibility that peer B is also calling Pba to replicate data from B to A concurrently. Accordingly, B is instructed to change Pba, because B knows when it is not calling Pba.FIG. 6illustrates such dependency as a line from B to Pba, and similarly for others. Likewise,FIG. 7illustrates a comprehensive dependency graph forFIG. 5andFIG. 6, which forms the basis for dropping a column for all peers without quiescing.

Accordingly, the modification component of the subject innovation can implement a two phase topology-wide operation, for example. During the first phase,. any peer in the topology can broadcast a request to require all peers to send a command to their subscribers to change stored procedures employed thereby. Put differently, A is requested to send a command to B to change stored procedure Pab, and a command to C to change stored procedure Pac, and A does not acknowledge to the request initiator until all its subscriber's stored procedures Pab and Pac are changed. During the second phase, any peer in topology broadcasts a request to require all peers to drop the column. Put differently, A is requested to drop the column for table Ta. Moreover, in phase1, “send a command” is implemented by putting the command into the log. Thus, the command is replicated and there are no concurrent calls to the stored procedures to be changed from the “sender”. Phase2is not started until phase1has finished successfully. It is to be appreciated, that the failures could occur during phase1and phase2. The operations at each phase are made idempotent, so that when a phase is partially-finished due to failures of some peers, such phase can typically be re-started until this phase is finished completely and successfully. If the column to be dropped does not allow null values by definition, another phase can be inserted before phase1, to have all peers change this column to allow null values.

FIG. 8illustrates a related methodology800of modifying a code and dropping columns according to an aspect of the subject innovation. Initially at810, a column that is to be dropped can be identified from a peer and/or data. Subsequently at820, associated dependencies for stored procedures that refer to and/or depend on columns that are to be dropped, are verified. At830, a refreshing act can then be performed on such stored procedures, so that they no longer refer to columns that are to be dropped. Such dependencies can be propagated at840throughout the topology, wherein every peer is notified of such dependency change and not refer to a column that has been dropped. As such and from a user's perspective, columns can be dropped from a table without manual intervention or without shutting down the system.

FIG. 9illustrates a related methodology900, when a user drops a table or a column, such request can be detected at910and a code generated therefrom at920. Such code can then be propagated throughout the system to refresh dependencies, so that the code now remains independent of the changes. The codes can also be written to each peer transactional language at930. Subsequently, such codes are carried in the proper order and applied to other peers in the system. Hence, changes can be made to the system on-the-fly at940and such mitigates a requirement of halting activities or execution of the code (e.g., without affecting execution of stored procedures) or shutting the system, which can be cumbersome.

FIG. 10illustrates a peer to peer system1000that employs an artificial intelligence component1030, which interacts with the modification component of the subject innovation. Such artificial intelligence (AI) component1030can be employed to facilitate inferring and/or determining when, where, and how to dynamically modify codes on the fly when a code is to be changed in the peer to peer system. As used herein, the term “inference” refers generally to the process of reasoning about or inferring states of the system, environment, and/or user from a set of observations as captured via events and/or data. Inference can be employed to identify a specific context or action, or can generate a probability distribution over states, for example. The inference can be probabilistic—that is, the computation of a probability distribution over states of interest based on a consideration of data and events. Inference can also refer to techniques employed for composing higher-level events from a set of events and/or data. Such inference results in the construction of new events or actions from a set of observed events and/or stored event data, whether or not the events are correlated in close temporal proximity, and whether the events and data come from one or several events and data sources.

The AI component1030can employ any of a variety of suitable AI-based schemes as described supra in connection with facilitating various aspects of the herein described invention. For example, a process for learning explicitly or implicitly how a code or stored procedure is to be trusted can be facilitated via an automatic classification system and process. Classification can employ a probabilistic and/or statistical-based analysis (e.g., factoring into the analysis utilities and costs) to prognose or infer an action that a user desires to be automatically performed. For example, a support vector machine (SVM) classifier can be employed. Other classification approaches include Bayesian networks, decision trees, and probabilistic classification models providing different patterns of independence can be employed. Classification as used herein also is inclusive of statistical regression that is utilized to develop models of priority.

As will be readily appreciated from the subject specification, the subject innovation can employ classifiers that are explicitly trained (e.g., via a generic training data) as well as implicitly trained (e.g., via observing user behavior and/or receiving extrinsic information) so that the classifier is used to automatically determine according to a predetermined criteria which return an answer to a question. For example, with respect to SVM's that are well understood, SVM's are configured via a learning or training phase within a classifier constructor and feature selection module. A classifier is a function that maps an input attribute vector, x =(x1, x2, x3, x4, xn), to a confidence that the input belongs to a class—that is, f(x)=confidence(class).

The word “exemplary” is used herein to mean serving as an example, instance or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Similarly, examples are provided herein solely for purposes of clarity and understanding and are not meant to limit the subject innovation or portion thereof in any manner. It is to be appreciated that a myriad of additional or alternate examples could have been presented, but have been omitted for purposes of brevity.

In order to provide a context for the various aspects of the disclosed subject matter,FIGS. 11 and 12as well as the following discussion are intended to provide a brief, general description of a suitable environment in which the various aspects of the disclosed subject matter may be implemented. While the subject matter has been described above in the general context of computer-executable instructions of a computer program that runs on a computer and/or computers, those skilled in the art will recognize that the innovation also may be implemented in combination with other program modules. Generally, program modules include routines, programs, components, data structures, and the like, which perform particular tasks and/or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the innovative methods can be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, mini-computing devices, mainframe computers, as well as personal computers, hand-held computing devices (e.g., personal digital assistant (PDA), phone, watch . . . ), microprocessor-based or programmable consumer or industrial electronics, and the like. The illustrated aspects may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. However, some, if not all aspects of the innovation can be practiced on stand-alone computers. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

With reference toFIG. 11, an exemplary environment1110for implementing various aspects of the subject innovation is described that includes a computer1112. The computer1112includes a processing unit1114, a system memory1116, and a system bus1118. The system bus1118couples system components including, but not limited to, the system memory1116to the processing unit1114. The processing unit1114can be any of various available processors. Dual microprocessors and other multiprocessor architectures also can be employed as the processing unit1114.

The system memory1116includes volatile memory1120and nonvolatile memory1122. The basic input/output system (BIOS), containing the basic routines to transfer information between elements within the computer1112, such as during start-up, is stored in nonvolatile memory1122. By way of illustration, and not limitation, nonvolatile memory1122can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory1120includes random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM).

It is to be appreciated thatFIG. 11describes software that acts as an intermediary between users and the basic computer resources described in suitable operating environment1110. Such software includes an operating system1128. Operating system1128, which can be stored on disk storage1124, acts to control and allocate resources of the computer system1112. System applications1130take advantage of the management of resources by operating system1128through program modules1132and program data1134stored either in system memory1116or on disk storage1124. It is to be appreciated that various components described herein can be implemented with various operating systems or combinations of operating systems.

A user enters commands or information into the computer1112through input device(s)1136. Input devices1136include, but are not limited to, a pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, web camera, and the like. These and other input devices connect to the processing unit1114through the system bus1118via interface port(s)1138. Interface port(s)1138include, for example, a serial port, a parallel port, a game port, and a universal serial bus (USB) port. Output device(s)1140use some of the same type of ports as input device(s)1136. Thus, for example, a USB port may be used to provide input to computer1112, and to output information from computer1112to an output device1140. Output adapter1142is provided to illustrate that there are some output devices1140like monitors, speakers, and printers, among other output devices1140that require special adapters. The output adapters1142include, by way of illustration and not limitation, video and sound cards that provide a means of connection between the output device1140and the system bus1118. It should be noted that other devices and/or systems of devices provide both input and output capabilities such as remote computer(s)1144.

Computer1112can operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s)1144. The remote computer(s)1144can be a personal computer, a server, a router, a network PC, a workstation, a microprocessor based appliance, a peer device or other common network node and the like, and typically includes many or all of the elements described relative to computer1112. For purposes of brevity, only a memory storage device1146is illustrated with remote computer(s)1144. Remote computer(s)1144is logically connected to computer1112through a network interface1148and then physically connected via communication connection1150. Network interface1148encompasses communication networks such as local-area networks (LAN) and wide-area networks (WAN). LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet/IEEE 802.3, Token Ring/IEEE 802.5 and the like. WAN technologies include, but are not limited to, point-to-point links, circuit switching networks like Integrated Services Digital Networks (ISDN) and variations thereon, packet switching networks, and Digital Subscriber Lines (DSL).

Communication connection(s)1150refers to the hardware/software employed to connect the network interface1148to the bus1118. While communication connection1150is shown for illustrative clarity inside computer1112, it can also be external to computer1112. The hardware/software necessary for connection to the network interface1148includes, for exemplary purposes only, internal and external technologies such as, modems including regular telephone grade modems, cable modems and DSL modems, ISDN adapters, and Ethernet cards.

FIG. 12is a schematic block diagram of a sample-computing environment1200that can be employed for implementing data retrieval, in accordance with an aspect of the subject innovation. The system1200includes one or more client(s)1210. The client(s)1210can be hardware and/or software (e.g., threads, processes, computing devices). The system1200also includes one or more server(s)1230. The server(s)1230can also be hardware and/or software (e.g., threads, processes, computing devices). The servers1230can house threads to perform transformations by employing the components described herein, for example. One possible communication between a client1210and a server1230may be in the form of a data packet adapted to be transmitted between two or more computer processes. The system1200includes a communication framework1250that can be employed to facilitate communications between the client(s)1210and the server(s)1230. The client(s)1210are operatively connected to one or more client data store(s)1260that can be employed to store information local to the client(s)1210. Similarly, the server(s)1230are operatively connected to one or more server data store(s)1240that can be employed to store information local to the servers1230.

What has been described above includes various exemplary aspects. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing these aspects, but one of ordinary skill in the art may recognize that many further combinations and permutations are possible. Accordingly, the aspects described herein are intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims.

Furthermore, to the extent that the term “includes” is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.