Patent ID: 9575990
Date: 2017-02-21
CPC Classifications: G06F

Claim:
1. A non-transitory computer program product storing instructions that, when executed by at least one programmable processor forming part of at least one computing system, cause the at least one programmable processor to perform operations comprising: partitioning data in a file, stored on a source node of a distributed data storage system comprising a plurality of nodes including the source node and a destination node, into a source virtual file corresponding to one or more source partitions on the source node and a destination virtual file corresponding to one or more destination partitions on the destination node; generating, on the destination node, a destination record within the destination virtual file, the destination record comprising (i) a link directed to the source virtual file stored on the source node and (ii) partition criteria characterizing the one or more source partitions corresponding to the source virtual file, the source virtual file being mapped to a chain of linked pages stored in a page buffer of the distributed data storage system, the page buffer being a temporary persistency layer; appending, subsequent to the generation of the destination record, data generated at the destination node to the destination record, the appended data forming a delta log at the destination node; receiving, at the destination node from a requesting device, a request to access data defined by the destination virtual file; determining, at the destination node using the destination virtual file, that the data, requested by the requesting device, is stored in the one or more source partitions corresponding to the source virtual file; providing the data, requested by the requesting device, from the one or more source partitions corresponding to the source virtual file stored on the source node to the requesting device through the destination node using the link and the partition criteria; initiating a columnar table merge operation; overwriting, as part of the columnar table merge operation, (i) the link, in the destination record, directed to the source virtual file stored on the source node, and (ii) the destination virtual file and the delta log with a new version of the destination virtual file comprising data from the one or more source partitions corresponding to the source virtual file and the delta log; persisting, as part of the columnar table merge operation, the destination virtual file to a secondary storage; and dropping, from a logging schedule and as part of the columnar table merge operation, the source virtual file stored on the source node.