Patent ID: 7809774

Claim:
A distributed file system comprising: a plurality of compute nodes; and a plurality of input/output (I/O) nodes, the I/O nodes and the compute nodes being connected by an interconnection network; wherein the distributed file system includes a common data representation for physical and logical partitions of a file stored in the system, and elements of the physical and logical partitions are linearly addressable, so that each compute node is programmed to perform a plurality of file operations, including: computing a mapping between a linear space of a file and a linear space of a subfile; computing mappings between a linear space of a view and linear spaces of subfiles; performing an intersection algorithm between the view and the subfile; and performing a data operation; wherein a Processor Index Tagged FAmiLy of Line Segments (PITFALLS) is used for data representation representing a set of equally spaced FAmiLy of Line Segments (FALLS) forming outer FALLS, wherein each outer FALLS contains a set of inner PITFALLS forming nested PITFALLS for representing a physical partitioning of a file onto I/O nodes, a logical partitioning of a file onto compute nodes and mappings between them, so as to allow for arbitrarily partitioning data on available storage media and among several compute nodes, respectively.