Patent Description:
Network storage systems have developed in response to the increasing proliferation of data requirements. Network storage systems generally focus on the storage, protection and retrieval of data in large-scale environments. Many corporations and organizations have large sets of electronic content such as files to be stored and maintained. As time passes, these sets of content tend to grow, and ultimately reach a size which is often too great for a single repository. Nonetheless, the organization needs to manage this content in a uniform way, even if the content is spread across several physical stores. Managing such electronic content may present challenges since the size of a storage system containing the electronic content may be limited. As such, the storage system may need to be scaled to expand the storage capabilities. However, conventional scaling techniques may suffer from a number of disadvantages bound by hardware limitations (processor, memory, storage limitations, etc.).

In light of the above, it would be desirable to arrive at an approach to a storage system that may be scalable compared to conventional approaches. Thus, there is a need in the art for improvements in the scaling of storage systems across one or more storage devices. <NPL>, presents a metadata management technique, Directory Path Code Hash. This technique is to store directory and file metadata separately, and effectively solving the unbalanced metadata distribution and access hot point problems in Sub-tree partition and the excessive reading times and large metadata migration amount after directory property modification in hash algorithm.

It is the object of the present invention to improve scalability of prior art systems.

The present disclosure provides a method of deleting an entry from a storage system having a three tier volume storage structure, wherein at a first tier, a directory volume comprises directory metadata, at a second tier, a file volume comprises file metadata, and at a third tier, a shard volume comprises shard metadata and one or more shards storing file data, the directory metadata being used for accessing the file metadata at the file volume and the file metadata being used for accessing the one or more shards storing the file data. The method comprises receiving, at the storage system, a deletion request for the entry, the entry corresponding to either a file or a directory in the storage system, determining directory metadata from a directory volume, the directory metadata associated with the entry, and determining whether the entry corresponds to the file or the directory based on the directory metadata. The method further comprises, based on a determination that the entry corresponds to the directory, requesting deletion of the directory metadata associated with the entry from the directory volume, and, based on a determination that the entry corresponds to the file, requesting deletion of the directory metadata associated with the entry from the directory volume. The method further comprises determining, based on the directory metadata, a file volume containing file metadata for the file, the file metadata comprising a shard volume identifier identifying a shard volume where one or more shards are located and a shard identifier identifying the one or more shards within the shard volume that contain the file, requesting deletion of the file metadata from the file volume, requesting deletion of the shard identifier from the shard volume, and requesting deletion of the file from the file volume.

The present disclosure provides a method of adding an entry into a storage system having a three tier volume storage structure, wherein at a first tier, a directory volume comprises directory metadata, at a second tier, a file volume comprises file metadata, and at a third tier, a shard volume comprises shard metadata and one or more shards storing file data, the directory metadata being used for accessing the file metadata at the file volume and the file metadata being used for accessing the one or more shards storing the file data. The method comprises receiving, at the storage system, a request for adding the entry, determining that directory metadata associated with the entry does not exist in a directory volume, and determining whether the entry corresponds to a file or a directory. The method further comprises, based on a determination that the entry corresponds to the directory, generating the directory metadata associated with the directory in the directory volume, and based on a determination that the entry corresponds to the file, determining to add the file to a file volume. The method further comprises allocating file metadata in the file volume, the file metadata associated with the file, determining a shard volume to store data for the file as one or more shards, adding the data for the file to the shard volume as one or more shards, and generating the directory metadata associated with the file in the directory volume, the directory metadata comprising an identifier of the file metadata, wherein the file metadata comprises a shard volume identifier identifying the shard volume, and a shard identifier associated with each of the one or more shards and identifying the file within the shard volume.

The present disclosure provides a storage system having a three tier volume storage structure, wherein at a first tier, a directory volume comprises directory metadata, at a second tier, a file volume comprises file metadata, and at a third tier, a shard volume comprises shard metadata and one or more shards storing file data, the directory metadata being used for accessing the file metadata at the file volume and the file metadata being used for accessing the one or more shards storing the file data. The storage system comprises a memory configured to store data and at least one processor in communication with the memory. The at least one processor is configured to receive a deletion request for the entry, the entry corresponding to either a file or a directory in the storage system, determine directory metadata from a directory volume, the directory metadata associated with the entry, and determine whether the entry corresponds to the file or the directory based on the directory metadata. The at least one processor is further configured to, based on a determination that the entry corresponds to the directory, request deletion of the directory metadata associated with the entry from the directory volume, and, based on a determination that the entry corresponds to the file, request deletion of the directory metadata associated with the entry from the directory volume. The at least one processor is further configured to determine, based on the directory metadata, a file volume containing file metadata for the file, the file metadata comprising a shard volume identifier identifying a shard volume where one or more shards are located and a shard identifier identifying the one or more shards within the shard volume that contain the file, request deletion of the file metadata from the file volume, request deletion of the shard identifier from the shard volume, and request deletion of the file from the file volume.

Additional advantages and novel features relating to implementations of the present disclosure will be set forth in part in the description that follows, and in part will become more apparent to those skilled in the art upon examination of the following or upon learning by practice thereof.

The present disclosure will be better understood from the following detailed description read in light of the accompanying drawings, wherein like reference numerals are used to designate like parts in the accompanying description:.

The present disclosure relates to a scalable multi-tier storage structure for one or more computer devices and techniques for accessing data within the multi-tier storage structure. Specifically, organizations may rely on gathering, storing, and interpreting large quantities of information. The information may be stored, for example, in file storage systems. According to particular applications, file storage systems may have various needs such as scalability. Scalability may include the ability to expand the capabilities of a storage system. For example, it may be desirable to increase the amount of files that can be stored in a system. As another example, it may be desirable to increase the speed at which files may be accessed and/or the number of users that may simultaneously access stored files.

Current implementations may have drawbacks related to scalability. In particular, in order to scale up any one particular aspect of the system an entire computer device (e.g., server machine) may have to be added. However, the addition of such a file server device may not be the best use of resources. For instance, if a file server machine is added to service more requests, its underlying storage may be underutilized. On the other hand, if a file server device is added only for increased storage, the server process may be idle most of the time. Further, changes in a server process may have to be implemented on all machines simultaneously. As such, changes to a system are essentially monolithic (e.g., the addition of one or more file servers). Accordingly, it would be desirable to provide an approach to a storage system that may have more scalable components.

The present implementations provide a scalable multi-tier storage structure. Specifically, each tier in the multi-tier structure scales independently to meet the storage/capacity requirements associated with the respective tier. Each tier, or volume which is storage space or area configured to store distinct file-specific information is arranged in a hierarchical structure, with a top level tier corresponding to at least one directory volume, which stores and handles high level data such as a file name associated with a file. A subsequently hierarchical tier corresponding to at least one file volume stores and handles file metadata associated with the file. A lowest hierarchical tier corresponding to at least one shard volume stores the data associated with the file in one or more shards. Each of the aforementioned volumes may form a single cluster, whereby the present implementations provide scaling across multiple storage clusters. That is, each of the volumes scales independently of each other. Further, the scalable multi-tier storage structure permits various file manipulation operations at the volumes.

In an embodiment, a storage system receives a deletion request for the entry, the entry corresponding to either a file or a directory in the storage system. The storage system further determines directory metadata from a directory volume, the directory metadata associated with the entry. The storage system additionally determines whether the entry corresponds to the file or the directory based on the directory metadata. Moreover, the storage system requests deletion of the directory metadata associated with the entry from the directory volume based on a determination that the entry corresponds to the directory. The storage system further requestsdeletion of the directory metadata associated with the entry from the directory volume and requests deletion of the file from the file volume based on a determination that the entry corresponds to the file.

In another embodiment the, storage system receives, at the storage system, a request for adding the entry. The storage system determines that directory metadata associated with the entry does not exist in a directory volume. The storage system further determines whether the entry corresponds to the file or the directory. The storage system determines whether the entry corresponds to the file or the directory. The storage system generates the directory metadata associated with the entry in the directory volume based on a determination that the entry corresponds to the directory. The storage system determines to add the file to a file volume, allocates file metadata in the file volume, the file metadata associated with the file, generates the directory metadata associated with the entry in the directory volume based on a determination that the entry corresponds to the file.

Referring now to <FIG>, a block diagram of a data network <NUM> may include a client device <NUM> in communication with a network file management system <NUM>. The client device <NUM> may execute the network file management client <NUM> to connect to a network file management system <NUM> via a data network connection <NUM>. The network file management client <NUM> may be a separate application or integrated into an operating system or an internet browser platform at the client device <NUM>. The network file management system <NUM> may refer to a single server or a distributed set of servers that may access a cloud data set, such as a server farm. The data network connection <NUM> may be an internet connection, a wide area network connection, a local area network connection, or other type of data network connections. The network file management client <NUM> may access and/or manipulate one or more data files <NUM> stored in the network file management system <NUM> according to a multi-tier volume storage structure that stores directory metadata <NUM>, file metadata <NUM>, and one or more shards <NUM> in separate or distinct volumes. In some implementations, a volume may correspond to storage space storing a specific type or types of data.

For example, the network file management client <NUM> may allow a user to access one or more data files <NUM> stored at the network file management system <NUM> and/or perform a read operation or a write operation on the one or more data files <NUM> over a network. For example, a read operation may send a copy of the data stored in the one or more data files <NUM> to the client device <NUM>. A write operation may create or delete the one or more data file <NUM> and/or enters changes to the one or more data file <NUM> as stored in the network file management system <NUM>.

The network file management system <NUM> includes a multi-tier volume storage structure that permits independent scaling of individual tiers. The individual tiers store file-specific information unique to the individual tier, thereby allowing seamless scaling or additions to the individual tier as dictated at least by load requirements. The network file management system <NUM> includes one or more directory volumes <NUM>, one or more file volumes, and/or one or shard volumes <NUM>, each storing distinct file information in a partitioned manner. In particular, the one or more directory volumes <NUM> store directory metadata <NUM> associated with the one or more data files <NUM>. The one or more file volumes <NUM> store file metadata <NUM> associated with the one or more data files <NUM>. The one or more shard volumes <NUM> store one or more shards <NUM> or file partitions that each contain at least a portion of the one or more data files <NUM>.

The network file management system <NUM> includes a multi-tier file partitioning component <NUM>, which is configured to store the one or more data files <NUM> within the one or more directory volumes <NUM>, one or more file volumes, and/or one or shard volumes <NUM> according to the multi-tier volume storage structure. Specifically, the multi-tier volume storage structure stores the one or more data files <NUM> as partitions such that the directory metadata <NUM> and file metadata <NUM> are partitioned, and the file content or data is sharded. In some implementations, sharding file content or data may correspond to dividing or partitioning file content or data into discrete portions or shards <NUM> according to a shard size. The directory metadata <NUM> and file metadata <NUM> is stored in the one or more directory volumes <NUM> and the one or more file volumes <NUM>, respectively. By structuring the one or more data files <NUM> in such a way, scaling of individual tiers or volumes is readily accomplished without having to add needless components or extra storage space that may otherwise go under or unutilized.

The multi-tier file partitioning component <NUM> is part of at least one of the one or more directory volumes <NUM>, the one or more file volumes <NUM>, and/or the one or more shard volumes <NUM>. Further implementations with respect to the multi-tier volume storage structure of the network file management system <NUM> are described herein with respect to <FIG>.

In some implementations, the client device <NUM> may include any mobile or fixed computer device, which may be connectable to a network. The client device <NUM> may be, for example, a computer device such as a desktop or laptop or tablet computer, a server, a cellular telephone, a gaming device, a mixed reality or virtual reality device, a music device, a television, a navigation system, a camera, a personal digital assistant (PDA), or a handheld device, or any other computer device having wired and/or wireless connection capability with one or more other devices. Further implementations with respect to example components of the network file management system <NUM> are described herein with respect to <FIG>.

Referring now to <FIG>, a storage cluster set <NUM> distinct from the network file management system <NUM> may include at least a first cluster <NUM> and a second cluster <NUM> both associated with a cluster set unified namespace <NUM>. The first cluster <NUM> may include or otherwise correspond to a first cluster shared volume (CSV) and/or scale-out file server (SOFS) <NUM>. The second cluster <NUM> may include or otherwise correspond to a second CSV/SOFS <NUM>. Each of the first cluster <NUM> and the second cluster <NUM> may include one or more storage spaces, a storage bus layer (SBL), and/or one or more extended bunch of disks (eBod). The storage spaces may include one or more nodes each having individual subspaces.

In order to scale, the storage cluster set <NUM> may scale CPU and memory components by load balancing spaces across nodes. Additionally, free space aggregation may be supported through thin provisioning. Scaling capacity may also include by adding additional eBods. As such, the storage scale out is bound by a cluster. Each storage cluster may be an island and may scale independently up to a defined limit. A user may add additional storage clusters to the storage cluster set <NUM> forming islands of scale out. Further, a user may partition their load across clusters to obtain a larger scale. However, such scaling may introduce a number of potential bottlenecks at the CPU (e.g., largest consumers may be file segment (FS) stack, redundant array of independent disks (RAID) spaces, and virtual machines (VMs), the memory (e.g., largest consumers may be the FS stack, SBL cache, and VMs), storage input/output operations per second (IOPS), and storage capacity.

Referring now to <FIG>, an example three-tier volume storage structure <NUM> of the network file management system <NUM> may permit scaling across multiple storage clusters and in a heterogeneous storage environment. The three-tier volume storage structure <NUM> may include at least one directory volume <NUM> (e.g., first tier), one or more file volumes <NUM>-<NUM> and <NUM>-<NUM> (e.g., second tier), and one or more shard volumes <NUM>-<NUM> to <NUM>-<NUM> (e.g., third tier). The directory volume <NUM> may be in communication with both file volume <NUM>-<NUM> and file volume <NUM>-<NUM>. File volume <NUM>-<NUM> may be in communication with or may be used to access shard volumes <NUM>-<NUM> to <NUM>-<NUM>. File volume <NUM>-<NUM> may be in communication with or may be used to access shard volumes <NUM>-<NUM> to <NUM>-<NUM>. The three-tier volume storage structure <NUM> may split the directory metadata <NUM> from the file metadata <NUM> and shard file content/data (e.g., one or more shards <NUM>). In other words, the directory metadata <NUM>, the file metadata <NUM>, and the file content/data (e.g., shards <NUM>) may be extracted and/or partitioned from the one or more data files <NUM> and stored in separate tiers.

For example, the at least one directory volume <NUM> may be configured to store of the directory metadata <NUM>. Specifically, the at least one directory volume <NUM> may store, for each file, file names, organize files into directories, place files on the one or more file volumes <NUM>-<NUM> and <NUM>-<NUM> using a hash function and points to them using a file identifier, create persistent reference to files on the one or more file volumes <NUM>-<NUM> and <NUM>-<NUM>, may be aware of file attributes and/or file sizes, directory opportunistic locks (Oplocks), and/or may enforce quotas (e.g., related to disk storage space).

The one or more file volumes <NUM>-<NUM> and <NUM>-<NUM> may be configured to provide storage of the file metadata <NUM>. Specifically, the files may be addressable by a file identifier unique to each file of the one or more data files <NUM>. Further, multiple file volumes <NUM>-<NUM> and <NUM>-<NUM> may be grouped together using a hash function that allows load balancing of files across the file volumes <NUM>-<NUM> and <NUM>-<NUM>. The one or more file volumes <NUM>-<NUM> and <NUM>-<NUM> may be considered a master store of information about the file. Specifically, the one or more file volumes <NUM>-<NUM> and <NUM>-<NUM> may store, for each file, the file identifier, file size, file security descriptor, shards table including a shard volume identifier and shard identifier used to access the individual shards <NUM> or data partitions, file attributes such as, but not limited to FILE_ATTRIBUTE_READONLY, FILE_ATTRIBUTE_ARCHIVE, FILE_ATTRIBUTE_COMPRESSED, FILE_ATTRIBUTE_INTEGRITY_STREAM, FILE_ATTRIBUTE_NO_SCRUB_DATA, and FILE_ATTRIBUTE_SPARSE_FILE, timestamps including at least one of create time, last access time, and/or last modification time.

The one or more file volumes <NUM>-<NUM> and <NUM>-<NUM> may also be configured to place and rebalance file shards <NUM> across the one or more shard volumes <NUM>-<NUM> to <NUM>-<NUM>. Further, the one or more file volumes <NUM>-<NUM> and <NUM>-<NUM> may be configured to handle and/or resolve file sharing between volumes and/or devices. Additionally, advisory byte range locks may be determined and set by the one or more file volumes <NUM>-<NUM> and <NUM>-<NUM>. The one or more file volumes <NUM>-<NUM> and <NUM>-<NUM> may also establish Oplocks to back the file metadata <NUM> cache on the client.

The one or more shard volumes <NUM>-<NUM> to <NUM>-<NUM> may be configured to provide storage for the shards <NUM> or partitioned data from the one or more data files <NUM>. For instance, the one or more shard volumes <NUM>-<NUM> to <NUM>-<NUM> may provide available (free) space for the one or more file volumes <NUM>-<NUM> and <NUM>-<NUM> to store the shards <NUM> or partitioned data from the one or more data files <NUM>. The valid data length (VDL) may be tracked for each shard <NUM>. Further, the one or more shard volumes <NUM>-<NUM> to <NUM>-<NUM> may be aware of at least one file attribute including, but not limited to, FILE_ATTRIBUTE_READONLY, FILE_ATTRIBUTE_ARCHIVE, FILE_ATTRIBUTE_COMPRESSED, FILE_ATTRIBUTE_INTEGRITY_STREAM, FILE_ATTRIBUTE_NO_SCRUB_DATA, and FILE_ATTRIBUTE_SPARSE_FILE. The one or more shard volumes <NUM>-<NUM> to <NUM>-<NUM> may include mandatory byte range locks and/or Oplocks to back data cache on the client.

The volumes may be hierarchically ordered according to a manner of file content access. For instance, the at least one directory volume <NUM>, or first tier, may be logically arranged ahead of the one or more file volumes <NUM>-<NUM> and <NUM>-<NUM> (second tier), and in turn, the one or more file volumes <NUM>-<NUM> and <NUM>-<NUM> may be logically arranged ahead of the one or more shard volumes <NUM>-<NUM> to <NUM>-<NUM> (third tier). As such, when a user attempts to access or perform a read/write operation associated with a file at the network file management system <NUM> using the file management client <NUM>, the file management client <NUM> may first access the first tier, or the at least one directory volume <NUM>. Using the directory metadata <NUM> associated with the file at the directory volume <NUM>, the user may then access a file volume, or the second tier, storing the file metadata <NUM> associated with the file. For instance, the particular file the user desires to access may be stored at shard volume <NUM>-<NUM> as one or more shards <NUM> or file partitions. Accordingly, the user may use the file metadata <NUM> located at file volume <NUM>-<NUM> to then access shard volume <NUM>-<NUM> so as to access the one or more shards <NUM> or file partitions associated with the desired file.

Referring now to <FIG>, a file partitioning scheme <NUM> using a three-tier volume storage structure at the network file management system <NUM> is shown. Each tier, which may include one or more volumes, may include one or more resilient file system (REFS) roles. For example, at the directory volume <NUM> (e.g., first tier), REFS may implement a user visible directory structure which may be used when handling a number of file operations including, but not limited to, open, rename, delete, close, and/or when propagating the file metadata <NUM> from the file volume <NUM> to the directory cache. Further, at the file volume <NUM> (e.g., second tier), REFS may maintain metadata of or about each file and/or file shard assignments, which may be used when handling a number of file operations including, but not limited to, open, delete, set end-of-file (EOF) or allocation size, set file attributes, and/or close.

Additionally, at the shard volume <NUM> (e.g., third tier), REFS may correspond to a sub-locator that assists to allocate and manage shards <NUM> on a disk, rather than writing or implementing a separate or distinct allocator that would subdivide disk space to shards <NUM>. Moreover, accessing shards <NUM> as files may provide a convenient file interface that may be used for cutting access to the storage on layout revocation by invalidating existing file opens, deleting the file during garbage collection of unused shards <NUM> and/or relying on REFS to lazily grow a file or files that represent the shard <NUM> as application writes to the shard <NUM>.

Metadata scaling at the directory volume <NUM> and/or the file volume <NUM> may be accomplished by splitting the directory metadata <NUM> from the file metadata <NUM> so that IOPS may be distributed across multiple volumes, and metadata file cache across multiple machines/devices. As such, the directory metadata <NUM> and the file metadata <NUM> may scale independently. With respect to data scaling and free space aggregation at the shard volume <NUM>, placing data on different volumes than metadata, and sharding a file's data across multiple volumes may assist with distributing IOPS across a much larger set of disks providing a large total throughput on what appears to the user to be a single share. Additionally, file sharding may assist with providing single free space instead of having per volume islands of free space.

As an example, to read/write data on the network file management system <NUM>, the file management client <NUM> may open the file (e.g., Foo. vhd) on the directory volume <NUM> (e.g., which may correspond to a directory server) and query a file layout. The file layout may include the file identifier (e.g., File Id) the file volume <NUM> that hosts metadata for the file. The file layout may also include a cache of the file size and one or more attributes. The user may then, via the file management client <NUM>, open the file metadata <NUM> (e.g., File Id) on the file volume <NUM> and query file layout to provide one or more shard volume identifiers and corresponding shard identifiers <NUM>. The shard volume identifier may identify a shard volume where the one or more shards <NUM> are located. The shard identifier may identify the one or more shards <NUM> within the shard volume that contain the file. Each shard <NUM> may be of a fixed or variable size. Further, the file metadata <NUM> may include one or more attributes, a file security descriptor, a file size, an allocation size, and shard information. For each shard <NUM> in the file layout, the file management client <NUM> may open the shard <NUM> on the shard volume <NUM> and perform read/write operations.

Referring now to <FIG>, an example physical volume directory layout <NUM> of the three-tier volume storage structure may lay out logical entities such as the directory volume <NUM>, file volume <NUM>, and/or shard volume <NUM> on top of a physical entity (e.g., physical volume). The physical volume directory layout <NUM> may define bindings of the logical entities between each other. In some implementations, a physical volume may be a physical storage space and a logical volume may be a virtual or software defined storage space within a physical volume.

For example, at <NUM>, every directory volume <NUM> may have a distinct or independent directory so as to permit collocating multiple roles on the same physical volume. At <NUM>, directory volume(s) <NUM> hosted by the physical volume may be defined, and bindings between the directory volume(s) <NUM> and file volume(s) <NUM> may be included. Directory volume file system mini-filter may attach to the volume if the physical volume contains any directory volume(s) <NUM>.

Further, at <NUM>, every file volume <NUM> may have a distinct or independent directory to permit collocating multiple roles on the same physical volume. At <NUM>, every directory volume <NUM> may have a distinct or independent directory to permit quick location of all files placed by the given directory volume <NUM>. At <NUM>, file to directory ratio may be controlled. At <NUM>, file volume(s) <NUM> hosted by the physical volume may be defined, and contains bindings between the file volume(s) <NUM> and shard volume(s) <NUM> as well as bindings between directory volume(s) <NUM> and file volumes <NUM>. File volume file system mini-filter may attach to the volume if the physical volume contains any file volume(s) <NUM>.

At <NUM>, every shard volume <NUM> may have a distinct or independent directory to permit collocation of multiple roles on the same physical volume. At <NUM>, every file volume <NUM> may have a distinct or independent directory to allow quick location all files placed by the given file volume <NUM>. At <NUM>, file to directory ratio may be controlled. At <NUM>, Defines shard volume(s) hosted by the physical volume may be defined, and bindings between these shard volume(s) <NUM> and file volume(s) <NUM> may be included. Shard volume file system mini-filter may attach to the volume if the physical volume contains any shard volume(s) <NUM>.

In some implementations, a single physical volume can contain multiple logical directory volume(s) <NUM>. Further, each directory volume <NUM> can be associated with multiple file volume(s) <NUM>. In some implementations, a single physical volume can contain a single data or shard volume <NUM>. Moreover, a data or shard volume <NUM> can be associated with multiple metadata or file volume(s) <NUM>.

Referring now to <FIG>, the three-tier volume storage structure may include a hierarchical representation <NUM> of the network file management system <NUM>. At a first tier, the directory volume <NUM> may include the directory metadata <NUM> used for accessing the file metadata <NUM> at the corresponding file volume <NUM>. For example, using at least the file identifier of the file metadata <NUM>, a location of the file metadata <NUM> at the file volume <NUM> may be determined based on the directory metadata <NUM>. At a second tier, the file volume <NUM> may include the file metadata <NUM> used for accessing the one or more shards <NUM> storing the file data. For instance, the file metadata <NUM> at the file volume <NUM> may include at least a shard volume identifier that identifies the shard volume <NUM> storing the file data, or shards <NUM>, and the location of the one or more shards <NUM> within the shard volume <NUM> containing the file data based on a shard identifier.

At a third tier, the shard volume <NUM> may include the one or more shards <NUM> storing the file data/content. In some implementations, the shard size, or in other words, the size of the individual shards may be fixed or variable. That is, each shard may be of the same size (e.g., in megabytes (MB), gigabytes (GB), or another unit of memory size) or may be of different sizes between the one or more shards <NUM>. Further, a shard size may increase as the file size increases. For example, for a first storage size (e.g., <NUM> GB), the shard size, or each individual shard may be a first size (e.g., <NUM> MB). Subsequently, or for a second storage size (e.g., greater than <NUM> GB), the shard size, or each individual shard may be a second size (e.g., <NUM> MB) greater than the first size. By increasing the shard size, a number of records in a run table that identifies each individual shard may be decreased.

Referring now to <FIG>, an example directory volume scale out <NUM> using partitioning of the three-tier volume storage structure is shown. The directory volume scale out <NUM> includes a distributed file system namespace (DFSN) <NUM> for each of the directory volumes. For example, each DFSN may correspond to or otherwise be associated with at least a first share <NUM>, a second share <NUM>, a third share <NUM>, and a fourth share <NUM>. The shares may correspond to different servers or separate storage spaces within a single server. The first share <NUM> may include a directory volume <NUM>-<NUM>, the second share <NUM> may include a directory volume <NUM>-<NUM>, the third share <NUM> may include a directory volume <NUM>-<NUM>, and the fourth share <NUM> may include a directory volume <NUM>-<NUM>.

In some implementations, DFSN <NUM> may bring the multiple directory volumes <NUM>-<NUM> to <NUM>-<NUM> under a single file server name. Each directory volume may be visible to a user as a separate share on an high availability (HA) file server. In some cases, there may be multiple file servers. Such an implementation may assist scaling by distributing directory metadata IOPS across multiple disks, and by separating directory volume metadata file cache across multiple machines. DFSN may be used at the top or initial point to expose all shares under the same file server name. Further, the directory volumes <NUM>-<NUM> to <NUM>-<NUM> may be a shared pool of volumes to place data across and may provide one pool of free space across all directory volumes <NUM>. In some implementations, all directory volumes <NUM> may allocate file shards <NUM> from the common pool of file volumes <NUM> and shard volumes <NUM> maintaining an appearance of a single unified free space.

Referring now to <FIG>, a three-tier volume storage structure scale out <NUM> may provide efficient scaling of individual tiers in an independent manner. Scaling tiers or volume layers may not increase latency. For instance, the three-tier scale out may require at least three open actions/events before an application can read and/or write data regardless of a number of volumes in each tier/layer (e.g., each shard requires a separate open).

Specifically, a unified namespace <NUM> may provide access to multiple shares and/or directory volumes under a single file server name. The metadata scale out <NUM> may include scaling of one or both of the directory volumes <NUM> and/or the file volumes <NUM>. For example, the directory metadata scale out <NUM> may be implemented by adding additional directory volumes to the existing directory volumes <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM>, and associating each additional directory volume to the unified namespace <NUM>. In some implementations, adding a new or additional share may implicitly add an additional directory volume. Further, adding additional directory volumes may assist with scaling directory metadata IOPs by spreading them across multiple disks, and also helps with scaling directory metadata file cache across multiple nodes. Moreover, multiple directory volumes <NUM> may share the same pool of file volumes <NUM> and shard volumes <NUM> providing one pool of free space.

The file metadata scale out <NUM> may include providing additional file volumes to the existing file volumes <NUM>-<NUM> and <NUM>-<NUM> and associating each additional file volume with at least one existing directory volume <NUM>-<NUM>, <NUM>-<NUM>, or <NUM>-<NUM>, or an additional directory volume. In some implementations, if file metadata <NUM> updates/reads results in a bottleneck, then additional file volumes may be added to scale out. Adding file volumes <NUM> may assist with scaling file metadata IOPs by spreading them across multiple disks, and also helps with scaling metadata file cache across multiple nodes. Further, a single file volume <NUM> can host data from multiple directory volumes <NUM>.

Further, the data scale out <NUM> may include providing additional shard volumes to the existing shard volumes <NUM>-<NUM> to <NUM>-<NUM>. For instance, additional shard volumes may be added to the set of shard volumes <NUM>-<NUM> to <NUM>-<NUM> associated with file volume <NUM>-<NUM>. Alternatively, or in addition to, additional shard volumes may be added to the set of shard volumes <NUM>-<NUM> to <NUM>-<NUM> associated with file volume <NUM>-<NUM>. In some implementations, if free space falls below a defined level, or if additional IOPS/ bandwidth are needed, then additional shard volumes <NUM> may be added.

Referring now to <FIG>, a file load balancing scheme <NUM> may provide a two-tier volume storage structure for a network file management system <NUM>. The file load balancing scheme <NUM> may separate directory metadata <NUM> from file metadata <NUM> and file data, yet may not shard the file data. In other words, the file metadata <NUM> and the file data of the one or more data files <NUM> may be on the same volume (e.g., file volume <NUM>). For example, each file may have exactly one shard located on a file volume <NUM>. All file metadata <NUM> may be located on the file volume(s) <NUM> along with the data of the one or more data files <NUM>. The file load balancing scheme <NUM> may have a minimum impact on file opens (e.g., only two opens to access data on a given file). Further, a maximum file size may be limited by the free space on the volume it is placed.

For example, the directory volume <NUM> may be configured to store the directory metadata <NUM>. Specifically, the at least one directory volume <NUM> may store, for each file, file names, organize files into directories, place files on the one or more file volumes <NUM>-<NUM> to <NUM>-<NUM> using a hash function and points to them using a file identifier, create persistent reference to files on the one or more file volumes <NUM>-<NUM> to <NUM>-<NUM>, may be aware of file attributes and/or file sizes, Oplocks, and/or may enforce quotas (e.g., related to disk storage space).

The file volumes <NUM>-<NUM> to <NUM>-<NUM> may be configured to provide file metadata <NUM> store and file data store. In particular, each of the file volumes <NUM>-<NUM> to <NUM>-<NUM> may provide available (free) space for the directory volume <NUM>. The files may be addressable by a file identifier unique to each file of the one or more data files <NUM>. The one or more file volumes <NUM>-<NUM> to <NUM>-<NUM> may be considered a master store of information about the file. For instance, each of the one or more file volumes <NUM>-<NUM> to <NUM>-<NUM> may store, for each file, the file identifier, persistent reference count, file size, VDL tracker, file security descriptor, shards table, file attributes such as, but not limited to FILE_ATTRIBUTE_READONLY, FILE_ATTRIBUTE_ARCHIVE, FILE_ATTRIBUTE_COMPRESSED, FILE_ATTRIBUTE_INTEGRITY_STREAM, FILE_ATTRIBUTE_NO_SCRUB_DATA, and FILE_ATTRIBUTE_SPARSE_FILE, timestamps including at least one of create time, last access time, and/or last modification time. The one or more file volumes <NUM>-<NUM> to <NUM>-<NUM> may include mandatory byte range locks and/or Oplocks to back data cache on the client, and file sharing between file volumes.

Referring now to <FIG>, a two tier volume storage structure <NUM> corresponding to the file load balancing scheme <NUM> may include the network file management system <NUM>. Each tier, which may include one or more volumes, may include one or more REFS roles. For example, at the directory volume <NUM> (e.g., first tier), REFS may implement a user visible directory structure which may be used when handling a number of file operations including, but not limited to, open, rename, delete, close, and/or when propagating the file metadata <NUM> from the file volume <NUM> to the directory cache. Further, at the file volume <NUM> (e.g., second tier), REFS may maintain metadata of or about each file and/or file shard assignments, which may be used when handling a number of file operations including, but not limited to, open, delete, set EOF or allocation size, set file attributes, close, and/or reads/writes.

As an example, to read/write data on the network file management system <NUM>, the file management client <NUM> may open the file (e.g., Foo. vhd) on the directory volume <NUM> (e.g., which may correspond to a directory server) and query a file identifier and file volume <NUM> that hosts the file metadata <NUM> and data for the file. The directory metadata <NUM> may include a file identifier of the file on the file volume <NUM> and a cache of one or more file sizes and/or attributes. The user may then, via the file management client <NUM>, open the file metadata <NUM> (e.g., File Id) and/or data on the file volume <NUM> and performs read and/or write operations. The file metadata <NUM> may include one or more attributes, a security descriptor, a file size, and/or an allocation size.

Referring now to <FIG>, an example physical volume directory layout <NUM> of the file load balancing scheme <NUM> may lay out logical entities such as the directory volume <NUM> and the file volume <NUM> on top of a physical entity (e.g., physical volume). The physical volume directory layout <NUM> may define bindings of the logical entities between each other (e.g., between directory volume <NUM> and file volume <NUM>).

Further, at <NUM>, every file volume <NUM> may have a distinct or independent directory to permit collocating multiple roles on the same physical volume. At <NUM>, every directory volume <NUM> may have a distinct or independent directory to permit quick location of all files placed by the given directory volume <NUM>. At <NUM>, file to directory ratio may be controlled. At <NUM>, file volume(s) <NUM> hosted by the physical volume may be defined, and contains bindings between the file volume(s) <NUM> and shard volume(s) <NUM> as well as bindings between directory volume(s) <NUM> and file volumes <NUM>. File volume file system mini-filter may attach to the volume if the physical volume contains any file volume(s) <NUM>. In some implementations, a single physical volume can contain multiple logical directory volumes <NUM>. Further, each directory volume <NUM> can be associated with multiple file volumes <NUM>.

Referring now to <FIG>, the two-tier volume storage structure may include a hierarchical representation <NUM> of the network file management system <NUM>. At a first tier, the directory volume <NUM> may include the directory metadata <NUM> used for accessing the file metadata <NUM> at the corresponding file volume <NUM>. For example, using at least the file identifier of the file metadata <NUM>, a location of the file metadata <NUM> at the file volume <NUM> may be determined based on the directory metadata <NUM>. At a second tier, the file volume <NUM> may include the file metadata <NUM> used for accessing the file data. For instance, the file metadata <NUM> at the file volume <NUM> may include at least one or more attributes, a security descriptor, a file size, and/or an allocation size.

Referring now to <FIG>, a two-tier volume storage structure scale out <NUM> of the file load balancing scheme <NUM> may provide efficient scaling of individual tiers in an independent manner. For instance, the two-tier scale out may require at least two open actions/events before an application can read and/or write data regardless of a number of volumes in each tier/layer.

Specifically, a unified namespace <NUM> may provide access to multiple shares and/or directory volumes under a single file server name. The metadata scale out <NUM> may include scaling of the directory volumes <NUM>-<NUM> to <NUM>-<NUM>, each of which may be associated with a respective one of a first share <NUM>, second share <NUM>, or third share <NUM>. For example, the directory metadata scale out <NUM> may be implemented by adding additional directory volumes to the existing directory volumes <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM>, and associating each additional directory volume to the unified namespace <NUM>. In some implementations, adding a new or additional share may implicitly add an additional directory volume. Further, adding additional directory volumes may assist with scaling directory metadata IOPs by spreading them across multiple disks, and also helps with scaling directory metadata file cache across multiple nodes. Moreover, multiple directory volumes <NUM> may share the same pool of file volumes <NUM> and shard volumes <NUM> providing one pool of free space.

The file metadata scale out <NUM> and the data scale out <NUM> may include providing additional file volumes to the existing file volumes <NUM>-<NUM> to <NUM>-<NUM> and associating each additional file volume with at least one existing directory volume <NUM>-<NUM>, <NUM>-<NUM>, or <NUM>-<NUM>, or an additional directory volume. In some implementations, if file metadata <NUM> updates/reads results in a bottleneck, then additional file volumes may be added to scale out. Adding file volumes <NUM> may assist with scaling file metadata IOPs by spreading them across multiple disks, and also helps with scaling metadata file cache across multiple nodes. Adding file volumes <NUM> may also assist with help increasing total data IOPS and bandwidth by load balancing data across multiple disks. Further, a single file volume <NUM> can host data from multiple directory volumes <NUM>.

Referring now to <FIG>, a file data sharding scheme <NUM> may provide a two-tier volume storage structure for the network file management system <NUM>. The file data sharding scheme <NUM> may separate directory metadata <NUM> and file metadata <NUM> from file data, and may shard the file data. In other words, the directory metadata <NUM> and the file metadata <NUM> may be stored or maintained on a same volume, and the file data of the one or more data files <NUM> may be on a different volume (e.g., file volume <NUM>). For example, by co-locating the directory volume <NUM> and the file volume <NUM>, the file data sharding scheme <NUM> may have a minimum impact on file opens (e.g., only two opens to access data on a given file). Further, as the directory metadata <NUM> and the file metadata <NUM> are also co-located, they may also scale together.

For example, the co-located directory volume <NUM> and file volume <NUM> may be configured to store both the directory metadata <NUM> and file metadata <NUM>. Specifically, the co-located directory volume <NUM> and file volume <NUM> may store, for each file, file names, organize files into directories, enforce quotas (e.g., related to disk storage space), and may be considered a master store of information about the file. Further, the co-located directory volume <NUM> and file volume <NUM> may be configured to store, for each file, the file identifier, persistent reference count, file size, file security descriptor, shards table, file attributes such as, but not limited to FILE_ATTRIBUTE_READONLY, FILE_ATTRIBUTE_ARCHIVE, FILE_ATTRIBUTE_COMPRESSED, FILE_ATTRIBUTE_INTEGRITY_STREAM, FILE_ATTRIBUTE_NO_SCRUB_DATA, and FILE_ATTRIBUTE_SPARSE_FILE, timestamps including at least one of create time, last access time, and/or last modification time. The co-located directory volume <NUM> and file volume <NUM> may place and rebalance file shards across data or shard volumes <NUM>-<NUM> to <NUM>-<NUM>, may support file sharing, may provide an advisory byte range locks and/or Oplocks to support directory metadata <NUM> and file metadata <NUM> caching on the client.

The one or more shard volumes <NUM>-<NUM> to <NUM>-<NUM> may be configured to provide storage for the shards <NUM> or partitioned data from the one or more data files <NUM>. For instance, the one or more shard volumes <NUM>-<NUM> to <NUM>-<NUM> may provide available (free) space for the one or more co-located directory volumes <NUM> and file volumes <NUM> to store the shards <NUM> or partitioned data from the one or more data files <NUM>. The VDL may be tracked for each shard <NUM>. Further, the one or more shard volumes <NUM>-<NUM> to <NUM>-<NUM> may be aware of at least one file attribute including, but not limited to, FILE_ATTRIBUTE_READONLY, FILE_ATTRIBUTE_ARCHIVE, FILE_ATTRIBUTE_COMPRESSED, FILE_ATTRIBUTE_INTEGRITY_STREAM, FILE_ATTRIBUTE_NO_SCRUB_DATA, and FILE_ATTRIBUTE_SPARSE_FILE. The one or more shard volumes <NUM>-<NUM> to <NUM>-<NUM> may include mandatory byte range locks and/or Oplocks to back data cache on the client.

Referring now to <FIG>, a two tier volume storage structure <NUM> corresponding to the file sharding scheme <NUM> may include the network file management system <NUM>. Each tier, which may include one or more volumes, may include one or more REFS roles. For example, at the co-located directory volume <NUM> and file volume <NUM> (e.g., first tier), REFS may implement a user visible directory structure which may be used when handling a number of file operations including, but not limited to, open, rename, delete, close, and/or when propagating the file metadata <NUM> from the file volume <NUM> to the directory cache.

Further, at the shard volume <NUM> (e.g., second tier), REFS may correspond to a sub-locator that assists to allocate and manage shards <NUM> on a disk, rather than writing or implementing a separate or distinct allocator that would subdivide disk space to shards <NUM>. Moreover, accessing shards <NUM> as files may provide a convenient file interface that may be used for cutting access to the storage on layout revocation by invalidating existing file opens, deleting the file during garbage collection of unused shards <NUM> and/or relying on REFS to lazily grow a file or files that represent the shard <NUM> as application writes to the shard <NUM>.

As an example, to read/write data on the network file management system <NUM>, the file management client <NUM> may open the file (e.g., Foo. vhd) on the co-located directory volume <NUM> and file volume <NUM> (e.g., which may correspond to a directory server) and query shards layout including the shard volumes that store the one or more shards <NUM>. For each shard, the file management client <NUM> may open the shard on a shard volume <NUM> and perform read and/or write operations. The above example may include one open instance to obtain the shard layout, and another open instance for each file shard.

Referring now to <FIG>, an example physical volume directory layout <NUM> of the file sharding scheme <NUM> may lay out logical entities such as the directory volume <NUM> and the shard volume <NUM> on top of a physical entity (e.g., physical volume). The physical volume directory layout <NUM> may define bindings of the logical entities between each other (e.g., between directory volume <NUM> and shard volume <NUM>).

For example, at <NUM>, every directory volume <NUM> may have a distinct or independent directory so as to permit collocating multiple roles on the same physical volume. At <NUM>, directory volume(s) <NUM> hosted by the physical volume may be defined, and bindings between the directory volume(s) <NUM> and shard volume(s) <NUM> may be included. Directory volume file system mini-filter may attach to the volume if the physical volume contains any directory volume(s) <NUM>.

At <NUM>, every shard volume <NUM> may have a distinct or independent directory to permit collocation of multiple roles on the same physical volume. At <NUM>, every directory volume <NUM> may have a distinct or independent directory to allow quick location all files placed by the given directory volume <NUM>. At <NUM>, file to directory ratio may be controlled. At <NUM>, Defines shard volume(s) hosted by the physical volume may be defined, and bindings between these shard volume(s) <NUM> and directory volume(s) <NUM> may be included. Shard volume file system mini-filter may attach to the volume if the physical volume contains any shard volume(s) <NUM>. In some implementations, a single physical volume can contain multiple logical directory volume(s) <NUM>. Further, each directory volume <NUM> can be associated with multiple file volume(s) <NUM>.

Referring now to <FIG>, a file metadata location at a two tier volume storage structure corresponding to the file sharding scheme <NUM>. The two-tier volume storage structure may include a hierarchical representation <NUM> of the network file management system <NUM>. At a first tier, the co-located directory volume <NUM> and file volume <NUM> may include the directory metadata <NUM> and file metadata <NUM> used for accessing the one or more shards <NUM> storing the file data. For instance, the file metadata <NUM> at the file volume <NUM> may include at least a shard volume identifier that identifies the shard volume <NUM> storing the file data, or shards <NUM>, and the location of the one or more shards <NUM> within the shard volume <NUM> containing the file data based on a shard identifier. At a second tier, the shard volume <NUM> may include the one or more shards <NUM> storing the file data/content.

Referring now to <FIG>, a two-tier volume storage scale out <NUM> of the file sharding scheme <NUM> may provide efficient scaling of individual tiers in an independent manner. For instance, scaling tiers or layers may not increase latency as the two-tier scale out may require at least two open actions/events before an application can read and/or write data regardless of a number of volumes in each tier/layer (e.g., each shard, however, requires a separate open instance).

Specifically, a unified namespace <NUM> may provide access to multiple shares and/or directory volumes under a single file server name. The metadata scale out <NUM> may include scaling of the directory volumes <NUM>-<NUM> to <NUM>-<NUM>, each of which may be associated with a respective one of a first share <NUM>, second share <NUM>, or third share <NUM>. For example, the metadata scale out <NUM> may be implemented by adding additional directory volumes to the existing directory volumes <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM>, and associating each additional directory volume to the unified namespace <NUM>. In some implementations, adding a new or additional share may implicitly add an additional directory volume. Further, adding additional directory volumes may assist with scaling directory metadata IOPs by spreading them across multiple disks, and also helps with scaling directory metadata file cache across multiple nodes. Moreover, multiple directory volumes <NUM> may share the same pool of file volumes <NUM> and shard volumes <NUM> providing one pool of free space.

Further, the data scale out <NUM> may include providing additional shard volumes to the existing shard volumes <NUM>-<NUM> to <NUM>-<NUM>. In some implementations, if free space falls below a defined level, or if additional IOPS/ bandwidth are needed, then additional shard volumes <NUM> may be added.

Referring now to <FIG>, an example method <NUM> for opening an entry at the network file management system <NUM>. The actions illustrated in method <NUM> may overlap in time. For example, at an instant in time, two of the actions may be performed by different components. The execution of the actions may also be interleaved on a component. Additionally, the actions illustrated in method <NUM> may be performed in an order other than illustrated in <FIG>.

At block <NUM>, the method <NUM> may receive, at a storage system, a request to open an entry. For example, as described herein, the network file management system <NUM> may receive, at the directory volume <NUM>, a request to open entry. In some implementations, the entry may correspond to either a file or a directory in the network file management system <NUM>. In some implementations, the directory volume <NUM> may store directory metadata <NUM> associated with the one or more data files <NUM>.

At block <NUM>, the method <NUM> may determine directory metadata from a directory volume, the directory metadata associated with the entry. For instance, as described herein, the network file management system <NUM> may determine directory metadata <NUM> from a directory volume <NUM>, the directory metadata <NUM> may be associated with the entry.

At block <NUM>, the method <NUM> may determine whether the entry corresponds to a file or a directory based on the directory metadata. For instance, as described herein, the network file management system <NUM> may determine whether the entry corresponds to a file or a directory based on the directory metadata <NUM>.

At block <NUM>, the method <NUM> may open the directory metadata associated with the entry from the directory volume based on a determination that the entry corresponds to a directory. For instance, as described herein, the network file management system <NUM> may open the directory metadata <NUM> associated with the entry from the directory volume <NUM>.

At block <NUM>, the method <NUM> may open the directory metadata associated with the entry from the directory volume based on a determination that the entry does corresponds to a file. For instance, as described herein, the network file management system <NUM> may open the directory metadata <NUM> associated with the entry from the directory volume <NUM>.

At block <NUM>, the method <NUM> may open file metadata associated with the entry from the file volume. For instance, as described herein, the network file management system <NUM> may open file metadata <NUM> associated with the entry from the file volume <NUM>. In some implementations, the file volume <NUM> may store the file metadata <NUM> associated with the file of the one or more data files <NUM>.

Although not shown, the method <NUM> may further determine a shard volume identifier and one or more shard identifiers associated with the file based on the file metadata <NUM>, identify a shard volume <NUM> based on the shard volume identifier, and access one or more shards using the one or more shard identifiers at the shard volume <NUM> to access the file.

In some implementations, the directory volume <NUM>, the file volume <NUM>, and the shard volume <NUM> may be separate volumes forming a three tier hierarchical structure. In some implementations, the file volume <NUM> and the shard volume <NUM> may be co-located at a same volume, separate from the directory volume <NUM>, and forming a two tier structure. In some implementations, the file metadata <NUM> and the file data are stored at the same volume.

In some implementations, the directory volume <NUM> and the file volume <NUM> may be co-located at a same volume, separate from the shard volume <NUM>, and form a two tier structure. In some implementations, the directory metadata <NUM> and the file metadata <NUM> are stored at the same volume.

In some implementations, the directory volume <NUM> may provide a user visible directory structure, and the directory metadata <NUM> may include at least one of the file identifier, a cache of a file size, or a cache of at least one file attribute.

In some implementations, the file volume <NUM> may maintain the file metadata <NUM> associated with one or more data files <NUM> including the file and file shard assignments, and the file metadata <NUM> may include at least one of the file identifier, a file size, a file security descriptor, a shard table including at least a shard volume identifier and an individual shard identifier, or at least one file attribute.

In some implementations, the shard volume <NUM> may provide storage area to one or more linked file volumes <NUM>, and the file data includes at least one of a shard identifier, a valid data length, or written data.

In some implementations, each of the one or more shards <NUM> may correspond to a fixed size storage area, and the fixed size may correspond to at least one of a uniform size across each of the one or more shards <NUM> at the shard volume <NUM> or a varying size across each of the one or more shards <NUM> at the shard volume <NUM>.

In some implementations, the file volume <NUM> may be associated with an additional volume forming a set of file volumes (e.g., <NUM>-<NUM> to <NUM>-N) both associated with the directory volume <NUM>. Further, in some implementations, although not shown, the method <NUM> may include grouping the file volume <NUM> and the additional file volume using a hash function supporting load balancing of a set of files (e.g., one or more data files <NUM>) including the file.

In some implementations, the file volume <NUM> and the shard volume <NUM> may be accessible by the directory volume <NUM> and at least one other directory volume (e.g., <NUM>-N). In some implementations, the directory volume <NUM> may provide access to two or more distinct file volumes including the file volume (e.g., file volume <NUM>-<NUM> and <NUM>-N). In some implementations, the file volume <NUM> may provide access to two or more distinct shard volumes including the shard volume (e.g., shard volumes <NUM>-<NUM> and <NUM>-N).

Referring now to <FIG>, a method <NUM> for deleting an entry at the network file management system <NUM>. The actions illustrated in method <NUM> may overlap in time. For example, at an instant in time, two of the actions may be performed by different components. The execution of the actions may also be interleaved on a component.

At block <NUM>, the method <NUM> receives, at the storage system, a deletion request for the entry, the entry corresponding to either a file or a directory in the storage system. As described herein, the network file management system <NUM> receives a deletion request for the entry, the entry corresponding to either a file or a directory in the storage system.

At block <NUM>, the method <NUM> determines directory metadata from a directory volume, the directory metadata associated with the entry. As described herein, the network file management system <NUM> determines directory metadata <NUM> from a directory volume <NUM>, the directory metadata <NUM> associated with the entry.

At block <NUM>, the method <NUM> determines whether the entry corresponds to the file or the directory based on the directory metadata. As described herein, the network file management system <NUM> determines whether the entry corresponds to the file or the directory based on the directory metadata <NUM>.

At block <NUM>, the method <NUM> requests deletion of the directory metadata associated with the entry from the directory volume based on a determination that the entry corresponds to the directory. As described herein, the network file management system <NUM> requests deletion of the directory metadata <NUM> associated with the entry from the directory volume <NUM>.

At block <NUM>, the method <NUM> requests deletion of the directory metadata associated with the entry from the directory volume based on a determination that the entry corresponds to the file. As described herein, the network file management system <NUM> requests deletion of the directory metadata <NUM> associated with the entry from the directory volume <NUM>.

At block <NUM>, the method <NUM> requests deletion of the file from the file volume. As described herein, the network file management system <NUM> requests deletion of the file from the file volume <NUM> based on a determination that the entry corresponds to the file.

In some implementations, requesting deletion of the directory metadata <NUM> may include one or more actions selected from the group consisting of marking the directory metadata <NUM> associated with the entry as invalid, marking the directory metadata <NUM> associated with the entry as deleted, overwriting the directory metadata <NUM> associated with the entry, storing, within a persistent queue, the deletion request for the entry, storing, within a persistent queue, the directory metadata <NUM> associated with the entry, removing the directory metadata <NUM> associated with the entry from a tracking structure associated with the directory containing the entry, and decrementing a reference count for the directory metadata <NUM>.

Based on the determination that the entry corresponds to the file, the method <NUM> includes determining, based on the directory metadata <NUM>, the file volume <NUM> containing file metadata <NUM> for the file, the file metadata <NUM> comprising a shard identifier identifying at least a portion of the file data for the file within a shard volume <NUM>, requesting deletion of the file metadata <NUM> from the file volume <NUM>, and requesting deletion of the shard identifier from the shard volume <NUM>.

In some implementations, based on the determination that the entry corresponds to the file, requesting deletion of the file from the file volume <NUM> may include storing, within a persistent queue, a queue entry, the queue entry comprising at least one of the deletion request for the entry, the directory metadata <NUM> associated with the entry, or a file identifier corresponding to the file, determining, based on the persistent queue including the queue entry, a file metadata <NUM> associated with the file, the file metadata <NUM> comprising a shard volume identifier and an individual shard identifier, and requesting deletion of the individual shard identifier from the shard volume <NUM> based on the shard volume identifier.

The file volume <NUM> includes file metadata <NUM> associated with the file, and the file metadata <NUM> includes a file size and a shard identifier within a shard volume. The shard volume <NUM> includes shard metadata corresponding to the shard identifier, the shard metadata comprising a valid data length for the shard.

In some implementations, the file metadata <NUM> may include a maximum data length for the shard, the valid data length for the shard being less than the corresponding maximum data length.

In some implementations, the file metadata <NUM> may include an allocated data length for the shard, the valid data length for the shard being less than the corresponding allocated data length.

In some implementations, based on a determination that the entry corresponds to the file, although not shown, the method <NUM> may include deleting the directory metadata <NUM> from the directory volume <NUM>, and determining completion of the deletion request based on deleting the directory metadata <NUM> from the directory volume <NUM> and irrespective of a pending request to delete the file from the file volume <NUM>.

Referring now to <FIG>, an example method <NUM> for adding an entry to the network file management system <NUM>. The actions illustrated in method <NUM> may overlap in time. For example, at an instant in time, two of the actions may be performed by different components. The execution of the actions may also be interleaved on a component.

At block <NUM>, the method <NUM> receives at the storage system, a request for adding the entry. As described herein, the network file management system <NUM> receives at the storage system, a request for adding the entry.

At block <NUM>, the method <NUM> determines that directory metadata associated with the entry does not exist in a directory volume. As described herein, the network file management system <NUM> determines that directory metadata <NUM> associated with the entry does not exist in a directory volume <NUM>.

At block <NUM>, the method <NUM> determines whether the entry corresponds to a file or a directory. As described herein, the network file management system <NUM> determines whether the entry corresponds to the file or the directory.

At block <NUM>, the method <NUM> generates the directory metadata associated with the directory in the directory volume based on a determination that the entry corresponds to the directory. As described herein, the network file management system <NUM> generates the directory metadata <NUM> associated with the directory in the directory volume <NUM>.

At block <NUM>, the method <NUM> determines to add the file to a file volume. As described herein, the network file management system <NUM> determines to add the file to a file volume <NUM>.

At block <NUM>, the method <NUM> allocates file metadata in the file volume, the file metadata associated with the file. As described herein, the network file management system <NUM> allocates file metadata <NUM> in the file volume <NUM>, the file metadata <NUM> associated with the file.

At block <NUM>, the method <NUM> generates the directory metadata associated with the file in the directory volume. As described herein, the network file management system <NUM> generates the directory metadata <NUM> associated with the file in the directory volume <NUM>. The directory metadata <NUM> includes an identifier of the file metadata <NUM>.

Based on the determination that the entry corresponds to the file, the directory metadata <NUM> includes an identifier for the allocated file metadata <NUM>.

Based on the determination that the entry corresponds to the file, although not shown, the method <NUM> includes determining a shard volume <NUM> to store data for the file as one or more shards <NUM>, and adding the data for the file to the shard volume <NUM> as one or more shards <NUM>.

The file metadata <NUM> includes a shard volume identifier identifying the shard volume <NUM>, and a shard identifier associated with each of the one or more shards <NUM> and identifying the file within the shard volume <NUM>.

In some implementations, determining to add the file to the file volume <NUM> may include identifying at least one of the file volume <NUM> within a group of file volumes or the shard volume <NUM> within a group of shard volumes based on load information.

In some implementations, although not shown, the method <NUM> may include reallocating the one or more shards <NUM> across to another shard volume <NUM> within the group of shard volumes based on the load information.

Referring now to <FIG>, illustrated is an example network file management system <NUM> in accordance with an implementation, including additional component details (relative to multi-tier file partitioning component <NUM>) as compared to <FIG>. In one example, the network file management system <NUM>, which may be an computer device such as a server device, may include processor <NUM> for carrying out processing functions associated with one or more of components and functions described herein. Processor <NUM> can include a single or multiple set of processors or multi-core processors. Moreover, processor <NUM> can be implemented as an integrated processing system and/or a distributed processing system. In an implementation, for example, processor <NUM> may include a CPU. In an example, the network file management system <NUM> may include memory <NUM> for storing instructions executable by the processor <NUM> for carrying out the functions described herein.

The memory <NUM> may be configured for storing data and/or computer-executable instructions defining and/or associated with an operating system and/or application, and CPU may execute operating system and/or application. An example of memory can include, but is not limited to, a type of memory usable by a computer, such as random access memory (RAM), read only memory (ROM), tapes, magnetic discs, optical discs, volatile memory, non-volatile memory, and any combination thereof. Memory may store local versions of applications being executed by CPU.

The processor <NUM> may be any processor specially programmed as described herein, including a controller, microcontroller, application specific integrated circuit (ASIC), field programmable gate array (FPGA), system on chip (SoC), or other programmable logic or state machine. The processor <NUM> may include other processing components such as an arithmetic logic unit (ALU), registers, and a control unit. Further, the operating system may include instructions (such as one or more applications) stored in memory and executed by the CPU. The network file management system <NUM> may also include one or more applications including instructions stored in memory <NUM> and executed by the processor <NUM>. Additionally, the network file management system <NUM> may include an operating system (not shown) that coordinates the utilization of hardware and/or software resources on the network file management system <NUM>, as well as one or more applications that perform specialized tasks and/or provide additional functionality.

Further, the network file management system <NUM> may include a communications component <NUM> that provides for establishing and maintaining communications with one or more parties utilizing hardware, software, and services as described herein. Communications component <NUM> may carry communications between components on the network file management system <NUM>, as well as between the network file management system <NUM> and external devices, such as devices located across a communications network and/or devices serially or locally connected to the network file management system <NUM>. For example, communications component <NUM> may include one or more buses, and may further include transmit chain components and receive chain components associated with a transmitter and receiver, respectively, operable for interfacing with external devices.

Additionally, the network file management system <NUM> may include a data store <NUM>, which can be any suitable combination of hardware and/or software, that provides for mass storage of information, databases, and programs employed in connection with implementations described herein. For example, data store <NUM> may be a data repository for operating system and/or applications.

The network file management system <NUM> may also include a user interface component <NUM> operable to receive inputs from a user of the network file management system <NUM> and further operable to generate outputs for presentation to the user. User interface component <NUM> may include one or more input devices, including but not limited to a keyboard, a number pad, a mouse, a touch-sensitive display, a navigation key, a function key, a microphone, a voice recognition component, any other mechanism capable of receiving an input from a user, or any combination thereof. Further, user interface component <NUM> may include one or more output devices, including but not limited to a display, a speaker, a haptic feedback mechanism, a printer, any other mechanism capable of presenting an output to a user, or any combination thereof.

In an implementation, user interface component <NUM> may transmit and/or receive messages corresponding to the operation of operating system and/or application. In addition, processor <NUM> executes operating system and/or application, and memory <NUM> or data store <NUM> may store them.

As used in this application, the terms "component," "system" and the like are intended to include a computer-related entity, such as but not limited to hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computer device and the computer device can be a component. One or more components can reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures stored thereon. The components may communicate by way of local and/or remote processes such as in accordance with a signal having one or more data packets, such as data from one component interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems by way of the signal.

Various implementations or features may have been presented in terms of systems that may include a number of devices, components, modules, and the like. It is to be understood and appreciated that the various systems may include additional devices, components, modules, etc. and/or may not include all of the devices, components, modules etc. discussed in connection with the figures. A combination of these approaches may also be used.

The various illustrative logics, logical blocks, and actions of methods described in connection with the embodiments disclosed herein may be implemented or performed with a specially-programmed one of a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor may also be implemented as a combination of computer devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Additionally, at least one processor may comprise one or more components operable to perform one or more of the steps and/or actions described above.

Further, the steps and/or actions of a method or algorithm described in connection with the implementations disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium may be coupled to the processor, such that the processor can read information from, and write information to, the storage medium. Further, in some implementations, the processor and the storage medium may reside in an ASIC. Additionally, the ASIC may reside in a user terminal. Additionally, in some implementations, the steps and/or actions of a method or algorithm may reside as one or any combination or set of codes and/or instructions on a machine readable medium and/or computer readable medium, which may be incorporated into a computer program product.

In one or more implementations, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or code on a computer-readable medium. A storage medium may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs usually reproduce data optically with lasers.

Claim 1:
A method of deleting an entry from a storage system having a three tier volume storage structure, wherein at a first tier, a directory volume (<NUM>) comprises directory metadata (<NUM>), at a second tier, a file volume (<NUM>) comprises file metadata (<NUM>), and at a third tier, a shard volume comprises shard metadata and one or more shards storing file data, the directory metadata being used for accessing the file metadata at the file volume and the file metadata being used for accessing the one or more shards storing the file data, comprising:
receiving (<NUM>), at the storage system, a deletion request for the entry, the entry corresponding to either a file or a directory in the storage system;
determining (<NUM>) directory metadata from a directory volume, the directory metadata associated with the entry;
determining (<NUM>) whether the entry corresponds to the file or the directory based on the directory metadata;
based on a determination that the entry corresponds to the directory:
requesting (<NUM>) deletion of the directory metadata associated with the entry from the directory volume; and
based on a determination that the entry corresponds to the file:
requesting (<NUM>) deletion of the directory metadata associated with the entry from the directory volume;
determining, based on the directory metadata, a file volume containing file metadata for the file, the file metadata comprising a shard volume identifier identifying a shard volume where one or more shards are located and a shard identifier identifying the one or more shards within the shard volume that contain the file;
requesting deletion of the file metadata from the file volume;
requesting deletion of the shard identifier from the shard volume; and
requesting (<NUM>) deletion of the file from the file volume.