File system quota versioning

A system and method for handling file system quotas are disclosed. In one implementation, a processing device receives a first command to disable a quota capability of a file system. In response, one or more instructions are initiated to delete a first association of a quota attribute with a directory, where the first association assigns a first value to a first string comprising a combination (e.g., concatenation, etc.) of an attribute name and a first symbol. Prior to completion of the execution of the one or more instructions, one or more commands are received to enable the quota capability and assign a second value of the quota attribute to the directory. In response, a second association of the quota attribute with the directory is generated, the second association assigning a second value of the quota attribute to a second string comprising a combination of the attribute name and a second symbol.

TECHNICAL FIELD

This disclosure relates to computer systems, and more particularly, to file system quotas.

BACKGROUND

A file system quota (also referred to as a “disk quota” or “storage quota”) is a limit set by a system administrator that restricts certain aspects of file system usage on modern operating systems. A first type of quota, known as a usage quota or block quota, limits the amount of storage space that can be used. A second type of quota, known as a file quota or inode quota, limits the number of files and directories that can be created. File system quotas may be implemented on a per-directory basis.

DETAILED DESCRIPTION

In multi-user computer systems, a file system may be accessed simultaneously by a plurality of users. Similarly, a plurality of users may simultaneously access a quota capability of the file system. Such access may include enabling and disabling of the quota capability.

Enabling a quota capability may involve establishing quota attributes for directories of the file system. Quota attributes can specify a maximum amount of storage space that is permitted to be used for a directory, a maximum number of files that is permitted to be included in a directory. Similarly, disabling the quota capability may involve removing the quota attributes from every directory in the file system. Such removal can take time, particularly in distributed file systems where network latency may introduce delays. Consequently, if a first user submits a first command to disable the quota capability, and a second user subsequently submits a second command to enable the quota capability before the attribute removal process of the disabling has completed, there may be a collision between the establishment of and removal of a quota attribute. For example, even though the second command to enable the quota capability is submitted after the first command to disable the quota capability, an attribute may be removed by the first command after the attribute has been set by the second command. Accordingly, in conventional systems, a user or system administrator should wait until disabling has completed before the quota capability can be safely re-enabled.

Aspects of the present disclosure address the above and other deficiencies of the conventional systems. Described herein is a system and method for handling the disabling and enabling of a file system's quota capability that do not suffer from the above problem. In accordance with one embodiment, a processing device receives a first command to disable a quota capability of a file system, where the quota capability associates quota attributes with directories of the file system. In response to the first command, the processing device initiates execution of one or more instructions to delete a first association of a quota attribute with a directory, where the quota attribute has an attribute name, and where the first association assigns a first value of the quota attribute to a first string, and where the first string comprises a combination (e.g., a concatenation) of the attribute name and a first symbol. Prior to completion of the execution of the one or more instructions, the processing device receives one or more commands to enable the quota capability and assign a second value of the quota attribute to the directory. In response to the one or more commands, the processing device generates a second association of the quota attribute with the directory, where the second association assigns a second value of the quota attribute to a second string, and where the second string comprises a combination (e.g., a concatenation) of the attribute name and a second symbol.

In one example, the first symbol comprises one or more characters representing an integer N, and the second symbol comprises one or more characters representing an integer N+1. For example, the first association of the quota attribute with a directory might be represented by the string “<attribute name>0.1=<value1>”, where <attribute name> denotes the name of the quota attribute (e.g., not the literal string “<attribute name>”) and <value1> denotes a first value assigned to the quota attribute for the particular directory (e.g., not the literal string “<value1>”). Similarly, the second association of the quota attribute with the directory might be represented by the string “<attribute name>0.2=<value2>”, where <value2> denotes a second value assigned to the quota attribute for the particular directory (e.g., not the literal string “<value2>”). It should be noted that in some implementations, the symbol may be combined with the attribute name in an alternative manner (e.g., by prepending the symbol rather than appending the symbol, by using a separator other than “.”, by concatenating without a separator, by embedding the symbol within the attribute name, etc.).

In one embodiment, a counter is incremented each time the quota capability is enabled, and the counter is used to generate successive versions of quota attributes. In some implementations, a character string corresponding to the current value of the counter is used as the symbol concatenated with the attribute name. For example, the first time the quota capability is enabled, quota attributes for a directory might be represented by the strings “<attribute name>.1”; the second time the quota capability is enabled the quota attributes for the directory might be represented by “<attribute name>.2”; the tenth time the quota capability is enabled the quota attribute for the directory might be represented by “<attribute name>.10”; and so on.

As noted above, disabling of the quota capability may involve removing quota attributes from every directory in the file system, which can take time, particularly in distributed file systems where network latency can introduce delays. By employing versioning of the quota attributes, the quota capability may be enabled following a disabling of the quota capability without having to wait for the disabling to complete. In particular, while the disabling removes the existing versions of the quota attributes, the enabling creates new versions of the quota attributes and therefore does not interfere with the removing of the existing versions. Accordingly, the disabling and enabling processes can be executed simultaneously. After the disabling and enabling processes have been completed, there will be exactly one version of each attribute/directory pair. Consequently, the file system will be in a state that allows the versioning technique to be repeated for the next disabling and subsequent re-enabling of the quota capability. It should be noted that the versioning technique also works correctly in instances where the one or more commands to re-enable the quota capability are not submitted until after the disabling process has completed.

In one embodiment, the quota attributes for a directory may include a limit on the size of the directory e.g., 5.0 MB, etc.), the current size of the directory (e.g., 2.4 megabytes [MB], etc.), and the size of the directory's contribution to its parent directory. In one example, the attribute name for the limit is trusted.glusterfs.quota.limit-set, and the value of this attribute comprises a first 16-digit hexadecimal string representing a hard limit (e.g., a maximum that cannot be exceeded) followed by a second 16-digit hexadecimal string representing a soft limit (e.g., a limit that is less than the hard limit and that, when exceeded, results in a warning that the hard limit is close to being hit). In one example, the attribute name for the current size is trusted.glusterfs.quota.size, and the value of this attribute comprises a 16-digit hexadecimal string representing the size in bytes and a 16-digit binary string representing a directory count (e.g., the number of subdirectories contained in the directory plus 1, where the plus 1 counts the directory itself in the directory count. In one example, the attribute name for the directory's contribution to its parent directory is trusted.glusterfs.quota.<parent dir ID>.contri, and the value of this attribute comprises a 16-digit hexadecimal string representing the contribution in bytes and a 16-digit binary string representing a directory count, as described above.

FIG. 1depicts an illustrative system architecture100for various implementations of the disclosure. It should be noted that other architectures are possible, and that the implementation of a system utilizing embodiments of the disclosure are not necessarily limited to the specific architecture depicted byFIG. 1.

The system architecture100may include a distributed file system101coupled to one or more client machines102via a network108. A distributed file system101is a client/server based system that allows clients to access and process data stored on storage servers as if it were on their own computer. The network108may be a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or wide area network (WAN)), or a combination thereof. Network108may include a wireless infrastructure, which may be provided by one or multiple wireless communications systems, such as a wireless fidelity (WiFi) hotspot connected with the network108and/or a wireless carrier system that can be implemented using various data processing equipment, communication towers, etc.

The distributed file system101may be a network attached storage file system that includes a logical volume180and one or more storage server machines140A-B, each of which may control and manage any number of storage resources170A-B. It should be noted that the fact that two server machines140and two storage resources170are depicted inFIG. 1is merely illustrative; distributed file system101may have a fewer or greater number of storage server machines140, and a fewer or greater number of storage resources170.

A storage server machine140A-B may include a network-accessible server-based functionality (e.g., storage server143A-B) or other data processing equipment. The storage server machines140A-B may include, but are not limited to, any data processing device, such as a desktop computer, a laptop computer, a mainframe computer, a personal digital assistant, a rack-mount server, a hand-held device or any other device configured to process data. The storage server machines140A-B of the distributed file system101may be grouped into one or more clusters. A cluster may be a group of linked storage servers143A-B working together closely.

The storage resources170A-B may be mass storage devices, such as magnetic or optical storage based disks, solid-state drives (SSDs) or hard drives, coupled to the storage server machines140A-B (either directly or via the network108). The storage resources170A-B may also include memory such as random access memory (RAM), Flash memory, and so forth. For convenience, some embodiments provided herein may be described with reference to disks or other mass storage devices. However, it should be understood that such embodiments may also apply to memory storage resources.

Storage servers143A-B may each host a local file system that manages the storage of data on storage resources170A-B. Each of the storage resources170A-B may be formatted in accordance with a particular local file system (FS)148A-B. The local FS148A-B may create directories, partitions, logical volumes, and so forth on the storage resources170A-B as well as store data (e.g., files) thereon. Examples of local disk file systems that may be used on disk storage resources include, but are not limited to, EXT3, EXT4, XFS, NTFS, FAT16, FAT32, and so forth. Various flash file systems, tape file systems, optical disc file systems, memory files systems and so on may alternatively be used for other non-disk storage resources (e.g., for memory storage resources).

Storage servers143A-B may each additionally include an instance of a distributed file system (FS) service145. The distributed FS service145interfaces with local file systems148A-B to store files on the storage resources170A-B, as well as allow retrieval of stored files to the client system102. The distributed file system service145instances additionally interface with one another to form the distributed file system101.

The distributed file system services145, and thus the distributed file system101, may store data as files and may include directories, also referred to as folders, which are virtual containers within the distributed file system101, in which groups of computer files and possibly other directories may be kept and organized. The distributed file system101may organize data (e.g., files) in the disks170A-B using volumes. A volume may be a single accessible storage area of the distributed file system101that may be resident on a single partition or directory of a hardware resource170A-B (e.g., of a hard disk) of the file system101. A volume may be a representation of a logical location, rather than a physical location, of a storage area in the file system101. For example, a physical hard disk-1 may have two partitions, partition-1 and partition-2. A volume label “C:” may be assigned to partition-1, and a volume label “D:” may be assigned to partition-2. Hard disk-1 is a physical disk, and “C:” and “D:” are volumes representing logical locations on physical hard disk-1. Additionally, the volume may be a collection of multiple logical and/or physical locations.

In some embodiments, a volume of the distributed file system101may be associated with multiple storage servers and storage resources170A-B. Files of a volume may be stored across the storage resources170A-B. For example, a volume label “C:” may be assigned to a partition or portion of a first storage resource170A and a partition or portion of a second storage resource170B. As such, files of a single volume may be distributed between multiple storage resources170A-B that are managed by multiple storage servers140A-B. Thus, files of a volume may be stored across one or more storage servers. In some embodiments, the files of the volume may be distributed across multiple directories (e.g., a hierarchical path) that are associated with multiple storage servers.

In some embodiments, distributed file system145may include a quota bookkeeper146and a quota enforcer147. The quota bookkeeper146may keep track of changes in the size of files, creation of new files, deletion of files, etc., and may propagate these changes up the directory hierarchy. The quota enforcer147may monitor distributed file system145for violations of soft or hard quotas and take appropriate action (e.g., notify the owner of a directory and/or a system administrator that the quota has been violated, etc.).

Any number of client systems102may include a client file system136to communicate with the storage servers143A-B in the distributed file system101. The client system102may be a computing device such as a server computer, a desktop computer, a set-top box, a gaming console, a television, a portable computing device such as, and not limited to, mobile telephones, personal digital assistants (PDAs), portable media players, netbooks, laptop computers, electronic book reader, and the like. The client machines102may host one or more applications134. An application134may be any type of application including, but not limited to, a web application, a desktop application, a browser application, etc. An application134may request access (e.g., read, write, etc.) to the data in the distributed file system101. For example, the application134may request to read or write files stored in a volume associated with storage resources170A-B.

In one embodiment, distributed file system101may include an admin daemon190(e.g., a background process, etc.) that is capable of receiving commands from a system administrator (e.g., via a shell command console window, via a graphical user interface (GUI) of an administration application, etc.). The commands may include creating directories, deleting files, navigating through the file system (e.g., via a change directory command), enabling and disabling a quota capability, setting values of quota attributes, and so forth. In one implementation, admin daemon190may include a quota versioning manager195that is capable of managing multiple versions of quota attributes. Some operations of quota versioning manager195are described in detail below with respect toFIG. 2. It should be noted that although implementations of the present disclosure are described in the context of distributed file systems, implementations of the present disclosure may also be employed for other types of file systems (e.g., centralized file systems, etc.).

FIG. 2depicts a flow diagram of one embodiment of a method200by which a computer system handles disabling and enabling of file system quotas. The method is performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both. In one embodiment, the method is performed by the computer system100ofFIG. 1(e.g., by quota versioning manager193), while in some other embodiments, some or all of the method might be performed by another machine. It should be noted that in some embodiments blocks depicted inFIG. 2may be performed simultaneously or in a different order than that depicted.

At block201, a counter V is initialized to zero, and a flag quota is initialized to the value off. Counter V may represent the current version of quota attributes. At block202, a command is received to enable a quota capability of a file system (e.g., of distributed filed system101ofFIG. 1, etc.). The command may be submitted by a system administrator, a user, an automated script that runs according to a schedule, etc. At block203, one or more commands are received to set one or more quota attributes of one or more directories of the file system. It should be noted that the one or more commands received at block203may be submitted by the same entity that submitted the command at block202, or may be received by some other entity. It should be further noted that in some implementations, the quota capability may associate quota attributes with one or more volumes of a file system, either in addition to or instead of the directory quotas. Similarly, in some implementations, the quota capability may also associate quota attributes with individual files.

At block204, counter V is incremented and flag quota is set to on. At block205, new version-V quota attributes are created by combining counter V with each attribute name (e.g., appending “.V” to the end of each quota attribute name, etc.), indicating that the quota attributes are version V. At block206, values of the new quota attributes are set in accordance with the command(s) received at block203.

A command to disable the quota capability is subsequently received at block207(e.g., at some point while the quota capability is enabled, etc.). The command may be submitted by a system administrator, a user, an automated script, etc., and may be the same as, or different than, the entity (or entities) that submitted the enabling commands of blocks202and203.

At block208, flag quota is set to off in response to the command. At block209, one or more instructions are initiated to remove the version-V quota attributes. In some examples, a process may be spawned to execute the one or more instructions, while in some other examples a thread may be created to execute the one or more instructions.

As noted above, in some other implementations another type of naming scheme might be used for modifying quota attribute names. For example, rather than a counter, a flag may be toggled back and forth (e.g., the first version of an attribute name might be “<attribute name>0.0”, the second version of the attribute name might be “<attribute name>0.1”, the third version of the attribute name might be “<attribute name>0.0”, the fourth version of the attribute name might be “<attribute name>0.1”, and so on). Similarly, in some examples the symbols used to modify the quota attribute names might be something other than numerals (e.g., “<attribute name>.a”, “<attribute name>.b”, etc.), whether used in a toggling fashion or in a sequential ordering. As noted above, in some implementations the naming scheme might combine attribute names with symbols in some fashion other than concatenation.

The illustrative computer system300includes a processing system (processor)302, a main memory304(e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM)), a static memory306(e.g., flash memory, static random access memory (SRAM)), and a data storage device316, which communicate with each other via a bus306.

The computer system300may further include a network interface device322. The computer system300also may include a video display unit310(e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device312(e.g., a keyboard), a cursor control device314(e.g., a mouse), and a signal generation device320(e.g., a speaker).

The data storage device316may include a computer-readable medium324on which is stored one or more sets of instructions326(e.g., instructions corresponding to the method ofFIG. 2, etc.) embodying any one or more of the methodologies or functions described herein. Instructions326may also reside, completely or at least partially, within the main memory304and/or within the processor302during execution thereof by the computer system300, the main memory304and the processor302also constituting computer-readable media. Instructions326may further be transmitted or received over a network via the network interface device322.

The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. In addition, embodiments of the present disclosure are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the disclosure as described herein.

Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus. Embodiments of the present disclosure may be provided as a computer program product, or software, that may include a machine-readable medium having stored thereon instructions, which may be used to program a computer system (or other electronic devices) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer). For example, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., a computer) readable storage medium (e.g., read only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices, etc.), a machine (e.g., computer) readable transmission medium (electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.)), etc.