Automated storage provisioning and management using a centralized database

An information processing system comprises a centralized database and at least one processing device implementing a selection algorithm for automated provisioning and management of a plurality of storage arrays based on information collected from the storage arrays and stored in the centralized database. The information stored in the centralized database may comprise at least one of storage utilization, performance, network bandwidth and end of service for each of the storage arrays. The selection algorithm is configured to determine optimal placement of data across the storage arrays based on the information stored in the centralized database. For example, the selection algorithm may be configured to determine one or more locations within the plurality of storage arrays to allocate as new storage responsive to a provisioning request.

FIELD

The field relates generally to information processing systems, and more particularly to storage provisioning and management.

BACKGROUND

In information technology (IT) infrastructure, storage systems may be deployed by respective service providers, each providing storage services to multiple tenants. The IT infrastructure used to implement such storage systems may comprise multiple storage arrays and associated processing platforms, including processing platforms comprising distributed virtual infrastructure or other types of cloud infrastructure. Relative to a given company, organization or other enterprise, such service provider storage systems may be internal or external. An example of an internal service provider storage system may comprise an IT group of an enterprise that acts as a service provider to enterprise users. External service providers may comprise entities that are otherwise unrelated to the enterprise but nonetheless provide storage services to the enterprise users.

Arrangements of this type can be problematic in that there is typically little or no global control provided across the IT infrastructure resources used to implement the multiple storage systems of the various service providers. Instead, it is common for each service provider storage system to be separately provisioned and managed locally, without regard to the storage systems of other service providers sharing the IT infrastructure. Moreover, such localized provisioning and management is often insufficiently automated, and therefore relies heavily on labor-intensive manual oversight by administrative personnel.

It is therefore apparent that a need exists for improved techniques for provisioning and management of storage systems in IT infrastructure.

SUMMARY

Embodiments of the present invention provide improved techniques for storage provisioning and management in cloud infrastructure and other types of IT infrastructure. These embodiments can provide centralized and fully automated control of storage provisioning and management across multiple storage arrays in one or more service provider storage systems or other types of storage systems.

In one embodiment, an information processing system comprises a centralized database and at least one processing device implementing a selection algorithm for automated provisioning and management of a plurality of storage arrays based on information collected from the storage arrays and stored in the centralized database. The information stored in the centralized database may comprise at least one of storage utilization, performance, network bandwidth and end of service for each of the storage arrays. The selection algorithm is configured to determine optimal placement of data across the storage arrays based on the information stored in the centralized database. For example, the selection algorithm may be configured to determine one or more locations within the plurality of storage arrays to allocate as new storage responsive to a provisioning request.

The processing device may implement a collection mechanism for controlling collection of the information from the storage arrays for storage in the centralized database by periodically capturing a current state of each storage array in terms of one or more of configuration, storage utilization and performance.

One or more of the illustrative embodiments described herein exhibit improved performance relative to conventional arrangements in the provisioning and management of storage systems. For example, the disclosed arrangements can be used to ensure that data is optimally distributed across multiple storage arrays having different pools of available storage resources at each of a plurality of performance levels.

DETAILED DESCRIPTION

Illustrative embodiments of the present invention will be described herein with reference to exemplary information processing systems and associated computers, servers, storage devices and other processing devices. It is to be appreciated, however, that the invention is not restricted to use with the particular illustrative system and device configurations shown. Accordingly, the term “information processing system” as used herein is intended to be broadly construed, so as to encompass, for example, processing systems comprising private or public cloud computing or storage systems, as well as other types of processing systems comprising physical or virtual processing resources in any combination.

FIG. 1shows an information processing system100configured in accordance with an illustrative embodiment of the present invention. The information processing system100comprises a centralized storage provisioning and management system102coupled to a plurality of service provider storage systems104-1,104-2, . . .104-N. The centralized storage provisioning and management system102is configured to control provisioning and management of the service provider storage systems104utilizing information collected from the service provider storage systems. By way of example, the service provider storage systems104may comprise respective service provider data centers, while the centralized storage provisioning and management system102may comprise an additional data center separate from the service provider data centers104. The functionality of the centralized storage provisioning and management system102may be provided to the service provider storage systems104on a software-as-a-service basis.

The service provider storage systems104are implemented with the centralized storage provisioning and management system102in a common set of IT infrastructure106comprising one or more processing platforms. The IT infrastructure106or portions thereof may be implemented using one or more processing devices of a processing platform. Examples of processing platforms that may form portions of the IT infrastructure106in system100will be described in more detail below in conjunction withFIGS. 3 and 4.

User devices108-1,108-2, . . .108-L utilize storage services provided by the various service provider storage systems104implemented in the IT infrastructure106. The user devices108may be configured to allow their corresponding users to access the IT infrastructure106via connections established over a network. Such a network, although not expressly illustrated inFIG. 1, may comprise, for example, a global computer network such as the Internet, a wide area network (WAN), a local area network (LAN), a satellite network, a telephone or cable network, a cellular network, a wireless network such as WiFi or WiMAX, or various portions or combinations of these and other types of networks.

A given one of the user devices108in the system100may be associated with a single user, or multiple users may be associated with a single device. The term “user” as utilized herein is intended to be broadly construed so as to encompass, for example, a human user associated with a given device or set of devices, an automated user such as an application or other hardware or software component associated with a given device, or any other entity that may control that device.

The centralized storage provisioning and management system102comprises a centralized database110and a centralized control path112implemented apart from the service provider storage systems104. The centralized control path112operates in conjunction with the centralized database110using information stored in that database and characterizing various aspects of the service provider storage systems104. For example, the centralized database110in the present embodiment stores configuration information, resource usage information, provider-to-resource mapping information, and performance information regarding each of the service provider storage systems104.

By way of example, the storage arrays115may comprise storage products such as VNX and Symmetrix VMAX, both commercially available from EMC Corporation of Hopkinton, Mass. A variety of other storage products may be utilized to implement portions of the IT infrastructure106.

The localized control paths116are also referred to herein as local control paths. The centralized control path112is configured to interact with the localized control paths116implemented in respective ones of the service provider storage systems104. The centralized control path112manages the localized control paths116which in turn interface with the respective storage arrays115.

In the present embodiment, the centralized storage provisioning and management system102implements at least one selection algorithm for automated provisioning and management of the storage arrays115based on information collected from the storage arrays via the control paths112and116and stored in the centralized database110. For example, a given selection algorithm implemented by the centralized storage provisioning and management system102may be configured to determine optimal placement of data across the storage arrays115based on the information stored in the centralized database110. Accordingly, the selection algorithm may be configured to determine one or more locations within the storage arrays115to allocate as new storage responsive to provisioning requests, or to implement similar functionality relating to centralized provisioning and management. Examples of selection algorithms suitable for use in embodiments of the invention will be described in greater detail below in conjunction withFIGS. 2, 5 and 6.

The user devices108and portions of the centralized storage provisioning and management system102and service provider storage systems104may be implemented using a variety of different arrangements of what are generally referred to herein as “processing devices.” A given such processing device generally comprises at least one processor and an associated memory, and implements one or more functional modules for controlling certain features of the system100.

The memory may comprise random access memory (RAM), read-only memory (ROM) or other types of memory, in any combination. The memory and other memories disclosed herein may be viewed as examples of what are more generally referred to as “computer program products” storing executable computer program code.

In addition to a processor and a memory, a processing device will generally include a variety of other types of circuitry, such as network interface circuitry that allows the processing device to communicate with other processing devices over one or more networks. The network interface circuitry may comprise one or more conventional transceivers.

The centralized database110, centralized control path112and localized control paths116may be implemented at least in part in the form of software that is stored and executed by respective memory and processor elements of one or more processing devices. Accordingly, as indicated previously, system elements such as elements102,104,106and108may therefore be implemented using one or more processing platforms each comprising multiple processing devices. Such processing platforms may comprise cloud infrastructure of a cloud service provider.

The system100in the present embodiment implements one or more processes to provide centralized storage provisioning and management for the service provider storage systems104of the IT infrastructure102. An example of such a process will be described below in conjunction withFIG. 2, but it is to be appreciated that numerous other types of processes may be used in other embodiments.

It is to be appreciated that the particular set of elements shown inFIG. 1for providing centralized storage provisioning and management for multiple service provider storage systems104in IT infrastructure106is presented by way of example, and in other embodiments additional or alternative elements may be used.

For example, although illustrated in the context of multiple service provider storage systems, techniques disclosed herein can be adapted in a straightforward manner for use with a single server provider storage system, or for use with other types of storage systems not necessarily associated with one or more service providers.

The operation of the system100will now be described in greater detail with reference to the flow diagram ofFIG. 2, which illustrates a set of operations performed by the IT infrastructure106in an illustrative embodiment. The process as shown includes steps200through220. The steps may be performed by at least one processing device that includes a processor coupled to a memory and is implemented in one or more processing platforms of the IT infrastructure106.

Each of the steps in this embodiment involves interaction with the centralized database110of the centralized storage provisioning and management system102, as indicated generally at the right side of each step in the figure, and certain steps such as step212also involve interaction with shadow entries in a shadow database120.

In step200, a request is received for storage resources at a particular performance level. It will be assumed for this embodiment and one or more other embodiments described herein that the system100supports multiple performance levels that include gold, silver and bronze performance levels, although other numbers and types of performance levels can be used. The performance levels can be specified in terms of a wide variety of different parameters, such as input-output operations per second (IOPS), latency, etc. The request received in step200is assumed to be for new storage at the gold performance level.

In step202, one or more storage arrays having storage resources available at the gold performance level are identified.

In step204, any of the storage arrays identified in step202that the particular requesting client cannot access are eliminated from consideration.

In step206, any of the storage arrays remaining after the elimination of step204that cannot provide a designated replication strategy associated with the requesting client are also eliminated from consideration.

In step208, the remaining identified storage arrays are sorted into a list based on array utilization.

In step210, the remaining identified storage arrays are sorted into a list based on storage area network (SAN) utilization.

In step212, the lists resulting from the respective sorting steps208and210are utilized to reserve an amount of storage in one or more of the storage arrays as required in order to satisfy the request. More particularly, in the present embodiment this involves reserving required storage for a particular scheduled execution time. As indicated above, this process may involve utilizing shadow entries from a shadow database120that is maintained in association with the centralized database110.

In step214, the process waits until the scheduled time referred to in step212.

In step216, automation is performed in order to allocate the reserved storage resources. This automation generally comprises one or more automated provisioning operations, and serves to automatically configure one or more of the storage arrays to provide the reserved amount of storage at the scheduled time.

In step218, additional time passes during which the requesting client is utilizing the allocated storage resources.

In step220, previously-collected statistics from the centralized database110relating to the storage resources are updated to reflect the latest usage patterns.

Collection of information for storage in the centralized database110may involve use of a collection mechanism which may utilize the control paths112and116. An exemplary such control mechanism may be implemented at least in part in the form of software and may be configured for controlling collection of the information from the storage arrays115for storage in the centralized database110by periodically capturing a current state of each of the storage arrays in terms of one or more of configuration, storage utilization and performance. Other characteristics of the storage arrays and their various operations may be collected in this manner, as required in a given embodiment. The information stored in the centralized database110may comprise, for example, configuration information for each of the storage arrays115and any associated SAN switches.

As a more particular example, the information stored in the centralized database110may include information such as storage utilization, performance, network bandwidth and end of service indicators for each of the storage arrays115of the IT infrastructure106. In such an arrangement, the shadow database120associated with the centralized database110may be configured to store historical information relating to storage utilization, performance, network bandwidth and end of service for each of the storage arrays.

The end of service indicators for the storage arrays will generally indicate the expected lifespan of those arrays. The selection algorithm can be advantageously configured to take such end of service information into account in order to ensure that the provisioned storage timeframe will not exceed the expected lifetime of any of the selected storage arrays, thereby avoiding the need for later data migration.

In some embodiments, the service provider storage systems104may be configured to provide no local provisioning and management functionality for their associated storage arrays115, other than that provided under direct control of the centralized storage provisioning and management system102. This allows the service provider storage systems104to essentially offload all of their storage provisioning and management functionality to the centralized storage provisioning and management system102. In such arrangements, all provisioning and management requests relating to the service provider storage systems104are passed through the centralized system102.

In order to facilitate centralized control of the service provider storage systems104, the centralized storage provisioning and management system102may be configured to associate user identifiers with respective tenants of each of the service provider storage systems104and to control user access to those storage systems in a manner that allows the centralized storage provisioning and management system102to more accurately track resource usage by the service provider storage systems104. Such an arrangement facilitates the implementation of chargebacks and trending. In addition, the centralized storage provisioning and management system102can be configured to require users to login with appropriate credentials and access privileges.

The particular processing operations and other system functionality described in conjunction with the flow diagram ofFIG. 2are presented by way of illustrative example only, and should not be construed as limiting the scope of the invention in any way. Alternative embodiments can use other types of processing operations for providing centralized storage provisioning and management for multiple service provider storage systems.

For example, the ordering of the process steps may be varied in other embodiments, or certain steps may be performed concurrently with one another rather than serially. Also, one or more of the process steps may be repeated periodically or on an as-needed basis responsive to particular events. As mentioned above, the steps of theFIG. 2process are assumed to be implemented in a processing platform comprising at least one processing device having a processor coupled to a memory.

It is to be appreciated that centralized storage provisioning and management functionality such as that described in conjunction with the flow diagram ofFIG. 2can be implemented at least in part in the form of one or more software programs stored in memory and executed by a processor of a processing device such as a computer or server. As mentioned previously, a memory or other storage device having such program code embodied therein is an example of what is more generally referred to herein as a “computer program product.”

The embodiments described in conjunction withFIGS. 1 and 2can provide a number of significant advantages relative to conventional practice. For example, a given such embodiment can be configured to ensure that the service provider storage systems104are deployed in an optimal manner across the resources of the IT infrastructure106. The storage provisioning and management is fully automated and policy driven, resulting in a significantly improved allocation of system resources. Moreover, these arrangements allow storage provisioning and management to be provided to the service provider storage systems using a software-as-a-service model.

As indicated above, the disclosed techniques can also provide significant advantages outside of the context of service provider storage systems. More generally, a wide variety of other types of storage systems can be benefit by the automated provisioning and management functionality achieved by utilizing the centralized database110in conjunction with one or more selection algorithms to determine optimal placement of storage across multiple storage arrays or portions thereof, thereby reducing the cost and complexity associated with implementation and maintenance of storage systems.

Referring now toFIG. 3, portions of the information processing system100in this embodiment comprise cloud infrastructure300. The cloud infrastructure300comprises virtual machines (VMs)302-1,302-2, . . .302-M implemented using a hypervisor304. The hypervisor304runs on physical infrastructure305. The cloud infrastructure300further comprises sets of applications310-1,310-2, . . .310-M running on respective ones of the virtual machines302-1,302-2, . . .302-M under the control of the hypervisor304.

Although only a single hypervisor304is shown in the embodiment ofFIG. 3, the system100may of course include multiple hypervisors each providing a set of virtual machines using at least one underlying physical machine.

An example of a commercially available hypervisor platform that may be used to implement hypervisor304and possibly other portions of the cloud infrastructure300of information processing system100in one or more embodiments of the invention is the VMware® vSphere™ which may have an associated virtual infrastructure management system such as the VMware® vCenter™.

The processing platform400in this embodiment comprises a portion of the system100and includes a plurality of processing devices, denoted402-1,402-2,402-3, . . .402-K, which communicate with one another over a network404. The network404may comprise any type of network, such as a WAN, a LAN, a satellite network, a telephone or cable network, or various portions or combinations of these and other types of networks.

The processing device402-1in the processing platform400comprises a processor410coupled to a memory412. The processor410may comprise a microprocessor, a microcontroller, an ASIC, an FPGA or other type of processing circuitry, as well as portions or combinations of such circuitry elements, and the memory412, which may be viewed as an example of a “computer program product” having executable computer program code embodied therein, may comprise RAM, ROM or other types of memory, in any combination.

Also included in the processing device402-1is network interface circuitry414, which is used to interface the processing device with the network404and other system components, and may comprise conventional transceivers.

The other processing devices402of the processing platform400are assumed to be configured in a manner similar to that shown for processing device402-1in the figure.

Multiple elements of information processing system100may be collectively implemented on a common processing platform of the type shown inFIG. 3 or 4, or each such element may be implemented on a separate processing platform.

Referring now toFIGS. 5 and 6, additional illustrative embodiments of the present invention will be described.

In theFIG. 5embodiment, an information processing system500comprises a centralized database510, three storage arrays515-1,515-2and515-3, a SAN switch520, and a selection algorithm525. The selection algorithm525is implemented on at least one processing device, which is not expressly shown in the figure. In this embodiment, each of the storage arrays515maintains pools of storage resources for respective performance levels, illustratively gold, silver and bronze performance levels.

The different pools in each of the storage arrays515contain different amounts of storage resources, and have occupancy levels which vary dynamically over time. Thus, as illustrated in the figure, the gold performance pool for storage array515-1is at 45% occupancy level, while the bronze performance pool for storage array515-3is at 50% occupancy level. Also, in this embodiment the various storage arrays515have different numbers of connections to the SAN switch520. More particularly, the storage arrays515-1,515-2and515-3have 8, 16 and 4 connections, respectively, to the SAN switch520.

The selection algorithm525makes use of several statistics collected from the storage arrays515, including information relating to the occupancy levels and number of SAN connections, in order to choose an optimal placement for new storage. After selection, the centralized database510is updated to reflect the selection. At a later time, the collection mechanism previously described herein is utilized to update the centralized database510such that any associated shadow entries in a shadow database can be removed.

FIG. 6illustrates an arrangement involving a shadow database600. In this embodiment, the selection algorithm625is configured to use a combination of the statistics referred to in theFIG. 5embodiment as well as data from the shadow database600in making decisions regarding selection of storage resources. The shadow database may be implemented in the form of one or more shadow tables arranged within or otherwise associated with the centralized database510. The use of a shadow database helps to reduce the overhead associated with gathering of configuration information.

It should again be emphasized that the above-described embodiments of the invention are presented for purposes of illustration only. Many variations and other alternative embodiments may be used. For example, the disclosed techniques are applicable to a wide variety of other types of information processing systems, processing platform and processing device configurations, centralized databases, centralized and localized control paths, and service provider storage systems. Accordingly, the particular configuration of system and device elements shown inFIGS. 1 and 3-6, and the process shown inFIG. 2, can be varied in other embodiments. Moreover, any assumptions made above in the course of describing the illustrative embodiments should also be viewed as exemplary rather than as requirements or limitations of the invention. Numerous other alternative embodiments within the scope of the appended claims will be readily apparent to those skilled in the art.