Patent ID: 12248354

Various features and advantages will become more apparent from the following detailed description of exemplary embodiments in conjunction with the drawings.

DETAILED DESCRIPTION

The terminology used in this disclosure should be interpreted broadly within the limits of subject matter eligibility. The terms “disk,” “drive,” and “disk drive” are used interchangeably to refer to non-volatile storage media and are not intended to refer to any specific type of non-volatile storage media. The terms “logical” and “virtual” refer to features that are abstractions of other features such as, for example, and without limitation, tangible features. The term “physical” refers to tangible features that possibly include, but are not limited to, electronic hardware. For example, multiple virtual computers could operate simultaneously on one physical computer. The term “logic,” if used, refers to special purpose physical circuit elements, firmware, software, computer instructions that are stored on a non-transitory computer-readable medium and implemented by multi-purpose tangible processors, and any combinations thereof. Embodiments are described in the context of a data storage system that includes host servers and storage arrays. Such embodiments are not limiting.

Some embodiments, aspects, features, and implementations include machines such as computers, electronic components, optical components, and processes such as computer-implemented procedures and steps. The computer-implemented procedures and steps are stored as computer-executable instructions on a non-transitory computer-readable medium. The computer-executable instructions may be executed on a variety of tangible processor devices, i.e., physical hardware. For practical reasons, not every step, device, and component that may be part of a computer or data storage system is described herein. Those steps, devices, and components are part of the knowledge generally available to those of ordinary skill in the art. The corresponding systems, apparatus, and methods are therefore enabled and within the scope of the disclosure.

FIG.1illustrates a storage system with autonomous power control for management of consumption of renewable energy. The specifically illustrated storage system is a storage array, but other types of storage systems could be used with autonomous power management for consumption of renewable energy. Storage array100is shown with two engines106-1,106-2, but might include any number of engines. Each engine includes disk array enclosures (DAEs)160,162and a pair of peripheral component interconnect express (PCI-E) interconnected compute nodes112,114(aka storage directors) in a failover relationship. Within each engine, the compute nodes and DAEs are interconnected via redundant PCI-E switches152. Each DAE includes managed drives101that are non-volatile storage media that may be of any type, e.g., solid-state drives (SSDs) based on nonvolatile memory express (NVMe) and EEPROM technology such as NAND and NOR flash memory. Each compute node is implemented as a separate printed circuit board (PCB) and includes resources such as at least one multi-core processor116and local memory118. Processor116may include central processing units (CPUs), graphics processing units (GPUs), or both. The local memory118may include volatile media such as dynamic random-access memory (DRAM), non-volatile memory (NVM) such as storage class memory (SCM), or both. Each compute node allocates a portion of its local memory118to a shared memory that can be accessed by all compute nodes of the storage array using direct memory access. Each compute node includes one or more adapters and ports for communicating with host servers150to service IOs from the host servers. Each compute node also includes one or more adapters for communicating with other compute nodes via redundant inter-nodal channel-based InfiniBand fabrics130. Power control subsystems175manage the processing, memory, and storage resources to help achieve sustainable power usage goals such as using relatively more renewable energy and less non-renewable energy compared with unmanaged operation. The power control subsystems175may include one or more of software stored on the managed drives and memory, software running on the processors, hardware, firmware, and any combinations thereof.

Referring toFIGS.1and2, each compute node112,114runs emulations for completing different storage-related tasks and functions. Front-end emulations220handle communications with the host servers150. For example, front-end emulations receive IO commands from host servers and return data and write acknowledgements to the host servers. Each front-end emulation has exclusively allocated resources for servicing IOs from the host servers. The exclusively allocated resources include processor cores, volatile memory, and ports via which the hosts may access the storage array. Back-end emulations228handle communications with managed drives101in the DAEs160,162. Each back-end emulation has exclusively allocated resources for accessing the managed drives. The exclusively allocated resources include processor cores, volatile memory, and ports. Data services emulations222process IOs. For example, data services emulations maintain metadata that maps between logical block addresses of the storage objects to which IOs from the host applications are directed and the physical addresses on the managed drives. The data services emulations maintain the shared memory in which data is temporarily copied to service IOs. For example, Read IOs characterized as “read-miss” prompt a data services emulation to cause a back-end emulation to copy the data from the managed drives into the shared memory. The data services emulation then causes the front-end emulation to return the data to the host application. In the case of a “read-hit” in which the data is already in shared memory when the IO is received, the data is accessed from the shared memory without being copied from the managed drives by the back-end emulation in response to the IO. Write IOs prompt the data services emulations to copy the data into the shared memory, generate a write acknowledgement, and eventually prompt a back-end emulation to destage the data to the managed drives. Each data services emulation has exclusively allocated resources for servicing IOs from the host servers. The exclusively allocated resources include processor cores, volatile memory, and ports via which the hosts may access the storage array. Remote data services emulations221handle communications with other storage systems, e.g., other storage arrays for remote replication and remote snapshot creation. Each remote data services emulation has exclusively allocated resources for servicing IOs from the host servers. The exclusively allocated resources include processor cores, volatile memory, and ports via which the hosts may access the storage array. Each emulation includes a process with multiple worker threads.

Each power control subsystem175manages the emulations running locally on the same compute node via adjustment of task scheduling and management of the hardware resources described above. Power consumption management decisions are made based on information in a data structure containing a “green quality of service” (GQoS) configuration202. The GQoS configuration defines multiple power consumption modes that are dynamically selected and implemented based on dynamically changing forecasted power availability and other variables. Each power control subsystem obtains forecasts of non-renewable energy availability and forecasts of renewable energy availability from energy generators and distributors200. The forecasts may be obtained using a push, pull, or push/pull model and may include use of an API. Each power control subsystem also obtains renewable energy availability information from local renewable energy sources206that are not provided by the energy generators and distributors, e.g., a directly connected solar array or wind power farm. The information is used by the power control subsystem to forecast availability of renewable energy from local renewable energy sources that are not provided by the energy generators and distributors. The non-renewable energy availability forecasts indicate when power from the generators and distributors is expected to be in limited supply, e.g., brownouts or when the energy distribution grid is under strain due to extreme temperatures or natural disasters. The renewable energy availability forecasts, including forecasts for local sources and the generators and distributors, indicate when and how much renewable energy is expected to be available and/or unavailable, e.g., when solar arrays or wind farms are expected to be active and/or inactive and how much of their output is available for consumption by the storage system. The power control subsystem uses the forecasts to dynamically reconfigure storage system resources to increase reliance on renewable energy in a manner consistent with the GQoS configuration.

FIG.3illustrates selection of power control modes. The illustrated example has three distinct power control modes, but any number of power control modes might be implemented. In a maximum power mode302there are no GQoS limits on power consumption by the storage system. There may be limits placed on power consumption by systems outside of GQoS unrelated to renewable energy, but such systems are beyond the scope of the present disclosure. In a middle power mode304the power control subsystem implements CPU throttling and reduces cooling fan speed. Some background tasks are rescheduled to be performed with renewable power if necessary. In a low power mode306there is further CPU throttling and reduced fan speed and the drives are placed in a lower power state. Further, background tasks are rescheduled to be performed with renewable power. CPU throttling may be implemented in different ways such as by reducing CPU clock speed or placing selected cores in a sleep or hibernation state. The degree to which CPUs are throttled and fan speed is reduced for each power mode is specified in the GQoS configuration202. The GQoS configuration also indicates which of the power modes to select and implement based on all energy availability forecasts300. For example, the GQoS configuration might indicate that the low power mode306is implemented when energy from all sources300is in limited supply and that the middle power mode304is implemented when renewable energy is available.

FIG.4Aillustrates power consumption versus IO latency for a given IO workload. IO latency is an indication of the delay between receipt of an IO by the storage system and transmission of a response by the storage system to service the IO, e.g., data in the case of a read IO and an acknowledgment in the case of a write IO. There is an inverse, non-linear relationship between power consumption and IO latency such that storage system performance improves in terms of reduced IO latency by running components that consume more power, but with diminishing returns. In the illustrated example the storage system IO latency is reduced by significantly more per additional watt of power consumed from 1100 watts to 1225 watts than from 1225 watts to 1400 watts. Further, consumption above 1400 watts provides negligible reduction of IO latency. The GQoS configuration can be designed to manage power consumption based on such performance versus power consumption curves by matching power consumption in the middle power mode, for example, with the inflection of efficiency at 1225 watts.

FIG.4Billustrates drive power states. There is an inverse, non-linear relationship between drive power state and drive performance in terms of entry latency, exit latency, relative read throughput, relative read latency, relative write throughput, and relative write latency. The GQoS configuration can be designed to manage drive power consumption based on such drive performance versus drive power consumption curves by using power state 5 for the middle power mode, for example, because power state 5 is associated with an inflection of efficiency.

FIG.5illustrates enabling and disabling of GQoS power control based on date and time in order to achieve storage system performance objectives. The GQoS configuration indicates when it will be enabled and disabled in terms of date and time. For example, GQoS may be enabled during non-business hours such as evenings, weekends, and holidays to reduce power consumption and disabled during regular business hours to maximize performance in terms of IO latency or IOPS. When GQoS is disabled, the storage system consumes a mix of renewable and non-renewable power, as available, and achieves improved performance in terms of IO latency or IOPS at the expense of greater power consumption. When GQoS is enabled, the storage system increases reliance on renewable power to promote sustainable operation.

FIG.6illustrates power control mode selection based on availability of renewable power and whether GQoS is enabled. When GQoS is enabled and renewable power is available, the power control subsystems select and implement the middle power mode. When GQoS is disabled the power control subsystems select the maximum power mode regardless of whether renewable power is available. When GQoS is enabled and renewable power is unavailable, the power control subsystems select and implement the low power mode. When GQoS is enabled and there is a warning of limited availability of power from all sources, the power control subsystems select and implement the low power mode. When GQoS is disabled and there is a warning of limited availability of power from all sources the power control subsystems select and implement one of the power modes as defined by user preferences.

FIG.7illustrates a method for implementing autonomous power management for sustainable storage. Step700is creating and inputting a GQoS configuration for a storage system. Within a data center, each storage system may have a separate and distinct GQoS configuration that differs from the GQoS configurations of other storage systems. As such, storage systems that support lower priority applications may be configured to be more reliant on renewable energy than storage systems that support higher priority applications. Step702is obtaining and/or calculating forecasts of non-renewable energy availability and renewable energy availability. For example, warnings of reduced availability of non-renewable energy may be pushed to the storage system from the generators and distributors. Forecasts of availability of renewable energy may be pulled from the generators and distributors by the storage system. The storage system may calculate forecasts of availability of renewable energy from local sources other than the generators and distributors. Step704is selecting a power mode based on the GQoS configuration and the forecasts of availability of non-renewable and renewable energy. The GQoS configuration defines the power consumption actions to be implemented in each mode and the conditions under which each mode is selected and implemented. Step706is setting fan speed based on the selected power mode. Step708is setting CPU throttling based on the selected power mode. Step710is setting drive power mode based on the selected power mode. Step712is rescheduling background tasks based on the selected power mode. The selected power mode may include only some or none of the power consumption reduction actions associated with steps706through712. For example, if GQoS is disabled, then step706may be setting fan speed to full, step708may be disabling any CPU throttling, step710may include setting the drives to power state 0, and step712may include resumption of background tasks. Steps702through712are performed iteratively so the storage system dynamically implements different power modes in response to changes in availability of renewable and non-renewable energy.

Although advantages are not considered necessary to any embodiments, some embodiments may help decrease reliance on non-renewable energy. Moreover, decreased reliance on non-renewable energy may be accomplished without significantly degrading IO latency and IOPS performance of the storage system and the host applications that are supported by the storage system.

Specific examples have been presented to provide context and convey inventive concepts. The specific examples are not to be considered as limiting. A wide variety of modifications may be made without departing from the scope of the inventive concepts described herein. Moreover, the features, aspects, and implementations described herein may be combined in any technically possible way. Accordingly, modifications and combinations are within the scope of the following claims.