Unified and flexible control of multiple data center cooling mechanisms

Techniques are described for controlling the climate in a data center. Using the input of an administrator, multiple desired attributes of a data center (e.g., temperature, energy consumption, costs, or system performance) may be balanced using a utility function that maximizes the utility of the computing systems in the data center according to the administrator's preferences. Additionally, a cooling model is generated that represents the affect of a control parameter (e.g., the fan speed of a CRAC) on the desire attributes of the data center. The cooling model may then be substituted into the utility function to replace the desired attributes. Using this new utility function, the control parameters may be varied such that the maximum utility is achieved.

BACKGROUND

Many data centers are typically cooled by operation of one or more different types of air conditioning units. Primarily, Computer Room Air Conditioning (CRAC) units and water-based cooling distribution units (CDU) perform the majority of the cooling needs. However, a substantial percentage of existing data centers will have insufficient power and cooling capacity in the near future. Even if this increasing need is met, power is one of the highest operating costs (after labor) in the majority of all data centers. Moreover, data centers are responsible for the emission of tens of million of metric tons of carbon dioxide emissions annually.

A data center may be defined as a location that houses numerous IT devices that contain printed circuit (PC) board electronic systems arranged in a number of racks. A standard rack may be configured to house a number of PC boards, e.g., about forty boards. The PC boards typically include a number of components, e.g., processors, micro-controllers, high-speed video cards, memories, semiconductor devices, and the like, that emanate relatively significant amounts of heat during operation. For example, a typical PC board comprising multiple microprocessors may consume approximately 250 W of power. Thus, a rack containing forty PC boards of this type may consume approximately 10 KW of power.

The power required to dissipate the heat produced by the components in the racks is generally equal to about 30 percent of the power needed to operate the components. However, the power required to dissipate the heat produced by a plurality of racks in a data center is equal to about 50 percent of the power needed to operate the components in the racks. The disparity in the amount of power required to dissipate the various heat loads between racks and data centers stems from, for example, the additional thermodynamic work needed in the data center to cool the air. In one respect, racks are typically cooled with fans that operate to move cooling fluid, e.g., air, across the heat emanating components; whereas, data centers often implement reverse power cycles to cool heated return air. In addition, various cooling mechanisms have different cooling efficiencies. For example, water-cooling units operate more efficiently than air-cooling units, but are costlier to install. The additional work required to achieve the temperature reduction, in addition to the work associated with moving the cooling fluid in the data center and the condenser, often add up to the 50 percent power requirement. As such, the cooling of data centers presents problems in addition to those faced with the cooling of racks.

SUMMARY

Embodiments of the invention provide a method, system and computer program product for controlling temperature in a computer environment by receiving a plurality of desired attributes for the computing environment, wherein a first desired attribute relates to a first system condition of the computing environment and a second desired attribute relates to a second system condition of the computing environment, wherein the first system condition is temperature. The method, system and computer program further provide at least one control parameter in a computing environment, where the control parameter affects at least the temperature and the second system condition of the computing environment. The method, system and computer program generate a cooling model representing the effect of the control parameter on at least the first and second desired attributes and a utility function for the computing environment based on the cooling model and at least the first and second desired attributes. Finally, the method, system and computer program set a value of the control parameter based on the utility function.

Embodiments of the invention provide a method for controlling a system condition in a computing environment by receiving a desired attribute for the computing environment, wherein the desired attribute specifies a desired value of the at least one system condition and providing a plurality of control parameters in a computing environment, wherein the control parameters affect the at least one system condition of the computing environment. The method generates a cooling model to relate the at least one system condition and the control parameters, where each control parameter manages a different type of cooling system, and where each type of cooling system uses a different cooling technique of affecting the at least one system condition of the computing environment. The method determines a value for each of the control parameters based on the cooling model and operates the cooling systems based on the values to achieve the desired attribute.

DETAILED DESCRIPTION

To meet the future demands of data center cooling, embodiments of the present invention coordinate multiple cooling system types, such as CRAC, CDU or workload schedulers, to determine a unified solution for a particular data center. By collectively considering a plurality of cooling systems, a more efficient solution may be obtained. Moreover, utility functions are used to prioritize an administrator's preferences. Thus, an embodiment of the invention may use an administrator's preferences to determine a maximum utility, and coordinate different methods of cooling to efficiently achieve the desired result.

An embodiment of the present invention prompts an administrator to set desired attributes of a data center, e.g., the maximum temperature or energy consumption. These attributes are then input to a utility function to identify the maximum utility. Additionally, a model is developed that represents the effect of the different control parameters—e.g., the fan speed of a CRAC or the flow rate of liquid from a CDU—on the desired attributes. That is, changing the control parameters affects the system conditions of the data center (i.e., temperature at a sensor or energy consumed by a certain server). The model describes what setting must be used on the control parameters to change the system conditions such that the system conditions satisfy the desired attribute or attributes—e.g., maintaining a temperature below a maximum value. Using this relationship, the model is then substituted into the utility function to yield the maximum utility in terms of the control parameters rather than in terms of the desired attributes. After optimization, the control parameters are assigned the values from the utility function that produce the desired attributes.

In another embodiment, after the control parameters are assigned values, the data center continues to collect information from various sensors to record any changes in the system—i.e., a feedback loop. If, for example, the workload increases and additional heat is emitted, this new information can then be used to determine or implement a new model based on the changed system conditions. The new model is substituted into the utility function to yield the maximum utility in terms of the control parameters. This monitoring feature allows the invention to dynamically maintain the administrator's desired attributes even as the system conditions vary with time.

FIG. 1is a block diagram illustrating a data center100with an integrated cooling system130, according to one embodiment of the invention. As shown,FIG. 1includes various inputs which describe and record the environment of the data center100, an integrated cooling system130, various cooling systems which implement the control parameters assigned by the integrated cooling system130, and a hardware system190which includes any number of servers195and racks198.

A plurality of temperature sensors105may be placed around the data center. Each temperature sensor105may send information directly to the integrated cooling system130, or alternatively, the aggregate information is used to extrapolate a temperature gradient for the data center100. Power meters110are also used to record how different components are consuming energy. For example, each CRAC165or CDU160may have an individual power meter110. The rack and server layout component115records the layout of the servers195and racks195respective to the floor plan of the data center and transmits that layout to the integrated cooling system130. Because an output vent of a CRAC unit165may cool racks198which are closer more efficiently than racks198that are located farther away, the layout aids the integrated cooling system130in determining a cost-effective cooling plan. In addition to temperature sensors105and power meters110, the data center100may include pressure sensors that record the output of each plenum from a CRAC165. The user interface125gives an administrator the ability to communicate with the integrated cooling system130. An administrator uses the user interface125to establish, for example, the acceptable temperature and energy consumption of the data center100. Furthermore, as will be understood by one of ordinary skill in the art, any type of recording or measuring device that performs the functions described herein may be used.

The integrated cooling system130gathers all of the various inputs, determines the proper values for the control parameters, and outputs these parameters to the cooling systems. The details of this function will be discussed in further detail below. The integrated cooling system130includes a cooling model component135, utility component140, optimizer145, and I/O component150. The cooling model component135creates a model that relates the control parameters (i.e., the different values the integrated cooling system130can manipulate to change the desired attributes such as temperature) to the maximum temperature or energy consumption requested by the administrator. In general, the administrator conveys to the integrated cooling system130what settings she wants via the user interface125, and the cooling model component135produces a model relating the control parameters to each setting or attribute. The utility component140performs two functions. First, the utility component140takes each attribute inputted by an administrator and creates a utility function in terms of the combined attributes. This utility function balances the different desired attributes to yield the maximum utility. Second, the utility component140uses the models created by the cooling model component135to change the utility functions from being in terms of the desired attributes to being in terms of the control parameters. This altered utility function is then sent to the optimizer145which uses an optimizing algorithm to output values for the control parameters that result in the greatest utility between the desired attributes.

In one embodiment, the integrated cooling system130may be a program located in memory, which when executed by a processor, controls the system conditions of the data center100by setting control parameters. Though not shown, the program that performs the functions of the integrated cooling system130may be run on one of the servers195of the hardware systems190or be executed on a separate computing system that is located either inside or outside the data center100.

The data center100contains three types of cooling systems. The first, a cooling mechanism155, refers to any kind of fluid-cooling device, whether liquid or air. The rear-door heat exchanger160is an example of a liquid-based cooling mechanism155, while the CRAC165is an air-based cooling mechanism155. With these devices, the integrated cooling system130outputs control parameters such as fan speed or pump pressure to affect the temperature of the data center100. Different types of cooling methods may include the fans on the servers195, embedded cooling systems (e.g., associated with a processor in a server195), heat containment systems, in-row cooling, and overhead cooling. All of these methods and cooling mechanisms155may be used to create a unified and flexible cooling plan.

Second, the integrated cooling system130can output control parameters to the optimized layout component170which then creates an optimized layout for the data center. Instead of changing the amount of cooling, the suggested layout of the optimized layout170may offer a more efficient use of current resources. For example, the optimized layout may tell the administrator to move a rack from one location to another. Alternatively, the layout may specify adding different data center cooling equipment to the layout which will be discussed later. Though not shown inFIG. 1, the optimized layout may be transmitted to an administrator via the user interface125or other similar method.

Third, the hardware tasking system175is another alternative to satisfy an administrator's desired attributes without having to increase power to the cooling mechanisms155. The hardware workload monitor120evaluates the different workloads of the servers195(the connection is not shown inFIG. 1) and sends that information to the integrated cooling system130which then transmits instructions to the hardware tasking system175. These instructions control the function of the workload scheduler180and workload migration component185.

The workload scheduler180delays tasks from a time when the hardware systems190are in demand to a time when the hardware systems190are idle. Moreover, the workload schedules180can either work in conjunction with, or independent from, a scheduler found on the hardware system190itself. Because not all servers195and racks198are cooled equally, the hardware tasking system175also includes a workload migration component185. As an example, a temperature sensor105on one rack198may report a higher temperature than a temperature sensor105on a different rack198based purely on distance from an output plenum of a CRAC165. The integrated cooling system130can output a control parameter to the workload migration component185to shift workload from the hotter rack198to the cooler rack198. As one of ordinary skill in the art will recognize, the workload migration component185can work in tandem with the workload scheduler180to lower the temperature without requiring more energy. Furthermore, as will be understood by one of ordinary skill in the art, any type of cooling system that performs the functions described herein, or any combination thereof, may be used.

FIG. 2illustrates a flow diagram implementing the system described inFIG. 1, according to one embodiment of the invention. At step205, the integrated cooling system130receives as an input the administrator's desired attributes via the user interface125. In many cases, temperature is one of these desired attributes. However, the desired temperature is not limited to a static value. Instead, temperature may be a range, a maximum or minimum, or a function. For example, an administrator may desire that the temperature in a data center100not exceed 80 degrees. Alternatively, the administrator could draw a graphical representation of a function representing desired temperature on the user interface125. Or the user interface125may display the layout of the data center with indicated hot spots and allow the administrator to choose which spots to maintain at a certain temperature. Moreover, the administrator may provide a range from which the integrated cooling system130could extrapolate a function. Besides temperature, an administrator may specify energy, cost, system performance, or expected hardware lifetime as a desired attribute. Each of these possible desired attributes will be discussed in detail.

Corresponding with the rising demand for more cooling is the need for more energy to power the cooling mechanisms155. As discussed previously, energy consumption can increase operating expenses and harm the environment. As such, an administrator may identify temperature and energy consumption as desired attributes. Similarly to temperature, the administrator may input a fixed amount, a maximum or minimum, a range, or a function. The integrated cooling system130can also extrapolate a function from a range or graphical representation.

Cost is another desired attribute which can be implemented in at least two ways: first, the cost of the energy, and second, the cost of the hardware. As to the former, the economic cost of energy may be an important concern to the operation of a data center100. Thus, an administrator can input into the user interface125the cost per unit of energy. Alternatively, the cost may change according to the time of day. In such a case, the administrator inputs the different costs according to the time of the day, and the integrated cooling system130could, for example, extrapolate that information into a combined step function. As for the latter, hardware systems190deteriorate at rate that depends at least in part on temperature. Thus, a hardware device that remains consistently cool is generally expected to have a longer lifespan than one that experiences frequent overheating. An administrator may input an equation representing the replacement costs of hardware according to temperature as a desired attribute. Finally, the two ways of implementing cost may be combined to model the total cost of ownership (TCO). The TCO accounts for operational costs such as water usage for operating a CDU160.

Environmental concerns are a desired attribute related to energy. Given the impact on the environment caused by carbon-dioxide emissions, an administrator may cap the maximum energy consumed by the cooling mechanisms155. For example, this cap could help to achieve a specific environmentally-friendly certification. In such a case, instead of merely increasing the fan speed of a cooling mechanism155, the integrated cooling system130could use the hardware tasking system175to maintain a specified temperature yet circumvent the need for more energy.

Another desired attribute may be system performance. For example, an administrator may require that (1) any request to a database under five gigabytes take less than half a second and (2) the temperature remain within certain range. One of ordinary skill in the art will recognize that servers195with a higher workload produce more heat than servers195with a lighter workload. Therefore, the integrated cooling system130balances these competing attributes using a utility function.

Finally, one of ordinary skill will recognize that the desired attributes can be any combination of attributes discussed previously—e.g., temperature, energy, environmental concerns, cost, system performance, and hardware lifetime—as well as related attributes that were not specifically discussed. For example, relative humidity—the percentage of water vapor that exists in a gaseous mixture of air—may also be modeled as a desire attribute since climate control depends heavily on the percentage of water vapor in the air.

At step210, the desired attributes are formed into utility functions. In this embodiment, utility functions are relationships based on user input that maximize utility. The utility component140first creates a utility function for each desired attribute based on the information provided by the administrator via the user interface125. The utility component140then combines these functions to create a single utility function. In order to provide a greater understanding of this process, a more detailed example is appropriate.

In this example, the desired attributes include energy (E) and temperature (T). Beginning with energy, the utility component takes an administrator's input and creates a utility function based solely on energy.
UE(E)=π(E0−E)  (1)

Equation 1 is a linear utility function (UE(E)) of energy consumed (E) according to some constant (E0) that represents a maximum desired energy. As the energy consumed approaches the value of the maximum desired energy, the utility decreases. Conversely, as energy consumed decreases, the utility increases. Another way of expressing utility is as the administrator's satisfaction. Accordingly, in equation 1, the administrator's satisfaction increases as less energy is consumed. Stated differently, the administrator configures the integrated cooling system130to recognize which energy values make her satisfied or unsatisfied. In equation 1, the administrator would be most satisfied if energy consumed (E) is zero and completely dissatisfied if energy consumed is above the maximum energy (E0). Because the equation is linear, the utility decreases at the same rate that energy consumed increases.

Next, the utility component140extrapolates the utility function for temperature.

Equation 2 is an example of a possible utility function dependent on temperature—i.e., a desired temperature at a single location in the data center100. Unlike the linearity of equation 1, equation 2 models a steep slope near the maximum temperature desired (Tmax). Accordingly, as the temperature at the location (Ti) approaches the maximum temperature, the utility decreases rapidly. This function closely represents the real-world situation where any temperature below a certain point has little impact on hardware, but a temperature slightly above that point disproportionately deteriorates the physical hardware. Thus, the administrator quickly becomes less satisfied (i.e., less utility) as the temperature moves closer to Tmax.

In order to balance both energy and temperature, the utility component140then combines the two utility equations.

Equation 3 is a simplified combination of equation 1 and 2. Moreover, equation 3 illustrates the each desired attribute is balanced against the other. For example, if energy consumed (E) is decreased, the overall utility increases; however, decreasing energy may cause the temperature to rise. Thus, equation 3 demonstrates how the utility component140combines the multiple utility functions to find the maximum utility. In this example, there is a single value for both energy (E) and temperature (T) where utility is maximized. In other examples, the utility function may have multiple solutions which require using an optimization technique to determine the best solution from a known set of solutions.

FIG. 4A-Care a graphical representation of creating a utility model, according to one embodiment. Much like equation 1,FIG. 4Aillustrates a utility function (UE(E)) where utility decrease linearly as energy consumed increases to a maximum possible value (E0). Unlike in equation 2, however, the temperature utility function (UT(T)) inFIG. 4Bshows temperature as a binary or step function. In other words, an administrator is perfectly content when the temperature at a certain location is below 80 degrees (TIEK), but any temperature above 80 degrees is unacceptable.FIG. 4Cis the combination ofFIGS. 4A and 4Binto a single utility function (U(E, T)). As expected, this combined utility function is completely governed by energy consumption until the temperature reaches 80 degrees. In this case, the maximum utility is achieved when as little energy as possible is consumed by the cooling mechanisms155so long as the temperature remains below 80 degrees.

Returning toFIG. 2, at step215the control parameters are correlated with the desired attributes. In general, a “control parameter” is any variable available to the integrated cooling system130that has an effect on a desired attribute. For the cooling mechanisms155, the control parameters may include the fan of a CRAC165or the compressor of a rear-door heat exchanger160. Alternatively, a control parameter may be a high-level instruction, such as a script, sent from the integrated cooling system130to the cooling mechanism155.

With regards to the optimized layout component170, a control parameter may be the optimal location of servers in the data center100to create cold or hot aisles, or alternative positions for perforated floor tiles. “Perforated floor tiles” increase air flow in cold aisles to help cool high density racks and hot spots. Further, a control parameter may include the addition of snorkels. “Snorkels” are Plexiglas casings that enclose at least a portion of a rack198, usually the bottom. The snorkel focuses air from the bottom of the rack up to the servers that are above the top level of the snorkel.

The control parameters for the hardware tasking system175may be much more complex depending on the level of integration between the integrated cooling system130and the hardware systems190. In general, the integrated cooling system130provides control parameters that instruct the workload scheduler180and workload migration component185to either postpone certain tasks or transfer tasks to a more efficiently cooled server198. The workload scheduler180may set a threshold for the amount of time a task may run or disallow certain types of processes. For example, if any task that runs for more than two seconds is known to raise the temperature, the hardware workload monitor120informs the integrated cooling system130of any pending jobs that meet this criterion. The integrated cooling system130may then instruct the workload scheduler180to postpone these jobs. The delayed tasks may then be scheduled to run at a time when the hardware system190is historically known to be idle, or alternatively, the integrated cooling system130can continue to monitor the hardware system190and inform the workload scheduler180to run the tasks once the workload decreases.

The workload migration component185may transfer workload from air-cooled racks to the more efficient liquid-cooled racks. Because the integrated cooling system130has information of the layout of the data center100from the rack and server layout component115, the system130has sufficient knowledge to correctly migrate workload. Alternatively, the workload migration component185may transfer work from a busier server195to an idle server195based on input from the hardware workload monitor120.

In many cases, a more efficient solution may be achieved by a combination of the control parameters discussed above. This could be a combination of (1) different cooling parameters within a cooling system, such as decreasing the fan speed of a CRAC165but increasing the pump pressure of a rear-door heat exchanger160, or (2) control parameters from the three different cooling systems. For example, one CRAC165might be turned completely off if snorkels were installed. Or a server195may be moved to a location closer to an A/C output plenum while simultaneously using the workload migration component185to assign additional jobs to the recently moved server195.

At step215, the cooling model component135uses these various types of control parameters to create models that represent the correlation between the parameters and the desired attributes. In one embodiment, the models are pre-loaded into the cooling model component135. Regarding optimizing the layout of the data center100, each control parameter (e.g., snorkel, perforated tile, or a location of a plenum) could already have a documented effect on temperature. For example, a snorkel enclosing half of a rack decreases the temperature of the servers in the bottom half of the rack by 40 percent and the servers at the top by 15 percent. Thus, the integrated cooling system can create a model for the snorkel's effect on temperature without experimentation. Models may also be obtained through basic physic models or from computational fluid dynamics. One of ordinary skill in the art will recognize that the present invention is not dependent on a particular model; thus, any model that has the characteristics described herein may be used.

In another embodiment, the model is created by experimentation on a particular data center. In such a case, the system conditions (e.g., power used by the hardware system190or temperature at a temperature sensor105) may be held constant while the control parameters are swept to record the consequences on each desired attribute. An example of such an experiment is illustrated byFIG. 5A-5E.

FIG. 5A-5Eare a series of charts demonstrating the effect of two fan speeds (C1and C2) on the temperature at the floor (FIG. 5B) and ceiling (FIG. 5C) of two CRAC units. In other embodiments, the measurements from pressure sensors, power meters, or some combination of sensors or meters may be used to create a model. During this experiment, the power consumed by the hardware system remained essentially constant. InFIG. 5A, the fan powers were swept through three values—100 percent, 60 percent, and 0 percent. All the possible combinations are shown except for when both fan powers were set to zero. The temperatures corresponding to each combination of fan power are recorded inFIGS. 5B and 5C. After running this experiment, the cooling model component135can create a model that correlates the fan speed, i.e., control parameter, to the temperature at the particular location, i.e., a desired attribute. A logical extension of this experiment is to change the power consumed by the hardware system190and sweep the fan powers again which provides the cooling model component135with yet another model. Note that these experimental models may all be done prior to full operation of the integrated cooling system190. In other words, the experimental models could be obtained at one time during the start-up or configuration of a data center100. Thus, whenever the system conditions change during normal operation (i.e., the power consumed by the hardware system increases and therefore emanates additional heat) the cooling model component135already contains the corresponding model.

At step220, the cooling model component135sends the model that correlates each desired attribute to the control parameters to the utility component215. In the basic example illustrated inFIG. 5D, the power consumed by the two fans is recorded. The total energy consumed by C1and C2is found by adding the power consumed by each fan and subtracting the energy consumed by the CDU (FIG. 5E). This sum may then be used to create an energy model based on the control parameters—i.e., the CRAC fan speeds.
E(Θ1,Θ2)=6.34((Θ1)2.75+(Θ2)2.75)).  (4)

Equation 4 illustrates a simplified relationship between energy consumed (E(Θ1, Θ2)) and the fan speeds of C1and C2—Θ1and Θ2respectively. With this model, the desired attribute (e.g., energy) is described in terms of only the control parameters, in this case, the fan speeds. The same can be done for temperature.

Referring back to the utility function of equation 3, this function is only described in terms of the desire attributes—i.e., energy and temperature. But the desired attributes are merely results. Conversely, the control parameters provide the integrated cooling system130with the means to actually achieve these results. Thus, the utility component140simply substitutes each model, which is in terms of the control parameters, into the utility function, which is in terms of the desired results. Equation 5 is an example of such a substitution.
U′(Θ1,Θ2)=U(E(Θ1,Θ2),T(Θ1,Θ2))  (5)

In Equation 5, U′(Θ1, Θ2) is a utility function in the terms of only the control parameters Θ1and Θ2. To achieve this, the previously developed model E(Θ1, Θ2) found in equation 4 was substituted into the original utility function of equation 3. Though not shown, a model that relates temperature to the control parameters Θ1and Θ2is similarly substituted into equation 3. The utility component140now has a combined utility function that is in terms of only the control parameters. Optimizing this function gives the values for the control parameters that result in the desired attributes.

At step225, the optimizer145uses the combined utility function U′(Θ1, Θ2) to discover the values for the control parameters that yield the maximum utility. Accordingly, the optimizer145may use any sort of optimization technique to determine the best values for the control parameters from some set of available alternatives. One of ordinary skill in the art will recognize that the present invention is not dependent on a particular optimization technique and any technique that has the characteristics described herein may be used.

At step230, the I/O component150of the integrated cooling system130transmits the optimized solution to the three different types of cooling systems: cooling mechanisms155, optimized layout component170and hardware tasking system175. Based on the desired attributes or user settings, the optimized solution from the optimizer145may use only one of the cooling systems or some combination thereof. Once the solution is implemented, the established control parameters continue to yield the desired attributes, such as maintaining a temperature range at a sensor by expending as little energy as possible.

However,FIG. 2illustrates a static system that does not change if system conditions change. This implementation succeeds if the workload or the system conditions of a data center100remains unchanged. In some case, however, a dynamic integrated cooling system130is preferred.

FIG. 3illustrates a flowchart that demonstrates a dynamic integrated cooling system, according to one embodiment.FIG. 3is essentially the same asFIG. 2except for the adding steps of monitoring and updating the models—steps335and340. After the optimized solution is transmitted to the various cooling systems at step330, the integrated cooling system130continues to monitor observable variables of the data center100. An “observable variable” is a general term used to describe any change in system conditions that affects a desired attribute. For example, an observable variable may include a change in temperature, workload of the hardware system190, energy consumption due to poorly maintained cooling mechanisms155, or the failure of a cooling mechanism155or server195.

At step335, the integrated cooling system130monitors the observable variables to determine if any change might affect a desired attribute. Advantageously, the integrated cooling system130monitors only the observable variables that affect the desired attributes established by the administrator. For example, if the administrator is concerned only about temperature and hardware failure, and the power meters110detect a rise in energy consumption, the integrated cooling system130may ignore the change since it does not affect a desired attribute.

In another embodiment, if a desired attribute is temperature and a sudden spike is recorded by a sensor335, the integrated cooling system130then updates the model associated with the new temperature. As discussed previously, each model is based on certain system conditions or observable variables. Because the temperature at a sensor has changed, so does the model. At step340, the cooling model component135then sends to the utility component140the model that relates the control parameters to the desired attributes given the new system conditions. In such a case, if the control parameters manage two CRAC units, the fan speeds may have to increase to dissipate the added heat.

Continuing the example above, a spike in temperature may not necessitate increasing the fan speed. Instead, the integrated cooling system130may use the hardware tasking system175and the workload scheduler180to postpone any jobs that have caused the temperature spike. Of course, this solution is dependent upon the appropriate model and the desired attributes originally set by the administrator.

In another example, a portion of the hardware system or a cooling mechanism155may fail. If it is the former, the integrated cooling system130may notice this change either by a decrease of temperature at a sensor105near the damaged hardware or an increase of temperature at sensors105near hardware systems190that are now shouldering this extra burden. In one embodiment, the integrated cooling system130may use the workload migration component185to move the extra workload to a server195that is cooled by a rear-door heat exchanger160(which is more efficient than a CRAC165). If a cooling mechanism155failed instead of a server195, the integrated cooling system130may notice this failure by the feedback loop shown inFIG. 1that connects the cooling mechanism155to the I/O component150, or by a rise in temperature near a rack198that is cooled by the particular cooling mechanism155. In response, the integrated cooling system130may postpone the jobs running on that rack198using the workload scheduler180or use a model that excludes that cooling mechanism155as a control parameter. Presumably, the integrated cooling system130already created this model during a start-up or configuration phase anticipating that a cooling mechanism155may fail.

If the integrated cooling system130has not detected any change in an observable variable, the integrated cooling system130continues to transmit to the various cooling systems the same control parameters (step330). The integrated cooling system130then loops back to step335to again verify that the system conditions remain unchanged.

Although not represented in the flowcharts ofFIG. 2orFIG. 3, the administrator may alter the desired attributes at anytime. Essentially, the integrated cooling system130repeats the steps ofFIG. 2, but uses the changes to the desired attributes. As an example, the price of power may change thereby altering the cost utility function. Combining this new utility function with temperature results in a different combined utility function. When the appropriate models are substituted into this function at step220, the optimizer145also produces a different solution for the control parameters than before. In another example, the administrator may discover that a particular hardware system190fails much more often at the current temperature than any other hardware component. Simply lowering the temperature is one solution but perhaps not the best when considering the cost of energy. Instead, the administrator can leave the desired temperature unchanged (since all the other hardware functions properly at that temperature) and use the cost of replacement for the failing hardware as another desired attribute. Thus, the integrated cooling system130finds the maximum utility between temperature, cost of energy and cost of replacement. Contrary to intuition, the optimal solution may be to keep the temperature constant and continue to replace the failing hardware components. The flexibility of utility functions provides the administrator with another tool to make the most economical decision possible.