Patent Description:
Cooling is a critical consideration in a computer system and data center design. The number of high performance electronics components such as high performance processors packaged inside servers has steadily increased, increasing the amount of heat generated and to be dissipated during the ordinary operations of the servers. The reliability of servers used within a data center decreases if the thermal environment in which they operate is permitted to increase in temperature over time. Maintaining a proper thermal environment is critical for normal operations of these servers in data centers, as well as for maximizing the server performance, reliability and lifetime. It requires more effective and efficient cooling solutions especially in the cases of cooling these high performance servers.

Servers and other high performance electronic components such as central processing units (CPU), graphical processing units (GPU), etc., are usually tightly packaged in clusters of highly integrated chips, boards, or racks to yield very high power and heat densities. In addition to meeting constantly increasing cooling capacity, increasing power density also demands that data centers be able to flexibility increase their electrical power capacity. Improvements in energy and power efficiency are needed to reduce the operating cost of powering high-performance data centers, mitigate the environmental impact and to satisfy sustainability goals and regulatory requirements. Existing solutions to meet the dual demands of increased power density and cooling capability may require the design of a large amount of power and cooling buffers that are difficult to scale or are inefficiently used during the lifetime of the data centers. The power and cooling systems may also not be integrated and controlled to meet short-term fluctuating demands. Thus, there is a need for integrated thermal and electrical system designs that are scalable, reliable, efficient, serviceable, and low cost to meet the thermal and power management needs of high performance electronic components in data centers. In addition, it becomes more important to develop and deploy renewable energy source for powering data centers due to environmental regulations. <CIT> discloses a two-phase cooling system. In <CIT>, the disclosed system includes a primary cooling loop and a heat rejection loop. The primary cooling loop can include a reservoir, a first pump fluidly connected to the primary cooling loop downstream of the reservoir, and a heat sink module fluidly connected to the primary cooling loop downstream of the first pump and upstream of the reservoir. The first pump can draw dielectric coolant from the reservoir and provide a primary flow of pressurized dielectric coolant through the primary cooling loop, with a first portion flowing through the heat sink module and a second portion flowing through a bypass. The heat rejection loop can be fluidly connected to the same reservoir and can include a second pump and a heat exchanger. The second pump can draw dielectric coolant from the reservoir and provide a secondary flow of pressurized coolant through the heat exchanger and back to the reservoir. US patent application <CIT> discloses cooling systems and methods. In <CIT>, the cooling system for a server may include an evaporator that is in thermal communication with the server so that the evaporator absorbs heat generated by the server. An inlet end of a condenser is fluidically connected to an outlet end of the evaporator, whereas an outlet end of the condenser is fluidically connected to an inlet end of the evaporator. A working fluid disposed within the evaporator and the condenser is substantially free of lubricant. The working fluid absorbs heat generated by the server in the evaporator, changing from a liquid phase to a vapor phase. The working fluid releases heat in the condenser, changing from the vapor phase back to the liquid phase. The phase changes of the working fluid in the evaporator and the condenser result in pressure changes sufficient to cause the working fluid to circulate between the evaporator and the condenser in a self-sustaining, phase-change thermodynamic cycle. US patent application <CIT> discloses a phase-change cooling apparatus and phase-change cooling method. In <CIT>, the phase-change cooling apparatus according to an exemplary aspect of the present invention includes heat receiving means; refrigerant liquid driving means for circulating the refrigerant liquid; a first refrigerant flow path in which the refrigerant liquid flowing away from the refrigerant liquid driving means circulates through the heat receiving means and the heat radiating means; a second refrigerant flow path of a flow path shortening the first refrigerant flow path in such a way that a branched refrigerant liquid being at least part of the refrigerant liquid flowing away from the refrigerant liquid driving means toward the heat receiving means circulates without passing through the heat receiving means and the heat radiating means; and control means for controlling a flow rate of a heat-receiving-side refrigerant liquid being a refrigerant liquid flowing into the heat receiving means based on a flow rate of the branched refrigerant liquid.

The present disclosure provides a full system self-regulating architecture.

In a first aspect, an embodiment of the present disclosure provides a liquid cooling apparatus of a data center, comprising: a primary cooling loop having a primary condenser to remove heat from one or more information technology (IT) components using a cooling fluid; a sensor to monitor a vapor pressure of the cooling liquid; a secondary cooling loop having a secondary condenser, the secondary condenser being configured to be connected to close the secondary cooling loop to supplement the primary cooling loop to remove heat from the IT components when the vapor pressure of the cooling liquid exceeds a threshold value; and a power distribution system including a renewable energy source and an energy storage system configured to distribute power to connect the secondary condenser to the secondary cooling loop and to control a cooling capacity of the secondary cooling loop in response to the vapor pressure, where the secondary condenser is disconnected from the secondary cooling loop when the vapor pressure does not exceed the threshold value, and wherein the renewable energy source is configured to charge the energy storage system when the secondary condenser is disconnected; and where the renewable energy source is configured to distribute power to connect the secondary condenser loop to the secondary cooling loop and to control the cooling capacity of the secondary cooling loop in response to the vapor pressure when a power of the renewable energy source is sufficient, and the energy storage system is configured to distribute power to connect the secondary condenser loop to the secondary cooling loop and to control the cooling capacity of the secondary cooling loop in response to the vapor pressure when the power of the renewable energy source is insufficient.

In a second aspect, an embodiment of the present disclosure provides a data center system, comprising: a power distribution system including at least one renewable energy source and at least one energy storage system; a plurality of electronic racks, each electronic rack containing a plurality of server chassis and each server chassis corresponding to one or more servers or IT components, wherein each electronic rack comprises a liquid cooling apparatus of the first aspect.

In a third aspect, an embodiment of the present disclosure provides a method of regulating a heat load and an electrical load of an electronic equipment, comprising: connecting a primary condenser of a primary cooling loop to circulate a cooling liquid to remove heat from the electronic equipment; connecting a renewable energy source to charge an energy storage system; monitoring a vapor pressure of the cooling liquid; connecting a secondary condenser to a secondary cooling loop to supplement the primary cooling loop in removing heat from the electronic equipment when the vapor pressure of the cooling liquid exceeds a threshold value; and distributing power from the renewable energy source and the energy storage system to regulate a cooling capacity of the secondary cooling loop based on the vapor pressure, where connecting a renewable energy source to charge an energy storage system comprises: disconnecting the secondary condenser from the secondary cooling loop when the vapor pressure does not exceed the threshold value, and charging the energy storage system when the secondary condenser is disconnected, and where distributing power from the renewable energy source and the energy storage system to regulate a cooling capacity of the secondary cooling loop based on the vapor pressure comprises: distributing power to connect the secondary condenser loop to the secondary cooling loop and to control the cooling capacity of the secondary cooling loop in response to the vapor pressure when a power of the renewable energy source is sufficient, and distributing power to connect the secondary condenser loop to the secondary cooling loop and to control the cooling capacity of the secondary cooling loop in response to the vapor pressure when the power of the renewable energy source is insufficient.

Disclosed are designs for an integrated thermal and electrical system with self-regulating capabilities to provide enhanced cooling capacity and auxiliary power to support high thermal and power density requirements of computer systems or data centers. In one aspect, the thermal system of the integrated design includes two cooling loops, a primary cooling loop whose cooling capacity is fixed for normal operation and a secondary cooling loop that supplements the primary cooling loop when the fixed cooling capacity of the primary cooling loop is insufficient. Both the primary and secondary cooling loops have a corresponding condenser that acts as a heat exchange unit. Each condenser may have a secondary loop to deliver cooling liquid to electronic racks of the data center to remove heat generated by the servers or other electronic components of the racks. Each condenser may have a primary loop coupled to external cooling liquid supply and return lines to carry fluid to exchange heat with the secondary loop. The secondary loop of each condenser may use a phase change fluid whose vapor pressure varies due to the thermal load of the servers or electronic components. When the secondary cooling loop is connected to supplement the cooling capacity of the primary cooling loop, the system may control the fluid flow rate in the primary loop of the secondary condenser or the airflow rate through the secondary condenser based on the vapor pressure of the phase change fluid to respond to fluctuations in the thermal load.

In one aspect, the electrical system includes a photovoltaic system and a power storage system to provide auxiliary power to supplement the main utility power. The photovoltaic system may power the primary loop of the secondary condenser to control the fluid flowrate or the airflow rate through the secondary condenser based on the vapor pressure of the phase change fluid. The photovoltaic system may charge the power storage system when not powering the secondary cooling loop. The power storage system may be a battery used to power the primary loop of the secondary condenser when the power of the photovoltaic system is unavailable or insufficient. In one aspect, the power storage system may power the primary cooling loop. In one aspect, the power storage system may provide auxiliary power to the electronic components in response to increased power demand of the computer systems or data centers. The integrated thermal and electrical system with self-regulating capabilities provides efficient thermal and power management of high performance IT clusters, increasing cooling performance, cooling reliability, power efficiency, system scalability, sustainability requirements, and lowering cost to meet the demands of high heat and power densities.

In one aspect, the photovoltaic and power storage systems are shared among multiple racks or IT clusters to regulate the cooling capacity and electrical loads of the racks or IT clusters. A controller may receive the vapor pressure to individually connect and control the cooling capacity of the secondary cooling loop to each rack to respond to the variation in the vapor pressure due to its corresponding thermal load. The controller may additionally receive the voltages and availabilities of the photovoltaic and power storage systems for use in conjunction with the information on the vapor pressure from each rack to regulate the power to the primary loop of the secondary condenser of each rack from the photovoltaic system or the power storage system.

In one aspect, a photovoltaic system is dedicated for each rack but the power storage system is shared among multiple racks to regulate the cooling capacity and electrical loads of the racks. A controller may receive the vapor pressure to individually connect and control the cooling capacity of the secondary cooling loop to each rack in response to the variation in the vapor pressure due to its corresponding thermal load. The controller may additionally receive the voltages of the dedicated photovoltaic systems and the shared power storage systems for use in conjunction with the information on the vapor pressure from each rack to regulate the power to the primary loop of the secondary condenser of each rack from its dedicated photovoltaic system or the shared power storage system.

In one aspect, a secondary condenser and a fluid storage system of the secondary cooling loop is shared among multiple racks. The multiple racks also share the photovoltaic system and the power storage system. A controller may receive the vapor pressure from each rack to individually connect and control the cooling capacity of the secondary cooling loop to each rack from the shared secondary condenser and fluid storage system. The fluid vapor line of the secondary cooling loop from each rack is connected to the shared secondary condenser. The return of the shared secondary condenser is connected to the fluid storage system. The liquid fluid supply line of the secondary cooling loop for each rack is connected to the fluid storage system and is regulated by the controller based on the corresponding vapor pressure of the rack in response to its thermal load. The controller may regulate the power to the primary loop of the shared secondary condenser from the photovoltaic system or the power storage system. The power storage system may be charged by the photovoltaic system or the main utility power to power the primary cooling loop or the electronic components of the racks.

<FIG> is a block diagram illustrating an example of a data center or data center unit according to one embodiment. In this example, <FIG> shows a top view of at least a portion of a data center. Referring to <FIG>, according to one embodiment, data center system <NUM> includes one or more rows of electronic racks of information technology (IT) components, equipment or instruments <NUM>-<NUM>, such as, for example, computer servers or computing nodes that provide data services to a variety of clients over a network (e.g., the Internet). In this embodiment, each row includes an array of electronic racks such as electronic racks 110A-110N. However, more or fewer rows of electronic racks may be implemented. Typically, rows <NUM>-<NUM> are aligned in parallel with frontends facing towards each other and backends facing away from each other, forming aisle <NUM> in between to allow an administrative person walking therein. However, other configurations or arrangements may also be applied. For example, two rows of electronic racks may back to back face each other without forming an aisle in between, while their frontends face away from each other. The backends of the electronic racks may be coupled to the room cooling liquid manifolds.

In one embodiment, each of the electronic racks (e.g., electronic racks 110A-110N) includes a housing to house a number of IT components arranged in a stack operating therein. The electronic racks can include a cooling liquid manifold, a number of server slots (e.g., standard shelves or chassis configured with an identical or similar form factor), and a number of server chassis (also referred to as server blades or server shelves) capable of being inserted into and removed from the server slots. Each server chassis represents a computing node having one or more processors, a memory, and/or a persistent storage device (e.g., hard disk), where a computing node may include one or more servers operating therein. At least one of the processors is attached to a liquid cold plate (also referred to as a cold plate assembly) to receive cooling liquid. In addition, one or more optional cooling fans are associated with the server chassis to provide air cooling to the computing nodes contained therein. Note that the cooling system <NUM> may be coupled to multiple data center systems such as data center system <NUM>.

In one embodiment, cooling system <NUM> includes an external liquid loop connected to a cooling tower or a dry cooler external to the building/housing container. The cooling system <NUM> can include, but is not limited to evaporative cooling, free air, rejection to large thermal mass, and waste heat recovery designs. Cooling system <NUM> may include or be coupled to a cooling liquid source that provide cooling liquid.

In one embodiment, each server chassis is coupled to the cooling liquid manifold modularly, such that a server chassis can be removed from the electronic rack without affecting the operations of remaining server chassis in the electronic rack and the cooling liquid manifold. In another embodiment, each server chassis is coupled to the cooling liquid manifold through a quick-release coupling assembly having a server liquid intake connector and a server liquid outlet connector coupled to a flexible hose to distribute the cooling liquid to the processors. The server liquid intake connector is to receive cooling liquid via a rack liquid intake connector from a cooling liquid manifold mounted on a backend of the electronic rack. The server liquid outlet connector is to emit warmer or hotter liquid carrying the heat exchanged from the processors to the cooling liquid manifold via a rack liquid outlet connector and then back to a coolant distribution unit (CDU) within the electronic rack.

In one embodiment, the cooling liquid manifold disposed on the backend of each electronic rack is coupled to liquid supply line <NUM> (also referred to as a room supply manifold) to receive cooling liquid from cooling system <NUM>. The cooling liquid is distributed through a liquid distribution loop attached to a cold plate assembly on which a processor is mounted to remove heat from the processors. A cold plate is configured similar to a heat sink with a liquid distribution tube attached or embedded therein. The resulting warmer or hotter liquid carrying the heat exchanged from the processors is transmitted via liquid return line <NUM> (also referred to as a room return manifold) back to cooling system <NUM>.

Liquid supply/return lines <NUM>-<NUM> are referred to as data center or room liquid supply/return lines (e.g., global liquid supply/return lines), which supply cooling liquid to all of the electronic racks of rows <NUM>-<NUM>. The liquid supply line <NUM> and liquid return line <NUM> are coupled to a heat exchanger of a CDU located within each of the electronic racks, forming a primary loop. The secondary loop of the heat exchanger is coupled to each of the server chassis in the electronic rack to deliver the cooling liquid to the cold plates of the processors. In one embodiment, liquid supply/return lines <NUM>-<NUM> may be connected to the primary loop of the condenser of the primary or secondary cooling loop of the integrated thermal and electrical system with two cooling loops to be discussed.

In one embodiment, data center system <NUM> further includes an optional airflow delivery system <NUM> to generate an airflow to cause the airflow to travel through the air space of the server chassis of the electronic racks to exchange heat generated by the computing nodes due to operations of the computing nodes (e.g., servers) and to exhaust the airflow exchanged heat to an external environment or a cooling system (e.g., air-to-liquid heat exchanger) to reduce the temperature of the airflow. For example, air supply system <NUM> generates an airflow of cool/cold air to circulate from aisle <NUM> through electronic racks 110A-110N to carry away exchanged heat. In one embodiment, the airflow of cool/cold air may be delivered to the condenser of the primary or secondary cooling loop of the integrated thermal and electrical system with two cooling loops to be discussed.

The cool airflows enter the electronic racks through their frontends and the warm/hot airflows exit the electronic racks from their backends. The warm/hot air with exchanged heat is exhausted from room/building or cooled using a separate cooling system such as an air-to-liquid heat exchanger. Thus, the cooling system is a hybrid liquid-air cooling system, where a portion of the heat generated by a processor is removed by cooling liquid via the corresponding cold plate, while the remaining portion of the heat generated by the processor (or other electronics or processing devices) is removed by airflow cooling.

<FIG> is block diagram illustrating an electronic rack according to one embodiment. Electronic rack <NUM> may represent any of the electronic racks as shown in <FIG>, such as, for example, electronic racks 110A-110N. Referring to <FIG>, according to one embodiment, electronic rack <NUM> includes, but is not limited to, CDU <NUM>, rack management unit (RMU) <NUM>, and one or more server chassis 203A-203E (collectively referred to as server chassis <NUM>). Server chassis <NUM> can be inserted into an array of server slots (e.g., standard shelves) respectively from frontend <NUM> or backend <NUM> of electronic rack <NUM>. Note that although there are five server chassis 203A-203E shown here, more or fewer server chassis may be maintained within electronic rack <NUM>. Also note that the particular positions of CDU <NUM>, RMU <NUM>, and/or server chassis <NUM> are shown for the purpose of illustration only; other arrangements or configurations of CDU <NUM>, RMU <NUM>, and/or server chassis <NUM> may also be implemented. In one embodiment, electronic rack <NUM> can be either open to the environment or partially contained by a rack container, as long as the cooling fans can generate airflows from the frontend to the backend.

In addition, for at least some of the server chassis <NUM>, an optional fan module (not shown) is associated with the server chassis. Each of the fan modules includes one or more cooling fans. The fan modules may be mounted on the backends of server chassis <NUM> or on the electronic rack to generate airflows flowing from frontend <NUM>, traveling through the air space of the sever chassis <NUM>, and existing at backend <NUM> of electronic rack <NUM>.

In one embodiment, CDU <NUM> mainly includes heat exchanger <NUM>, liquid pump <NUM>, and a pump controller (not shown), and some other components such as a liquid reservoir, a power supply, monitoring sensors and so on. Heat exchanger <NUM> may be a liquid-to-liquid heat exchanger. Heat exchanger <NUM> includes a first loop with inlet and outlet ports having a first pair of liquid connectors coupled to external liquid supply/return lines <NUM>-<NUM> to form a primary loop. The connectors coupled to the external liquid supply/return lines <NUM>-<NUM> may be disposed or mounted on backend <NUM> of electronic rack <NUM>. The liquid supply/return lines <NUM>-<NUM>, also referred to as room liquid supply/return lines, may be coupled to cooling system <NUM> as described above.

In addition, heat exchanger <NUM> further includes a second loop with two ports having a second pair of liquid connectors coupled to liquid manifold <NUM> (also referred to as a rack manifold) to form a secondary loop, which may include a supply manifold (also referred to as a rack liquid supply line or rack supply manifold) to supply cooling liquid to server chassis <NUM> and a return manifold (also referred to as a rack liquid return line or rack return manifold) to return warmer liquid back to CDU <NUM>. Note that CDUs <NUM> can be any kind of CDUs commercially available or customized ones. Thus, the details of CDUs <NUM> will not be described herein. In one embodiment, the heat exchanger <NUM> in the CDU <NUM> may be the condenser of the primary or secondary cooling loop of the integrated thermal and electrical system with two cooling loops to be discussed.

Each of server chassis <NUM> may include one or more IT components (e.g., central processing units or CPUs, general/graphic processing units (GPUs), memory, and/or storage devices). Each IT component may perform data processing tasks, where the IT component may include software installed in a storage device, loaded into the memory, and executed by one or more processors to perform the data processing tasks. Server chassis <NUM> may include a host server (referred to as a host node) coupled to one or more compute servers (also referred to as computing nodes, such as CPU server and GPU server). The host server (having one or more CPUs) typically interfaces with clients over a network (e.g., Internet) to receive a request for a particular service such as storage services.

(e.g., cloud-based storage services such as backup and/or restoration), executing an application to perform certain operations (e.g., image processing, deep data learning algorithms or modeling, etc., as a part of a software-as-a-service or SaaS platform). In response to the request, the host server distributes the tasks to one or more of the computing nodes or compute servers (having one or more GPUs) managed by the host server. The compute servers perform the actual tasks, which may generate heat during the operations.

Electronic rack <NUM> further includes optional RMU <NUM> configured to provide and manage power supplied to servers <NUM>, and CDU <NUM>. RMU <NUM> may be coupled to a power supply unit (not shown) to manage the power consumption of the power supply unit. The power supply unit may include the necessary circuitry (e.g., an alternating current (AC) to direct current (DC) or DC to DC power converter, battery, transformer, or regulator, etc.,) to provide power to the rest of the components of electronic rack <NUM>.

In one embodiment, RMU <NUM> includes optimization module <NUM> and rack management controller (RMC) <NUM>. RMC <NUM> may include a monitor to monitor operating status of various components within electronic rack <NUM>, such as, for example, computing nodes <NUM>, CDU <NUM>, and the fan modules. Specifically, the monitor receives operating data from various sensors representing the operating environments of electronic rack <NUM>. For example, the monitor may receive operating data representing temperatures of the processors, cooling liquid, and airflows, which may be captured and collected via various temperature sensors. The monitor may also receive data representing the fan power and pump power generated by fan modules and liquid pump <NUM>, which may be proportional to their respective speeds. These operating data are referred to as real-time operating data. Note that the monitor may be implemented as a separate module within RMU <NUM>. In one embodiment, RMU <NUM> or RMC <NUM> may be a controller that monitors the vapor pressure of the phase change fluid in the secondary loop of the condenser of the secondary cooling loop of the integrated thermal and electrical system with two cooling loops to be discussed.

Based on the operating data, optimization module <NUM> performs an optimization using a predetermined optimization function or optimization model to derive a set of optimal fan speeds for fan modules and an optimal pump speed for liquid pump <NUM>, such that the total power consumption of liquid pump <NUM> and fan modules reaches minimum, while the operating data associated with liquid pump <NUM> and cooling fans of the fan modules are within their respective designed specifications. Once the optimal pump speed and optimal fan speeds have been determined, RMC <NUM> configures liquid pump <NUM> and cooling fans of fan modules based on the optimal pump speeds and fan speeds. In one embodiment, RMU <NUM>, RMC <NUM>, or optimization module <NUM> may be a controller that controls the fluid flow rate in the primary loop of the condenser of the secondary cooling loop, or may control the airflow rate through the condenser of the secondary cooling loop, of the integrated thermal and electrical system with two cooling loops to be discussed.

As an example, based on the optimal pump speed, RMC <NUM> communicates with a pump controller of CDU <NUM> to control the speed of liquid pump <NUM>, which in turn controls a liquid flow rate of cooling liquid supplied to the liquid manifold <NUM> to be distributed to at least some of server chassis <NUM>. Similarly, based on the optimal fan speeds, RMC <NUM> communicates with each of the fan modules to control the speed of each cooling fan of the fan modules, which in turn control the airflow rates of the fan modules. Note that each of fan modules may be individually controlled with its specific optimal fan speed, and different fan modules and/or different cooling fans within the same fan module may have different optimal fan speeds.

Note that the rack configuration as shown in <FIG> is shown and described for the purpose of illustration only; other configurations or arrangements may also be applicable. For example, CDU <NUM> may be an optional unit. The cold plates of server chassis <NUM> may be coupled to a rack manifold, which may be directly coupled to room manifolds <NUM>-<NUM> without using a CDU. Although not shown, a power supply unit may be disposed within electronic rack <NUM>. The power supply unit may be implemented as a standard chassis identical or similar to a sever chassis, where the power supply chassis can be inserted into any of the standard shelves, replacing any of server chassis <NUM>. In addition, the power supply chassis may further include a battery backup unit (BBU) to provide battery power to server chassis <NUM> when the main power is unavailable. The BBU may include one or more battery packages and each battery package include one or more battery cells, as well as the necessary charging and discharging circuits for charging and discharging the battery cells.

<FIG> is a block diagram illustrating a processor cold plate configuration according to one embodiment. The processor/cold plate assembly <NUM> can represent any of the processors/cold plate structures of server chassis <NUM> as shown in <FIG>. Referring to <FIG>, processor <NUM> is plugged onto a processor socket mounted on printed circuit board (PCB) or motherboard <NUM> coupled to other electrical components or circuits of a data processing system or server. Processor <NUM> also includes a cold plate <NUM> attached to it, which is coupled to a rack manifold that is coupled to a liquid supply line and/or a vapor return line. In one embodiment, cold plate <NUM> may be a cooling device that is coupled to the secondary loop of the condenser of the primary or secondary cooling loop to be discussed in <FIG>. The liquid supply line may supply the cooling liquid of the phase change fluid to the cooling device. The return line may return the heated vapor of the phase change fluid from the cooling device. A portion of the heat generated by processor <NUM> is removed by the cooling liquid via cold plate <NUM>. The remaining portion of the heat enters into an air space underneath or above, which may be removed by an airflow generated by cooling fan <NUM>.

<FIG> illustrates an example of the thermal, power, and control architecture of an integrated system <NUM> to provide enhanced cooling capability and auxiliary power for a rack of electronic components of a data center according to one embodiment. The integrated system includes the thermal subsystem to remove heat generated by the electronic components, the electrical subsystem to power the thermal subsystem and the electronic components, and the control subsystem to control the operation of the thermal and electrical subsystem in response to fluctuations in the thermal and electrical loads. The electronic components in the rack may include a server <NUM> or other IT components on a server chassis inserted into a corresponding standard shelf of the rack.

The thermal subsystem includes two cooling loops, a primary cooling loop whose cooling capacity is fixed for normal operation and a secondary cooling loop that supplements the primary cooling loop when the fixed cooling capacity of the primary cooling loop is insufficient to remove heat generated by the electronic components. The control subsystem controls or adjusts only the cooling capacity of the secondary cooling loop to respond to fluctuations in the heat load. Both the primary and secondary cooling loops may use a phase change fluid that changes its phase from liquid to vapor when thermal energy is absorbed and from vapor to liquid when thermal energy is dissipated. The vapor pressure of the phase change fluid may correspond to the amount of thermal energy absorbed.

Both the primary and secondary cooling loops have a corresponding condenser that acts as a heat exchanger. Each condenser may have a secondary loop that is coupled to the server chassis in the electronic rack to deliver the cooling liquid to the cooling devices <NUM> or cold plates to remove heat generated by the server <NUM> or other IT components. Each condenser may have a primary loop coupled to external liquid supply and return lines to carry fluid to exchange heat with the secondary loop. For example, the primary cooling loop has a primary condenser <NUM> with a primary loop (not shown) connected to external liquid supply and return lines that recirculate fluid to remove heat carried by the heated vapor of the phase change fluid of a secondary loop, thereby changing the phase of the phase change fluid in the secondary loop from vapor to liquid. The cooled liquid in the secondary loop of the primary condenser <NUM> after the heat exchange is returned to the cooling devices <NUM> through the primary liquid return loop <NUM>.

Analogously, the secondary cooling loop has a secondary condenser <NUM> with a primary loop <NUM> coupled to external liquid supply and return lines that recirculate fluid to remove heat carried by the heated vapor of a secondary loop. In one aspect, the control subsystem controls the cooling capacity of the primary loop <NUM> of the secondary condenser <NUM> by adjusting the fluid flow rate supplied to the primary loop <NUM> through a secondary condenser valve <NUM>. In one aspect, the control subsystem may control the cooling capacity of the primary loop of the secondary condenser <NUM> by adjusting the pumping speed or frequency of a variable-speed pump (not shown) to vary the flow rate of the fluid. In one aspect, the control subsystem may control the cooling capacity of the primary loop of the secondary condenser <NUM> by adjusting the speed of a secondary condenser fan <NUM> to vary the airflow rate to the secondary condenser <NUM>.

The cooled liquid in the secondary loop of the secondary condenser <NUM> after the heat exchange with the primary loop <NUM> is returned to the cooling device <NUM> through the secondary liquid return loop <NUM>. The liquid from the primary liquid return loop <NUM> is combined with the liquid from the secondary liquid return loop <NUM> to form the rack liquid system <NUM> that supplies the phase change fluid to the cooling devices <NUM> in the multiple server chassis in the rack. The rack liquid system <NUM> also returns the phase change fluid from the multiple server chassis through the primary vapor line <NUM> to the primary condenser <NUM> to complete the secondary loop of the primary condenser <NUM>, or returns the phase change fluid through the secondary vapor line <NUM> to the secondary condenser <NUM> to complete the secondary loop of the secondary condenser <NUM>. The secondary vapor line <NUM> is connected to the secondary condenser through the pressure valve <NUM>. The pressure valve <NUM> is closed under normal operation when the vapor pressure is low to prevent the phase change fluid in the secondary vapor line <NUM> from returning to the secondary condenser <NUM>. The pressure valve <NUM> may open to discharge the phase change fluid in the secondary vapor line <NUM> to the secondary condenser <NUM> when the vapor pressure reaches a triggered pressure threshold.

When the phase change fluid in the rack liquid system <NUM> absorbs the heat generated within the server <NUM> from the cooling devices <NUM>, the phase change fluid may change its phase from liquid to vapor. Under normal operating condition, the phase change fluid is returned to the primary condenser <NUM> through the primary vapor line <NUM> to exchange heat with the primary loop of the primary condenser <NUM>. If the cooling capacity of the primary cooling loop is sufficient based on the current rack density, the vapor pressure in the secondary vapor line <NUM> stays below the triggered pressure threshold of the pressure valve <NUM>. As a result, the pressure valve <NUM> is closed and the phase change liquid is not circulated through the secondary loop of the secondary condenser <NUM>. The primary loop <NUM> of the secondary condenser <NUM> may also shut down to save power as the secondary condenser valve <NUM> is closed. In one embodiment, the secondary condenser fan <NUM> is powered off. In one embodiment, the secondary condenser <NUM> is equipped with either the secondary condenser fan <NUM> or the secondary condenser primary loop <NUM>. If the secondary condenser is a liquid cooling condenser using cooling liquid to cool the vapor in the secondary loop, the secondary condenser primary loop <NUM> is used to supply the cooling liquid. If the secondary condenser is an air cooling condenser using cooling airflow to cool the vapor in the secondary loop (such as data center room cooling air), the secondary condenser fan <NUM> is used for blowing the airflow to cool the vapor back to liquid. In this scenario, the secondary condenser primary loop <NUM> is not needed.

The electrical subsystem includes the photovoltaic system <NUM> and the storage system <NUM> to power the primary loop of the secondary condenser <NUM> in response to the thermal load of the rack. The photovoltaic system <NUM> or the storage system <NUM> may power the primary loop of the secondary condenser <NUM> through electrical lines <NUM> to the secondary condenser valve <NUM> or the secondary condenser fan <NUM>. The storage system <NUM> may be a battery based energy storage system that includes one or more battery cells, as well as the necessary charging and discharging circuits for charging and discharging the battery cells. A pressure sensor <NUM> measuring the vapor pressure of the secondary vapor line <NUM> on the discharge side of the pressure valve <NUM> controls the power distribution of the photovoltaic system <NUM> and the storage system <NUM>. During normal operating condition when the primary cooling loop provides sufficient cooling capacity, the pressure valve <NUM> is closed because the vapor pressure in the secondary vapor line <NUM> does not exceed the triggered pressure threshold of the pressure valve <NUM>. A low pressure reading <NUM> from the pressure sensor <NUM> causes an electrical switch to be in the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) switching position to enable the photovoltaic system <NUM> to charge the storage system <NUM>. In this scenario, the photovoltaic system <NUM> is not powering the secondary cooling loop.

When the cooling capacity of the primary cooling loop is not sufficient to remove heat from the secondary loop of the primary condenser <NUM>, such as when the ambient temperature is high, the vapor pressure in the secondary vapor line <NUM> builds up. When the vapor pressure exceeds the triggered pressure threshold of the pressure valve <NUM>, the pressure valve <NUM> opens to discharge the heated vapor of the secondary vapor line <NUM> to the secondary loop of the secondary condenser <NUM> and to the pressure sensor <NUM>. A high pressure reading <NUM> of the vapor pressure from the pressure sensor <NUM> may cause the electrical switch to switch to the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) position to power the primary loop <NUM> of the secondary condenser <NUM>. The secondary condenser valve <NUM> opens to supply cooling fluid to the primary loop <NUM> of the secondary condenser <NUM>, enabling the secondary cooling loop to enhance the cooling capacity of the primary cooling loop. In one embodiment, the secondary condenser fan <NUM> powers on to supply air flow to cool the secondary condenser <NUM>.

In one aspect, the pressure reading <NUM> controls the amount of power flowing to the primary loop <NUM> of the secondary condenser <NUM> to change the volumetric flow rate of the cooling liquid through the secondary condenser valve <NUM> or to change the rate of airflow from the secondary condenser fan <NUM> to regulate the cooling capacity of the secondary cooling loop in response to the vapor pressure of the secondary vapor line <NUM>. In one aspect, the electrical switch may switch to the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) position to use the storage system <NUM> to power the primary loop <NUM> of the secondary condenser <NUM> when the power from the photovoltaic system <NUM> is not sufficient. In one aspect, if the increased power load of the server <NUM> or other IT components of the server chassis causes the pressure reading <NUM> to increase beyond the pressure reading <NUM> associated with a nominal power load, the storage system <NUM> may provide auxiliary power to the IT load as well as powering the secondary cooling loop. In one aspect, the storage system <NUM> may power the primary cooling loop or the primary loop of the primary condenser <NUM>. The integrated thermal and electrical system <NUM> with self-regulating capabilities provides efficient thermal and power management, increasing cooling performance, cooling reliability, power efficiency, system scalability, sustainability requirements, and lowering cost.

<FIG> illustrates an example of the thermal, power, and control architecture of an integrated system <NUM> that shares a photovoltaic system <NUM> and an energy storage system <NUM> to regulate the cooling capacity and electrical loads among multiple racks according to one embodiment. Two racks are shown, a first rack <NUM> and a second rack <NUM>. Each rack has a primary cooling loop and a secondary cooling loop that enhances the cooling capacity of the primary cooling loop. The primary cooling loops and the secondary cooling loop each has a corresponding condenser to perform heat exchange between a primary loop and a secondary loop as described in <FIG>, the detailed operations of which are omitted for sake of brevity. Even though both a valve for a liquid cooling secondary condenser and a fan for an air cooling secondary condenser are shown for both racks in <FIG>, only one type of secondary condenser is needed. Either the valve or the fan may be omitted as discussed with regards to <FIG>.

The first rack <NUM> has a first rack secondary condenser valve <NUM> to adjust the volumetric flow rate of the fluid supplied to the primary loop of its secondary condenser. In one embodiment, the first rack <NUM> has a first rack secondary condenser fan <NUM> to adjust the airflow rate to the secondary condenser. Similarly, the second rack <NUM> has a second rack secondary condenser valve <NUM> to adjust the volumetric flowrate of the fluid supplied to the primary loop of its secondary condenser. In one embodiment, the second rack <NUM> has a second rack secondary condenser fan <NUM> to adjust the airflow rate to its secondary condenser.

During normal operating condition when the primary cooling loop provides sufficient cooling capacity to the first rack <NUM> or the second rack <NUM>, a first rack pressure valve <NUM> or a second rack pressure value <NUM>, respectively, is closed because the vapor pressure in the secondary loop of the secondary condenser does not exceed the triggered pressure threshold. A first rack pressure sensor <NUM> may measure a low pressure value on the first rack pressure reading <NUM> due to the vapor pressure being lower than the triggered pressure threshold for opening the first rack pressure valve <NUM>. A second rack pressure sensor <NUM> may measure a low pressure value on the second rack pressure reading <NUM> due to the vapor pressure being lower than the triggered pressure threshold for opening the second rack pressure valve <NUM>. A controller <NUM> may read the first rack pressure reading <NUM> or the second rack pressure reading <NUM> to cause an electrical switch to be in the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) switching position to enable the photovoltaic system <NUM> to charge the storage system <NUM>. In this scenario, the photovoltaic system <NUM> is not powering the secondary cooling loop of the first rack <NUM> or the second rack <NUM>.

When the cooling capacity of the primary cooling loop of either the first rack <NUM> or the second rack <NUM> is not sufficient to remove heat from the secondary loop of the corresponding primary condenser, the vapor pressure in the secondary loop of the corresponding secondary condenser may exceed the triggered pressure threshold of the first rack pressure valve <NUM> or the second rack pressure valve <NUM>, respectively. The first rack pressure valve <NUM> or the second rack pressure valve <NUM> may open to discharge the heated vapor to the secondary loop of the corresponding secondary condenser. The first rack pressure sensor <NUM> or the second rack pressure sensor <NUM> may produce a high pressure reading.

The controller <NUM> may read the first rack pressure reading <NUM> to switch an electrical switch to the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) position to power the primary loop of the secondary condenser of the first rack <NUM> from the photovoltaic system <NUM>. The first rack secondary condenser valve <NUM> opens to supply cooling fluid to the primary loop of the secondary condenser of the first rack <NUM>, enabling the secondary cooling loop to enhance the cooling capacity of the primary cooling loop of the first rack <NUM>. In one embodiment, the first rack secondary condenser fan <NUM> powers on to supply air flow to cool the secondary condenser of the first rack <NUM>.

Analogously, the controller <NUM> may read the second rack pressure reading <NUM> to switch the electrical switch to the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) position to power the primary loop of the secondary condenser of the second rack <NUM> from the photovoltaic system <NUM>. The second rack secondary condenser valve <NUM> opens to supply cooling fluid to the primary loop of the secondary condenser of the second rack <NUM>, enabling the secondary cooling loop to enhance the cooling capacity of the primary cooling loop of the second rack <NUM>. In one embodiment, the second rack secondary condenser fan <NUM> powers on to supply air flow to cool the secondary condenser of the second rack <NUM>.

The controller <NUM> controls the amount of power flowing to the primary loop of the secondary condenser of the first rack <NUM> or the second rack <NUM> to change the volumetric flow rate of the cooling liquid through the first rack secondary condenser valve <NUM> or the second rack secondary condenser valve <NUM> to regulate individually the cooling capacity of the secondary cooling loop of each rack in response to the vapor pressure measured at the designated locations (e.g., first rack pressure sensor <NUM> or second rack pressure sensor <NUM>) in the respective rack. In one embodiment, the controller <NUM> regulates individually the rate of airflow from the first rack secondary condenser fan <NUM> or the second rack secondary condenser fan <NUM> in response to the vapor pressure in the respective rack. In one aspect, the controller <NUM> may monitor the voltages of the photovoltaic system <NUM> and the storage system <NUM> to switch the electrical switch to the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) and/or the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) positions to use the storage system <NUM> to power the primary loop of the secondary condenser of one or both of the first rack <NUM> and the second rack <NUM> when the power from the photovoltaic system <NUM> is not sufficient. In one aspect, the storage system <NUM> may provide auxiliary power to the IT load as well as powering the secondary cooling loop of the first rack <NUM> or the second rack <NUM>.

<FIG> illustrates an example of a thermal, power, and control architecture of an integrated system <NUM> that provides a dedicated photovoltaic system for each rack but shares an energy storage system among multiple racks to regulate the cooling capacity and electrical loads of the racks according to one embodiment. Two racks are shown, a first rack <NUM> and a second rack <NUM>. The thermal architecture of the first rack <NUM> and second rack <NUM> are similar to that in <FIG>, the detailed operations of which are omitted for sake of brevity.

The electrical architecture includes a first rack photovoltaic system <NUM> dedicated to the first rack <NUM>, a second rack photovoltaic system <NUM> dedicated to the second rack <NUM>, and a storage system <NUM> shared between the first rack <NUM> and the second rack <NUM>. During normal operating condition when the primary cooling loop provides sufficient cooling capacity to the first rack <NUM>, a controller <NUM> may read a low pressure value of the first rack pressure reading <NUM> from the first rack pressure sensor <NUM> to generate a first rack switch control signal <NUM>. The first rack switch control signal <NUM> may cause an electrical switch of the power distribution system of the first rack <NUM> to be in the pol-<NUM> (<NUM>) to pole-<NUM> (<NUM>) switching position to enable the first rack photovoltaic system <NUM> to charge the storage system <NUM>. In this scenario, the first rack photovoltaic system <NUM> is not powering the secondary cooling loop of the first rack <NUM>.

When the cooling capacity of the primary cooling loop of the first rack <NUM> is not sufficient to remove heat from the secondary loop of the primary condenser, the controller <NUM> may read a high pressure value of the first rack pressure reading <NUM> to generate the first rack switch control signal <NUM>. The first rack switch control signal <NUM> may switch the electrical switch of the power distribution system of the first rack <NUM> to the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) position to power the primary loop of the secondary condenser of the first rack <NUM> from the first rack photovoltaic system <NUM>. The first rack secondary condenser valve <NUM> opens to supply cooling fluid to the primary loop of the secondary condenser of the first rack <NUM>, enabling the secondary cooling loop to enhance the cooling capacity of the primary cooling loop of the first rack <NUM>. In one embodiment, the first rack secondary condenser fan <NUM> powers on to supply air flow to cool the secondary condenser of the first rack <NUM>.

Analogously, when the primary cooling loop provides sufficient cooling capacity to the second rack <NUM>, the controller <NUM> may read a low pressure value of the second rack pressure reading <NUM> from the second rack pressure sensor <NUM> to generate a second rack switch control signal <NUM>. The second rack switch control signal <NUM> may cause an electrical switch of the power distribution system of the second rack <NUM> to be in the pol-<NUM> (<NUM>) to pole-<NUM> (<NUM>) switching position to enable the second rack photovoltaic system <NUM> to charge the storage system <NUM>. In this scenario, the second rack photovoltaic system <NUM> is not powering the secondary cooling loop of the second rack <NUM>.

When the cooling capacity of the primary cooling loop of the second rack <NUM> is not sufficient to remove heat from the secondary loop of the primary condenser, the controller <NUM> may read a high pressure value of the second rack pressure reading <NUM> to generate the second rack switch control signal <NUM>. The second rack switch control signal <NUM> may switch the electrical switch of the power distribution system of the second rack <NUM> to the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) position to power the primary loop of the secondary condenser of the second rack <NUM> from the second rack photovoltaic system <NUM>. The second rack secondary condenser valve <NUM> opens to supply cooling fluid to the primary loop of the secondary condenser of the second rack <NUM>, enabling the secondary cooling loop to enhance the cooling capacity of the primary cooling loop of the second rack <NUM>. In one embodiment, the second rack secondary condenser fan <NUM> powers on to supply air flow to cool the secondary condenser of the second rack <NUM>.

The controller <NUM> through the first rack switch control signal <NUM> controls the amount of power flowing to the primary loop of the secondary condenser of the first rack <NUM> to change the volumetric flow rate of the cooling liquid through the first rack secondary condenser valve <NUM> to regulate the cooling capacity of the secondary cooling loop of the first rack <NUM> in response to the first rack pressure reading <NUM>. In one embodiment, the first rack switch control signal <NUM> regulates the rate of airflow from the first rack secondary condenser fan <NUM> in response to the first rack pressure reading <NUM>. Analogously, the controller <NUM> through the second rack switch control signal <NUM> controls the amount of power flowing to the primary loop of the secondary condenser of the second rack <NUM> to change the volumetric flow rate of the cooling liquid through the second rack secondary condenser valve <NUM> to regulate the cooling capacity of the secondary cooling loop of the second rack <NUM> in response to the second rack pressure reading <NUM>. In one embodiment, the second rack switch control signal <NUM> regulates the rate of airflow from the second rack secondary condenser fan <NUM> in response to the second rack pressure reading <NUM>.

In one aspect, the controller <NUM> may monitor the voltages of the storage system <NUM> through the storage system status signal <NUM> and the voltage of the first rack photovoltaic system <NUM> to generate the first rack switch control signal <NUM>. The first rack switch control signal <NUM> may switch the electrical switch of the power distribution system of the first rack <NUM> to the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) position to power the primary loop of the secondary condenser of the first rack <NUM> from the storage system <NUM> when the power from the first rack photovoltaic system <NUM> is not sufficient. Analogously, the controller <NUM> may monitor the voltages of the storage system <NUM> through the storage system status signal <NUM> and the voltage of the second rack photovoltaic system <NUM> to generate the second rack switch control signal <NUM>. The second rack switch control signal <NUM> may switch the electrical switch of the power distribution system of the second rack <NUM> to the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) position to power the primary loop of the secondary condenser of the second rack <NUM> from the storage system <NUM> when the power from the second rack photovoltaic system <NUM> is not sufficient. In one aspect, the storage system <NUM> may provide auxiliary power to the IT load as well as powering the secondary cooling loop of the first rack <NUM> or the second rack <NUM>.

<FIG> illustrates an example of a thermal, power, and control architecture of an integrated system <NUM> that shares a photovoltaic system <NUM> and an energy storage system <NUM> to regulate the cooling capacity and electrical loads among multiple racks in which the racks also share a secondary condenser <NUM> and a fluid storage system <NUM> of a secondary cooling loop according to one embodiment. Each rack may have a dedicated primary cooling loop (not shown) with a corresponding condenser to perform heat exchange between a primary loop and a secondary loop. As in the other architectures of <FIG>, the cooling capacity of the primary cooling loop is fixed for each rack. However, unlike the other architectures, the secondary cooling loop is shared among the racks to enhance the cooling capacity of each primary cooling loop.

For the secondary cooling loop, a controller <NUM> may control a secondary condenser primary loop valve <NUM> to supply cooling fluid to the primary loop of the secondary condenser <NUM>. In one embodiment, the controller <NUM> may control the speed of a secondary condenser fan <NUM> to vary the airflow rate to the secondary condenser <NUM>. The cooled liquid in the secondary loop of the secondary condenser <NUM> after the heat exchange with the primary loop is returned to the fluid storage system <NUM>. The fluid storage system <NUM> acts as a system level liquid buffer for the phase change fluid in the secondary loop of the secondary condenser <NUM>. A secondary liquid supply line <NUM> supplies the phase change fluid from the fluid storage system <NUM> to each rack through a corresponding secondary condenser secondary loop valve <NUM>. A secondary vapor return line <NUM> returns the heated vapor from each rack to the secondary condenser <NUM> to complete the secondary loop of the secondary condenser <NUM>.

Each secondary vapor return line <NUM> is connected to the secondary condenser <NUM> through a pressure valve <NUM>. The pressure valve <NUM> is closed under normal operation when the vapor pressure is low. The pressure valve <NUM> may open to discharge the heated vapor in the secondary vapor return line <NUM> from a rack to the secondary condenser <NUM> when the vapor pressure reaches a trigger pressure threshold. A pressure sensor <NUM> measures the vapor pressure of each secondary vapor return line <NUM> on the discharge side of the pressure valve <NUM>. The controller <NUM> may read the pressure reading <NUM> from each pressure sensor <NUM> to generate a secondary condenser secondary loop control signal <NUM>. The secondary condenser secondary loop control signal <NUM> controls the secondary condenser secondary loop valve <NUM> to regulate the volumetric flow rate of phase change fluid supplied to the secondary liquid supply line <NUM> for a rack in response to the vapor pressure of the corresponding secondary vapor return line <NUM> for the rack. As shown, since there is a shared fluid storage system <NUM> between the racks, each individual secondary liquid supply line <NUM> between the fluid storage system <NUM> and the rack is assembled with a secondary condenser secondary loop valve <NUM>. Then the secondary condenser <NUM> is connected to the fluid storage system <NUM>. Each of the primary condenser dedicated for each rack may be directly connected to the rack liquid cooling distribution directly. Individual control of the cooling capacity of the secondary cooling loop for each rack from a shared secondary condenser <NUM> increases cooling efficiency and performance while lowering the cost.

During normal operating condition when the primary cooling loop for each rack provides sufficient cooling capacity, the pressure valves <NUM> for all of the racks may be closed. The controller <NUM> may read the low pressure values on the pressure readings <NUM> from all of the racks to cause an electrical switch to be in the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) switching position to enable the photovoltaic system <NUM> to charge the energy storage system <NUM>. In this scenario, the photovoltaic system <NUM> is not powering the secondary cooling loop of all the racks.

When the cooling capacity of the primary cooling loop for a rack is not sufficient to remove heat from the secondary loop of the corresponding primary condenser, the vapor pressure in the secondary vapor return line <NUM> for the rack may exceed the triggered pressure threshold of the corresponding pressure valve <NUM>. The pressure valve <NUM> may open to discharge the heated vapor to the secondary condenser <NUM>. The pressure sensor <NUM> for the rack may produce a high pressure reading. The controller <NUM> may read the pressure reading <NUM> for the rack to switch the electrical switch to the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) position to power the primary loop of the secondary condenser <NUM>. The secondary condenser primary loop valve <NUM> opens to supply cooling liquid to the primary loop of the secondary condenser <NUM>. As discussed, the controller <NUM> may also open the secondary condenser secondary loop valve <NUM> to regulate the volumetric flow rate of phase change fluid supplied to the rack in response to the pressure reading <NUM>, enabling the secondary cooling loop to enhance the cooling capacity of the primary cooling loop for the rack. In one embodiment, the first rack secondary condenser fan <NUM> powers on to supply air flow to cool the secondary condenser <NUM>.

In one aspect, the controller <NUM> may monitor the voltages of the photovoltaic system <NUM> and the energy storage system <NUM> to switch the electrical switch to the pole-<NUM> (<NUM>) to pole-<NUM> (<NUM>) position to use the energy storage system <NUM> to power the primary loop of the secondary condenser <NUM> when the power from the photovoltaic system <NUM> is not sufficient. In one aspect, the energy storage system <NUM> may be powered by the main utility power. In one aspect, the energy storage system <NUM> may provide auxiliary power to the IT load as well as powering the secondary cooling loop of the racks. In one aspect, the energy storage system <NUM> may power the primary cooling loops of the racks. The photovoltaic system <NUM> and the energy storage system <NUM> acts as a system level electrical buffer of the power distribution system.

<FIG> is a flow diagram illustrating an example of a method <NUM> for regulating the cooling capacity and electrical load of electronic equipment in a data center or computer system in response to the thermal and power density according to one embodiment. In one embodiment, the method <NUM> may be performed by the integrated thermal and electrical systems <NUM>, <NUM>, <NUM>, <NUM> of <FIG>, <FIG>, <FIG>, or <FIG>. In one aspect, the method <NUM> may be performed utilizing hardware logic, or combinations of hardware logic and programmable registers that store configuration values.

In operation <NUM>, the method <NUM> connects a primary condenser of a primary cooling loop that circulates a cooling fluid to the heat load of the electronic equipment. In one aspect, the cooling fluid may be a phase change fluid. Operation <NUM> may connect the liquid supply line and the vapor return line of a secondary loop of the primary condenser to remove heat from the heat load.

In operation <NUM>, the method <NUM> connects a renewable energy source to charge an energy storage system. In one aspect, the renewable energy source may be a photovoltaic system and the energy storage system may be a rechargeable battery.

In operation <NUM>, the method <NUM> monitors the vapor pressure of the cooling liquid such as the vapor pressure of the phase change fluid at the discharge side of the pressure based valve. The vapor pressure may increase when the cooling capacity of the primary cooling loop is insufficient for the heat load or when there is a high power load associated with the electronic equipment.

In operation <NUM>, the method <NUM> connects a secondary condenser of a secondary cooling loop to the heat load when the vapor pressure exceeds a threshold pressure value. In one aspect, a pressure valve may be triggered to open when the vapor pressure exceeds the threshold pressure value. When the pressure valve opens, a secondary loop of the secondary condenser may be connected to the heat load and a primary loop of the secondary condenser may be connected to a liquid supply line. The cooling capacity of the secondary cooling loop supplements the cooling capacity of the primary cooling loop.

In operation <NUM>, the method <NUM> controls the distribution of power from the renewable energy source and the energy storage system based on the vapor pressure to regulate the secondary cooling loop. In one aspect, if the power from the renewable energy source is sufficient, the renewable energy source may power the secondary cooling loop to control the cooling capacity of the secondary cooling loop in response to the vapor pressure. In one aspect, if the power from the renewable energy source is insufficient, the energy storage system may power the secondary cooling loop to control the cooling capacity of the secondary cooling loop in response to the vapor pressure.

Various configurations, layouts, and assemblies of the integrated thermal and electrical systems with self-regulating capabilities as described provide enhanced cooling capacity and auxiliary power to support high thermal and power density requirements of computer systems or data centers. The integrated design of the primary and secondary cooling loops for the thermal system, the photovoltaic system and the energy storage system for the power distribution system, and the self-regulating control of the thermal and storage systems provide efficient thermal and power management of high performance IT clusters, increasing cooling performance, cooling reliability, power efficiency, system scalability, sustainability requirements, and lowering cost to meet the demands of high heat and power densities.

Claim 1:
A liquid cooling apparatus of a data center, comprising:
a primary cooling loop having a primary condenser (<NUM>) to remove heat from one or more information technology, IT, components using a cooling fluid;
a sensor (<NUM>) to monitor a vapor pressure of the cooling liquid;
a secondary cooling loop having a secondary condenser, the secondary condenser being configured to be connected to the secondary cooling loop to supplement the primary cooling loop to remove heat from the one or more IT components when the vapor pressure of the cooling liquid exceeds a threshold value; and
a power distribution system including a renewable energy source and an energy storage system configured to distribute power to connect the secondary condenser to the secondary cooling loop and to control a cooling capacity of the secondary cooling loop in response to the vapor pressure,
wherein the secondary condenser is disconnected from the secondary cooling loop when the vapor pressure does not exceed the threshold value, and wherein the renewable energy source is configured to charge the energy storage system when the secondary condenser is disconnected; and
wherein the renewable energy source is configured to distribute power to connect the secondary condenser loop to the secondary cooling loop and to control the cooling capacity of the secondary cooling loop in response to the vapor pressure when a power of the renewable energy source is sufficient, and the energy storage system is configured to distribute power to connect the secondary condenser loop to the secondary cooling loop and to control the cooling capacity of the secondary cooling loop in response to the vapor pressure when the power of the renewable energy source is insufficient.