Systems and methods for datacenter thermal management

A thermal management system for cooling a computing device includes a cold aisle, a hot aisle, a radiator, and a plurality of source heat sinks thermally conductively connected to the radiator. The radiator connects the cold aisle to the hot aisle and flows a cooling fluid through an interior volume of the radiator. Each source heat sink is configured to connect to a heat-generating electronic component to thermally conductively connect the heat-generating component to a surface of the radiator.

BACKGROUND

Background and Relevant Art

As information technology equipment has changed, higher cooling capacity solutions are needed to support the computing power for Artificial Intelligence and Machine Learning applications. Air cooling requires airflow to carry away thermal energy from the higher power chips, and a fan failure or blockage can stop airflow to an entire row or rack of devices. Liquid cooled systems using cold plate technology come with a potential failure in leaks causing hardware damage and unsafe working conditions.

BRIEF SUMMARY

In some embodiments, a thermal management system for cooling a computing device includes a cold aisle, a hot aisle, a radiator, and a plurality of source heat sinks thermally conductively connected to the radiator. The radiator connects the cold aisle to the hot aisle and flows a cooling fluid through an interior volume of the radiator. Each source heat sink is configured to connect to a heat-generating electronic component to thermally conductively connect the heat-generating component to a surface of the radiator.

In some embodiments, a thermal management system includes a cold aisle, a hot aisle, a radiator, a plurality of source heat sinks thermally conductively connected to the radiator, and an ambient fan positioned and configured to blow ambient air toward at least one of the source heat sinks. The radiator connects the cold aisle to the hot aisle and flows a cooling fluid through an interior volume of the radiator. Each source heat sink is configured to connect to a heat-generating electronic component to thermally conductively connect the heat-generating component to a surface of the radiator.

In some embodiments, a thermal management system for cooling computing devices includes a cold aisle, a hot aisle, a radiator, and a plurality of server blades. The radiator connects the cold aisle to the hot aisle and flows a cooling fluid through an interior volume of the radiator. Each server blade of the plurality of server blades includes a heat-generating component, a source heat sink, and an ambient fan. The source heat sink is positioned on the heat-generating component and thermally conductively connected to the radiator. The ambient fan is positioned and configured to blow ambient air toward at least one of the source heat sinks.

DETAILED DESCRIPTION

The present disclosure relates generally to systems and methods for protecting electronic devices from thermal damage. More particularly, the present disclosure relates to devices, systems, and methods for cooling server blades using a radiator to air cool select components, such as a central processing unit (CPU) or memory (random access memory) of a server blade. In some embodiments, a central column radiator with a closed cold aisle connection is used in conjunction with ambient fans that blow ambient air over the remainder of the server blade.

Datacenters include a plurality of electronic devices, some of which are computing devices, that all generate thermal energy. The thermal energy needs to be transported away from the electronic devices to prevent damage to the electronic devices and/or protect the integrity of the data stored or computed on the electronic devices. Datacenters use thermal management systems to carry thermal energy away from the electronic devices by liquid cooling, air cooling, or a combination thereof. Even in liquid cooled systems, the liquid cooling may efficiently conduct thermal energy from the electronic devices but ultimately reject the thermal energy from the warmed fluid (e.g., liquid or vapor) into the ambient atmosphere. The warmed ambient air must then be moved from the liquid cooling system and/or the electronic devices to complete the thermal management of the electronic devices, which may be inefficient.

In some embodiments of datacenters and thermal management systems according to the present disclosure, a plurality of heat generating devices are located in an enclosed space and air is moved through a radiator to transfer heat from the heat-generating devices to cool the heat-generating devices. While the present disclosure will describe the use of airflow to cool heat-generating devices directly, such as cooling computing devices, hardware storage devices, networking devices, power supplies, and other electronic devices, it should be understood that the thermal management system may use liquid cooling fluid flow to cool heat sinks of heat-generating components. In some embodiments, the column radiator is subcooled below the ambient air temperature, and select heat-generating components are thermally conductively connected to the radiator to conductively exhaust waste heat from the heat-generating component.

FIG.1is a schematic representation of a conventional datacenter100with a thermal management system102. An example environment in which thermal management systems and methods according to the present disclosure may be used is a server array. In some embodiments, the datacenter100includes server computers104arranged in a row106, where the row contains a plurality of server racks108, each of which contain a plurality of server computers104, power supplies110, networking devices112, and other electronic devices. In some examples, the server computer104is a blade server. In some examples, the server computers are complete computers (e.g., each server computer can function as a standalone computer). In some examples, the server computers104are electronic components that can cooperate to provide scalable computational power.

The server row106can include a row manager114that is in communication with the server racks and/or rack manager116of the server row106. In some embodiments, the row manager114controls computational loads, such as process allocations, of the server racks108and/or server computers104. In doing so, the row manager114may control the amount of heat generated by the server computers104of the server racks108. In some embodiments, the row manager114controls thermal management of the server racks and/or server computers. For example, the row manager114can manage active thermal management for the server racks108and/or server computers104by changing fan speed or by controlling the flow rate of a cooling fluid for liquid cooling systems. In at least one example, the server row106is at least partially cooled by a liquid cooling system that delivers cooling fluid to the server racks108of the server row106. The row manager114is in communication with the cooling fluid pump to change or stop the flow of cooling fluid.

A server rack108can support a plurality of server computers104in the rack. The server computers may each have liquid cooling, such as localized immersion cooling, for at least some electronic components of the server computer, or a cooling plate with recirculating cooling fluid to cool the electronic component(s) of the server computer. In some embodiments, the server computers104or other electronic devices may be air-cooled, utilizing a cold aisle118and a hot aisle120that flow colder air122from the cold aisle118and evacuate hotter air124from the electronic devices through the hot aisle120. The air flows from the cold aisle118to the hot aisle120based on air pressure differentials established by pumps or blowers126of the thermal management system in series with the cold aisle118and the hot aisle120.

In some embodiments, the electronic components, such as server computers104, of the server rack108are connected to a rack manager116. The rack manager116may control power delivery to the server computers104or other electronic components. In some embodiments, the rack manager116may communicate with the server computers104or other electronic components to power cap or throttle the server computers104or other electronic components and manage power draw. The rack manager116, in some embodiments, is also connected to a cooling fluid pump that moves cooling fluid to one or more server computers or other electronic components in the server rack.

A system resource manager128may be connected to the row manager114and/or rack manager(s)116to communicate with the electronic devices, as well as be connected to thermal sensors130,132to measure one or more properties of the thermal management system102. In some embodiments, the resource manager128is the row manager114. In some embodiments, the resource manager128is the rack manager116. In some embodiments, the resource manager128is a dedicated controller.

The system resource manager128includes a processor129and a hardware storage device131. The processor129may receive information from the thermal sensors130,132and communicate with one or more other devices according to instructions stored on the hardware storage device131that cause the processor to perform any of the methods described herein. In some embodiments, the devices in communication with the system resource manager128may receive instructions from the system resource manager128in response to detecting an increase in temperature include a cooling fluid pump, fan, valve, another thermal management device (e.g., blower126), or combinations thereof. For example, the system resource manager128may adjust the flow of cooling fluid by turning on the cooling fluid pump or by actuating a valve to direct airflow.

The hardware storage device131can be any non-transient computer readable medium that may store instructions thereon. The hardware storage device131may be any type of solid-state memory; volatile memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM); non-volatile memory, such as read-only memory (ROM) including programmable ROM (PROM), erasable PROM (ERPOM) or EEPROM; magnetic storage media, such as magnetic tape; a platen-based storage device, such as hard disk drives; optical media, such as compact discs (CD), digital video discs (DVD), Blu-ray Discs, or other optical media; removable media such as USB drives; non-removable media such as internal SATA or non-volatile memory express (NVMe) style NAND flash memory; or any other non-transient storage media.

The air122is provided to the entire row106and/or rack108to cool the ambient air around the components of the rack108. Cooling the entire environment may not be necessary and, in fact, inefficient, when the heat generation by the components of the server computers104and other electronic devices is uneven. For example, a central processing unit (CPU) or system memory of the server computer104may generate considerably greater heat than a non-volatile storage device. To limit and/or prevent thermal damage to the CPU, however, the entire ambient air is cooled based off of the CPU temperature and load. Localized cooling can efficiently cool the hottest areas and/or components without expending additional energy to cool the entire room in which the row106or rack108is located.

In some embodiments, a thermal management system according to the present disclosure provides localized cooling for the greatest heat-generating components of the server blades. A hybrid thermal management system uses a shared radiator to provide a subcooled heat sink in addition to the ambient air. In some embodiments, the radiator has an interior volume through which subcooled air or other gas flows to cool the radiator. In some embodiments, a cooling liquid is pumped through the radiator to cool the radiator. Select heat-generating components, such as processors (e.g., CPU, graphical processing unit (GPU)), system memory (e.g., RAM), network connection devices, and power supplies, are thermally conductively connected to the radiator to conduct heat away from the components to the cold radiator. The rest of the components, which generate less heat, on the motherboard are cooled via ambient air and/or fans to blow ambient air across those components.

FIG.2is a side schematic representation of a thermal management system202including a rack208of server computers or server blades204. A rack manager216or other controller is connected to thermal sensors230and control structures for controlling the flow of air222through the radiator234. As described above, the radiator may be gas-cooled or liquid-cooled. While the present disclosure will describe the thermal management system as using air, it should be understood that other cooling fluids may be used. The air222flowing through the radiator234cools the radiator234. In some embodiments, the radiator234connects the cold aisle218to the hot aisle220. The air222warms as the radiator234receives heat from the heat-generating components, such as the CPU236of each blade204. WhileFIG.2illustrates the radiator234oriented vertically to connect the cold aisle218to the hot aisle220, the radiator234may be positioned at other orientations. For example, at least a portion of the radiator234may be oriented at an angle to the direction of gravity. In some examples, at least a portion of the radiator234may be oriented horizontally. In some examples, at least a portion of the radiator234may be curved or have a corner therein. In at least one example, the radiator234may provide cold air across a plurality of racks208.

The heat-generating component, such as the CPU236, is thermally conductively connected to the radiator234by a thermal conductor. In some embodiments, a thermal conductor is a solid structure that conducts heat from the heat-generating component to the radiator234within convective or radiative thermal interfaces. A thermally conductively connected element does not rely upon heat transfer through the air or other medium, but rather is physically connected to the radiator234.

The thermal conductor includes at least one source heat sink238positioned in contact with the heat-generating component. In some embodiments, a thermal paste244or other interface material is positioned between the source heat sink238and the heat-generating component to fill gaps and/or provide a thermally efficient interface.

The source heat sink238may be thermally conductively connected to a radiator heat sink240in contact with the radiator234. In some embodiments, the source heat sink238may be thermally conductively connected with the radiator234directly, such as having one or more thermal conductors integrally formed with the radiator234. In some embodiments, the radiator heat sink240has an interface material positioned between the radiator heat sink240and the radiator234.

In some embodiments, and as will be described in greater detail below, the source heat sink238is thermally conductively connected to the radiator234and/or radiator heat sink240by a thermal conductor. The thermal conductor may be or include a heat pipe, vapor chamber, solid thermally conductive rods or fins, other thermal conductors, or combinations thereof. InFIG.2, each of the server blades204has a source heat sink238that is thermally conductively connected to a radiator heat sink240by heat pipes242. In some embodiments, the source heat sink238is thermally conductively connected to the radiator heat sink240by a plurality of heat pipes242or other thermal conductors. In some embodiments, the source heat sink238is thermally conductively connected to the radiator heat sink240by a single heat pipe242or other thermal conductor. For example, the number and/or size of the thermal conductor may be related to the distance from the source heat sink238to the radiator234or radiator heat sink240.

FIG.2also illustrates a rank manager216in communication with one or more components of the rack208. In some embodiments, the rack manager216is the resource manager. In some embodiments, a dedicated controller is connected to one or more radiator blowers226and/or ambient fans248. The radiator blower226may be positioned and/or configured to flow air (or other cooling fluid) through the radiator between the cold aisle218and the hot aisle220. In some embodiments, the thermal management system202includes a plurality of radiator blowers226, such as a first radiator blower226proximate the cold aisle218and a second radiator blower226proximate the hot aisle220.

The ambient fans248may be positioned on or proximate to the blade204to blow ambient air across a surface of the motherboard246and/or lower heat-generating components of the server blade204. For example, the server blade204may include non-volatile memory, such as a magnetic hard disk drive. The magnetic platen hard disk drive may require fewer cooling resources than the CPU236, and the ambient air may provide sufficient cooling capacity to cool the magnetic platen hard disk drive without being thermally conductively connected to the radiator234. In some embodiments, such as illustrated inFIG.2, the ambient fan(s)248are positioned and oriented to blow ambient air from the room into the server blade204and toward the radiator234. In at least one embodiment, the ambient fan248is positioned proximate the radiator234to blow ambient air cooled by the surface of the radiator234away from the radiator234across the server blade204and/or motherboard246to cool the motherboard246and components thereon.

The rack manager216or other resource manager controller may be in communication with the thermal sensor(s)230and adjust the radiator blower226and/or ambient fans248according to measurements from the thermal sensor(s)230. For example, the CPU236is passively thermally conductively connected to the radiator234, and additional cooling capacity to the CPU236can be provided by using the radiator234to generate cooler air and to increase the temperature gradient across the thermal conductor (e.g., heat pipes242). If a thermal sensor230positioned on the CPU236or on an interface with the CPU236measures a CPU temperature approaching or exceeding a threshold temperature, the rack manager216or other resource manager controller may send a signal to the radiator blower226to increase the flow of air222through the radiator234from the cold aisle218to further cool the radiator234.

In another example, a thermal sensor230positioned on the motherboard246may measure a motherboard temperature that is approaching or exceeding a threshold temperature. The rack manager216or other resource manager controller may send a signal to the ambient fan(s)248to blow ambient air across the motherboard246and/or other components toward or away from the radiator234to cool the motherboard246and/or other components.

In some embodiments, the radiator heat sink240is selectively connected to the radiator234, allowing the radiator heat sink240and the server blade204to be removed (e.g., disconnected) from the radiator234and/or rack208for replacement or maintenance. For example, the server blade204may slide into the rack208toward the radiator234during installation and/or may be removed from the rack208away from the radiator234. The server blade204may include a source heat sink238and radiator heat sink240installed on the server blade204. When the server blade204is pushed into the rack208, the server blade204may electrically connect to a power supply and/or networking components (such as described in relation toFIG.1) and, at the same time, connect the radiator heat sink240to the radiator234.

To provide efficient convective heat transfer between the air222in the interior volume of the radiator234and the radiator walls, the radiator234may include one or more thermal surface features to increase surface area of an inner or outer surface of the radiator234.FIGS.3-1and3-2are transverse cross-sectional views of an embodiment of a radiator334. In some embodiments, the radiator334has an interior volume350through which the air322from the cold aisle flows. The air322from the cold aisle cools the radiator walls352to which a radiator heat sink or thermal conductor may be connected. The radiator334includes one or more thermal surface features to increase the surface area of the inner surface of the radiator walls352and more efficiently transfer heat between the air322and radiator walls352and cool the radiator walls352. In some embodiments, the thermal surface features include fins354oriented in the longitudinal direction of the radiator334(e.g., the direction of airflow through the interior volume350). By orienting the fins354in the direction of airflow, the resistance to the air322is minimized while exposing a larger surface area of the fins354to the air322.

In some embodiments, the air322warms as the air flows through the radiator334, which decreases the temperature gradient between the air322and the radiator334. A decrease in the temperature gradient can decrease heat transfer rates. To compensate for the warming of the air along the longitudinal length of the radiator334, the thermal surface features can vary in size, shape, or type along the longitudinal direction of the radiator334. For example,FIG.3-2is a transverse cross-sectional view of the same radiator334ofFIG.3-1at a different longitudinal position in the radiator334.FIG.3-2illustrates the radiator334proximate the cold aisle, where the air322is coldest in the radiator334. The fins354are shorter (e.g., protrude into the interior volume350less) than those illustrated inFIG.3-1, and there are less fins354than inFIG.3-1. The increase in surface area ofFIG.3-1relative toFIG.3-2may compensate for the lower temperature gradient and lower heat transfer rate. In other examples, the thermal surface features may taper, move, twist (e.g., a helix), start or stop mid-way along the longitudinal length, or have perforations or surface textures thereon along the longitudinal length of the radiator334to adjust the surface area of the inner surface of the radiator334to balance thermal transfer into the radiator334along the length of the radiator334.

Referring now toFIG.4, a variety of thermal surface features may be used. In some embodiments, the thermal surface features include rods456that extend into the interior volume450of the radiator434. The rods456may be solid rods that provide an increase in surface area for radiator walls452. In some embodiments, the thermal surface features include heat pipes458, which transfer heat efficiently and increase surface area of the radiator walls452.

In at least one embodiment, the radiator434includes an outer thermal surface feature460on an outer surface of the radiator434to cool the ambient air immediately surrounding the radiator434. For example, an outer thermal surface feature460of the radiator434may allow an ambient fan or other fan outside of the radiator434to blow the cooled air (cooled by the outer thermal surface feature460) away from the radiator434and over the motherboard or other components of the server blade. WhileFIG.4depicts an embodiment of a radiator434with a heat pipe outer thermal surface feature460, it should be understood than any thermal surface feature or combination thereof may be used as an outer thermal surface feature460.

FIG.5is a schematic representation of an embodiment of a thermal management system502with a radiator534including two columns562-1,562-2. As described herein, the air522provided from the cold aisle518through the radiator534warms as the air522receives heat from the server blades508and components thereof. In some embodiments, a radiator534includes separate columns562-1,562-2to distribute the air522between the two columns562-1,562-2and isolate the heat transferred to the air522therein. For example, each of the first column562-1and the second column562-2have two of the four radiator heat sinks540connected thereto. Reducing the number of radiator heat sinks540connected to the outer surface of the column562-1,562-2reduces the amount of heat transferred to the air522therein. Further, the radiator heat sinks540are staggered to alternate to which column562-1,562-2each neighboring radiator heat sink540is connected. By longitudinally spacing the radiator heat sinks540connected to each column562-1,562-2, the thermal gradients may be further improved.

In some embodiments, the radiator534is a center column or center radiator534in a rack508. To efficiently use the available space and surface area of the radiator, server blades504may be positioned on and connected to opposite sides of the radiator534, such as illustrated inFIG.5. In some embodiments, such as illustrated in the transverse cross-sectional view (top view) ofFIG.6, a rack608includes radially positioned server blades604around a central radiator634. In some embodiments, the radiator634may provide and/or be the structural support to which the server blades604are connected. By supporting the server blades604through a direct mounting to the radiator634, the radiator634may provide not only cooling, but function as a center spine for the rack608. The air622flows through the center of the radiator634, cooling the radiator634and receiving heat from the radiator heat sinks640connected to the outer surface of the radiator634.

A radial arrangement of server blades may use conventional server blades604staggered radially and longitudinally (e.g., helixed) around the central radiator634. In some embodiments, the server blades604may have wedge-shaped motherboards646to provide a more efficient surface area for components and cooling. A plurality of wedge-shaped server blades604may, when installed on the radiator634, form a complete disc or circle (or other shape) around the radiator634. In some embodiments, a wedge-shaped motherboard646allows for the CPU or other component to which the source heat sink638is connected to be positioned on the motherboard646near the radiator634. Greater freedom in component location on the motherboard646can allow for shorter heat pipes642or other thermal conductor between the source heat sink638and the radiator heat sink640to efficiently transfer heat.

FIG.7throughFIG.9are side cross-sectional views of embodiments of thermal conductors to thermally conductively connect heat sources to the radiator734.FIG.7illustrates an embodiment of a motherboard CPU736with source heat sink738thereon. The source heat sink738is thermally conductively connected to the radiator734by a plurality of heat pipes742and a radiator heat sink740.FIG.8illustrates an embodiment of a motherboard CPU836with source heat sink838thereon. The source heat sink838is thermally conductively connected to the radiator834by a vapor chamber864and a radiator heat sink840. The vapor chamber864includes a working fluid866therein that further improved heat transfer between the source heat sink838and the radiator heat sink840.FIG.9illustrates an embodiment of a motherboard CPU936with source heat sink938thereon. The source heat sink938is thermally conductively connected to the radiator934by a solid thermally conductive rod968(or another solid element) and a radiator heat sink940.

FIG.10through12are schematic illustrations of connection mechanisms to thermally conductively connect the heat-generating components of the server blade to the radiator. As described herein, the server blades may be selectively removable from or installable into the rack and/or radiator thermal management system. To facilitate the selective installation of the server blades, the thermal conductor may be disconnected from the heat-generating component, the radiator, or both. For example, the thermal conductor may be selectively connectable to the source heat sink and/or the radiator heat sink. In some examples, the radiator heat sink may be selectively connectable to the radiator. In some examples, the source heat sink may be selectively connectable to the heat-generating component.

In some embodiments, a spring-loaded mechanism connects the source heat sink to radiator heat sink.FIG.10illustrates an embodiment of a thermal management system with an elastically deformable thermal conductor, such as a heat pipe1042. The elastically deformable thermal conductor may be a coiled heat pipe that allows the heat pipe to function as a spring. The heat pipe assembly between the source heat sink1038and the radiator heat sink1040follows a spring/helix structure that is elastically compressible at least 3 centimeters. In some embodiments, the source heatsink1038is attached to the heat-generating component while the radiator heat sink1040is self-supported by one or more supports1070. The transverse action pushing in the server blade will convert to a pressure action on the spring via the elastic deformation thereof to provide the compression for thermal contact between both heat sinks1038,1040.

In some embodiments, a thermal conductor and/or heat sinks are connected to and supported by a frame of the rack or server chassis, allowing the assembly to be mechanically moved into place after the server blade installation onto the rack.FIG.11is a schematic representation of a thermal management system with a movable thermal conductor1142and heat sinks1138,1140that are selectively connectable to the heat-generating component and the radiator1134.

InFIG.11, the thermal conductor1142and heat sinks1138,1140are attached to the server chassis1172. A cantilever mechanism1174supports the thermal conductor1142(which may be elastic or inelastic, such as heat pipes) and can be extended and/or retracted by either manual or motorized operation, such as by an external actuator1176. The cantilever or other movement mechanism1174provides for contact between the heat-generating component and the column radiator1134. In some embodiments, the cantilever or other movement mechanism1174moves the radiator heat sink1140and/or at least a portion of the thermal conductor1142toward the radiator1134after the server blade is inserted into the rack. For example, after the server blade and chassis1172are inserted into and connected to the rack, the server chassis1172may provide a rigid mechanical ground for the cantilever mechanism1174to apply a force to the radiator heat sink1140and compress the radiator heat sink1140against the radiator1134. In some embodiments, the cantilever or other movement mechanism1174moves the source heat sink1138and/or at least a portion of the thermal conductor1142toward the heat-generating component after the server blade is inserted into the rack. During storage, transport, or installation of the server blade, the heat-generating component may be at risk of damage from the mass of the source heat sink, thermal conductor, and radiator heat sink applying forces to the heat-generating component due to their mass. Disconnecting the source heat sink1138and/or the thermal conductor1142from the heat-generating component during storage, transport, or installation may protect the heat-generating component. After installation of the server blade into the rack, the server chassis1172may provide a rigid mechanical ground for the cantilever mechanism1174to apply a force to the source heat sink1138and compress the source heat sink1138against the heat-generating component after all individual components are stable and connected to the rack.

In some embodiments, the thermal conductor and/or radiator heat sink are connected to and/or integrally formed with the radiator. The thermal conductor may be selectively connected to the source heat sink, or the source heat sink may be coupled to the thermal conductor and selectively connected to the heat-generating component. In some embodiments, a thermal conductor that is connected to or part of the radiator itself can be mechanically lowered onto the heat-generating component after installation of the server blade.

FIG.12is a schematic representation of a server blade with a heat-generating component, with a source heat sink1238thereon, positioned adjacent to a radiator1234. The radiator1234includes a thermal conductor1242that is deployable from proximate the radiator1234toward the heat-generating component to thermally conductively connect the heat-generating component to the radiator1234. In some embodiments, the thermal conductor1242is part of the column radiator1234that can be cantilevered onto the source heat sink1238. The joint1278of the movable thermal conductor1242may allow heat transfer via additional thermal conducting materials, such as copper mesh or elastic thermal conducting elements, that provide reduced thermal conduction between the source heat sink1238and column radiator1234. In some embodiments, the thermal conductor1242is movable via manual or motorized operation.

INDUSTRIAL APPLICABILITY

The present disclosure relates generally to systems and methods for protecting electronic devices from thermal damage. More particularly, the present disclosure relates to devices, systems, and methods for cooling server blades using a column radiator to air cool select components, such as a central processing unit (CPU) or memory (random access memory) of a server blade. In some embodiments, a central column radiator with a closed cold aisle connection is used in conjunction with ambient fans that blow ambient air over the remainder of the server blade.

Datacenters include a plurality of electronic devices, some of which are computing devices and some of which are not, that all generate thermal energy. The thermal energy needs to be transported away from the electronic devices to prevent damage to the electronic devices and/or protect the integrity of the data stored or computed on the electronic devices. Datacenters use thermal management systems to carry thermal energy away from the electronic devices by liquid cooling, air cooling, or a combination thereof. Even in liquid cooled systems, the liquid cooling may efficiently conduct thermal energy from the electronic devices but ultimately reject the thermal energy from the warmed liquid (or vapor) into the ambient atmosphere. The warmed ambient air must then be moved from the liquid cooling system and/or the electronic devices to complete the thermal management of the electronic devices, which may be inefficient.

In some embodiments of datacenters and thermal management systems according to the present disclosure, a plurality of heat generating devices are located in an enclosed space and air is moved through a radiator to transfer heat from the heat-generating devices to cool the heat-generating devices. While the present disclosure will describe the use of airflow to cool heat-generating devices directly, such as cooling computing devices, hardware storage devices, networking devices, power supplies, and other electronic devices, it should be understood that the thermal management system may use liquid cooling fluid flow to cool heat sinks of heat-generating components. In some embodiments, the column radiator is subcooled below the ambient air temperature, and select heat-generating components are thermally conductively connected to the radiator to conductively exhaust waste heat from the heat-generating component.

An example environment in which thermal management systems and methods according to the present disclosure may be used is a server array. In some embodiments, a datacenter includes server computers arranged in a row, where the row contains a plurality of server racks, each of which contain at a plurality of server computers, power supplies, networking devices, and other electronic devices. In some examples, the server computer is a blade server. In some examples, the server computers are complete computers (e.g., each server computer can function as a standalone computer). In some examples, the server computers are electronic components that can cooperate to provide scalable computational power.

The server row can include a row manager that is in communication with the server racks and/or rack manager of the server row. In some embodiments, the row manager controls computational loads, such as process allocations, of the server racks and/or server computers. In doing so, the row manager may control the amount of heat generated by the server computers of the server racks. In some embodiments, the row manager controls thermal management of the server racks and/or server computers. For example, the row manager can manage active thermal management for the server racks and/or server computers by changing fan speed or by controlling the flow rate of a cooling fluid for liquid cooling systems. In at least one example, the server row is at least partially cooled by a liquid cooling system that delivers cooling fluid to the server racks of the server row. The row manager is in communication with the cooling fluid pump to change or stop the flow of cooling fluid.

A server rack can support a plurality of server computers in the rack. The server computers may each have liquid cooling, such as localized immersion cooling for at least some electronic components of the server computer, or a cooling plate with recirculating cooling fluid to cool the electronic component(s) of the server computer. In some embodiments, the server computers or other electronic devices may be air-cooled, utilizing a cold aisle and a hot aisle that flow colder air from the cold aisle and evacuate hotter air from the electronic devices through the hot aisle. The air flows from the cold aisle to the hot aisle based on air pressure differentials established by pumps or blowers of the thermal management system in series with the cold aisle and the hot aisle.

In some embodiments, the electronic components, such as server computers, of the server rack are in data communication with a rack manager. The rack manager may control power delivery to the server computers or other electronic components. In some embodiments, the rack manager may communicate with the server computers or other electronic components to power cap or throttle the server computers or other electronic components and manage power draw. The rack manager, in some embodiments, is also in communication with a cooling fluid pump that moves cooling fluid to one or more server computers or other electronic components in the server rack.

A system resource manager may be in data communication with the row manager and/or rack manager(s) to communicate with the electronic devices, as well as be in communication with thermal sensors to measure one or more properties of the thermal management system. In some embodiments, the resource manager is the row manager. In some embodiments, the resource manager is the rack manager. In some embodiments, the resource manager is a dedicated controller.

The system resource manager includes a processor and a hardware storage device. The processor may receive information from the thermal sensors and communicate with one or more other devices according to instructions stored on the hardware storage device that cause the processor to perform any of the methods described herein. In some embodiments, the devices in communication with the system resource manager that may receive instructions from the system resource manager in response to detecting an increase in temperature include a cooling fluid pump, fan, valve, or another thermal management device (e.g., blower). For example, the system resource manager may adjust the flow of cooling fluid by turning on the cooling fluid pump or by actuating a valve to direct airflow.

The hardware storage device can be any non-transient computer readable medium that may store instructions thereon. The hardware storage device may be any type of solid-state memory; volatile memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM); non-volatile memory, such as read-only memory (ROM) including programmable ROM (PROM), erasable PROM (ERPOM) or EEPROM; magnetic storage media, such as magnetic tape; a platen-based storage device, such as hard disk drives; optical media, such as compact discs (CD), digital video discs (DVD), Blu-ray Discs, or other optical media; removable media such as USB drives; non-removable media such as internal SATA or non-volatile memory express (NVMe) style NAND flash memory; or any other non-transient storage media.

The air is provided to the entire row and/or rack to cool the ambient air around the components of the rack. Cooling the entire environment may not be necessary and, in fact, inefficient, when the heat generation by the components of the server computers and other electronic devices is uneven. For example, a central processing unit (CPU) or system memory of the server computer may generate considerably greater heat than a non-volatile storage device. To limit and/or prevent thermal damage to the CPU, however, the entire ambient air is cooled based off of the CPU temperature and load. Localized cooling can efficiently cool the hottest areas and/or components without expending additional energy to cool the entire room in which the row or rack is located.

In some embodiments, a thermal management system according to the present disclosure provides localized cooling for the greatest heat-generating components of the server blades. A hybrid thermal management system uses a shared radiator to provide a subcooled heat sink in addition to the ambient air. In some embodiments, the radiator has an interior volume through which subcooled air or other gas flows to cool the radiator. In some embodiments, a cooling liquid is pumped through the radiator to cool the radiator. Select heat-generating components, such as processors (e.g., CPU, graphical processing unit (GPU)), system memory (e.g., RAM), network connection devices, and power supplies, are thermally conductively connected to the radiator to conduct heat away from the components to the cold radiator. The rest of the components, which generate less heat, on the motherboard are cooled via ambient air and/or fans to blow ambient air across those components.

In some embodiments, a thermal management system includes a rack of server computers or server blades. A rack manager or other controller is in data communication with thermal sensors and control structures for controlling the flow of air through the radiator. As described above, the radiator may be gas-cooled or liquid-cooled. While the present disclosure will describe the thermal management system as using air, it should be understood that other cooling fluids may be used. The air flowing through the radiator cools the radiator. In some embodiments, the radiator connects the cold aisle, through the radiator, to the hot aisle. The air warms as the radiator receives heat from the heat-generating components, such as the CPU of each blade. While some embodiments have the radiator oriented vertically to connect the cold aisle to the hot aisle, the radiator may be positioned at other orientations. For example, at least a portion of the radiator may be oriented at an angle to the direction of gravity. In some examples, at least a portion of the radiator may be oriented horizontally. In some examples, at least a portion of the radiator may be curved or have a corner therein. In at least one example, the radiator may provide cold air across a plurality of racks.

The heat-generating component, such as the CPU, is thermally conductively connected to the radiator by a thermal conductor. In some embodiments, a thermally conductor is a solid structure that conducts heat from the heat-generating component to the radiator within convective or radiative thermal interfaces. A thermally conductively connected element does not rely upon heat transfer through the air or other medium, but rather is physically connected to the radiator.

The thermal conductor includes at least one source heat sink positioned in contact with the heat-generating component. In some embodiments, a thermal paste or other interface material is positioned between the source heat sink and the heat-generating component to fill gaps and/or provide a thermally efficient interface.

The source heat sink may be thermally conductively connected to a radiator heat sink in contact with the radiator. In some embodiments, the source heat sink may be thermally conductively connected with the radiator directly, such as having one or more thermal conductors integrally formed with the radiator. In some embodiments, the radiator heat sink has an interface material positioned between the radiator heat sink and the radiator.

In some embodiments, and as will be described in greater detail below, the source heat sink is thermally conductively connected to the radiator and/or radiator heat sink by a thermal conductor. The thermal conductor may be or include a heat pipe, vapor chamber, solid thermally conductive rods or fins, or combinations thereof. In some embodiments, each of the server blades has a source heat sink that is thermally conductively connected to a radiator heat sink by heat pipes. In some embodiments, the source heat sink is thermally conductively connected to the radiator heat sink by a plurality of heat pipes or other thermal conductors. In some embodiments, the source heat sink is thermally conductively connected to the radiator heat sink by a single heat pipe or other thermal conductor. For example, the number and/or size of the thermal conductor may be related to the distance from the source heat sink to the radiator or radiator heat sink.

A rank manager may be in communication with one or more components of the rack. In some embodiments, the rack manager is the resource manager. In some embodiments, a dedicated controller is in communication with one or more radiator blowers and/or ambient fans. The radiator blower may be positioned and configured to flow air (or other cooling fluid) through the radiator between the cold aisle and the hot aisle. In some embodiments, the thermal management system includes a plurality of radiator blowers, such as a first radiator blower proximate the cold aisle and a second radiator blower proximate the hot aisle.

The ambient fans may be positioned on or proximate to the blade to blow ambient air across a surface of the motherboard and/or lower heat-generating components of the server blade. For example, the server blade may include non-volatile memory, such as a magnetic platen hard disk drive. The magnetic platen hard disk drive may require fewer cooling resources than the CPU, and the ambient air may provide sufficient cooling capacity to cool the magnetic platen hard disk drive without being thermally conductively connected to the radiator. In some embodiments, such the ambient fan(s) are positioned and oriented to blow ambient air from the room into the server blade and toward the radiator. In at least one embodiment, the ambient fan is positioned proximate the radiator to blow ambient air cooled by the surface of the radiator away from the radiator across the server blade and/or motherboard to cool the motherboard and components thereon.

The rack manager or other resource manager controller may be in communication with the thermal sensor(s) and adjust the radiator blower and/or ambient fans according to measurements from the thermal sensor(s). For example, the CPU is passively thermally conductively connected to the radiator, and additional cooling capacity to the CPU can be provided by cooling the radiator to increase the temperature gradient across the thermal conductor (e.g., heat pipes). If a thermal sensor positioned on the CPU or on an interface with the CPU measures a CPU temperature approaching or exceeding a threshold temperature, the rack manager or other resource manager controller may send a signal to the radiator blower to increase the flow of air through the radiator from the cold aisle to further cool the radiator.

In another example, a thermal sensor positioned on the motherboard may measure a motherboard temperature that is approaching or exceeding a threshold temperature. The rack manager or other resource manager controller may send a signal to the ambient fan(s) to blow ambient air across the motherboard and/or other components toward or away from the radiator to cool the motherboard and/or other components.

In some embodiments, the radiator heat sink is selectively connected to the radiator, allowing the radiator heat sink and the server blade to be disconnected from the radiator and/or rack for replacement or maintenance. For example, the server blade may slide into the rack toward the radiator during installation. The server blade may include a source heat sink and radiator heat sink installed on the server blade204. When the server blade is urged into the rack, the server blade may electrically connect to a power supply and/or networking components and, at the same time, connect the radiator heat sink to the radiator.

To provide efficient convective heat transfer between the air in the interior volume of the radiator and the radiator walls, the radiator may include one or more thermal surface features to increase surface area of an inner or outer surface of the radiator. In some embodiments, the radiator has an interior volume through which the air from the cold aisle flows. The air from the cold aisle cools the radiator walls to which a radiator heat sink or thermal conductor may be connected. The radiator includes one or more thermal surface features to increase the surface area of the inner surface of the radiator walls and more efficiently transfer heat between the air and radiator walls and cool the radiator walls. In some embodiments, the thermal surface features include fins oriented in the longitudinal direction of the radiator (e.g., the direction of airflow through the interior volume). By orienting the fins in the direction of airflow, the resistance to the air is minimized while exposing a larger surface area of the fins to the air.

In some embodiments, the air warms as the air flows through the radiator, which decreases the temperature gradient between the air and the radiator. A decrease in the temperature gradient can decrease heat transfer rates. To compensate for the warming of the air along the longitudinal length of the radiator, the thermal surface features can vary in size, shape, or type along the longitudinal direction of the radiator. For example, another transverse cross-section of the same radiator at a different longitudinal position in the radiator may have a different shape or thermal surface features thereon. Where the radiator is proximate the cold aisle (e.g., where the air is coldest in the radiator), the fins are shorter (e.g., protrude into the interior volume less) than those proximate the hot aisle, and there may be less fins than proximate the hot aisle. The increase in surface area proximate the hot aisle relative to the cold aisle may compensate for the lower temperature gradient and lower heat transfer rate. In other examples, the thermal surface features may taper, move, twist (e.g., a helix), start or stop mid-way along the longitudinal length, or have perforations or surface textures thereon along the longitudinal length of the radiator to adjust the surface area of the inner surface of the radiator to balance thermal transfer into the radiator along the length of the radiator.

A variety of thermal surface features may be used. In some embodiments, the thermal surface features include rods that extend into the interior volume of the radiator. The rods may be solid rods that provide an increase in surface area for radiator walls. In some embodiments, the thermal surface features include heat pipes, which transfer heat efficiently and increase surface area of the radiator walls.

In at least one embodiments, the radiator includes an outer thermal surface feature on an outer surface of the radiator to cool the ambient air immediately surrounding the radiator. For example, an outer thermal surface feature of the radiator may allow an ambient fan or other fan outside of the radiator to blow the cooled air (cooled by the outer thermal surface feature) away from the radiator and over the motherboard or other components of the server blade. It should be understood than any thermal surface feature or combination thereof may be used as an outer thermal surface feature.

In some embodiments of a thermal management system, a radiator includes a plurality of columns. As described herein, the air provided from the cold aisle through the radiator warms as the air receives heat from the server blades and components thereof. In some embodiments, a radiator includes separate columns to distribute the air between the columns and isolate the heat transferred to the air therein. For example, each of a first column and a second column may have two of four radiator heat sinks connected thereto. Reducing the number of radiator heat sinks connected to the outer surface of the column reduces the amount of heat transferred to the air therein. Further, the radiator heat sinks may be staggered to alternate to which column each neighboring radiator heat sink is connected. By longitudinally spacing the radiator heat sinks connected to each column, the thermal gradients may be further improved.

In some embodiments, the radiator is a center column or center radiator in a rack. To efficiently use the available space and surface area of the radiator, server blades may be positioned on and connected to opposite sides of the radiator. In some embodiments, a rack includes radially positioned server blades around a central radiator. In some embodiments, the radiator may provide and/or be the structural support to which the server blades are connected. By supporting the server blades through a direct mounting to the radiator, the radiator may provide not only cooling, but function as a center spine for the rack. The air flows through the center of the radiator, cooling the radiator and receiving heat from the radiator heat sinks connected to the outer surface of the radiator.

A radial arrangement of server blades may use conventional server blades staggered radially and longitudinally (e.g., helixed) around the central radiator. In some embodiments, the server blades may have wedge-shaped motherboards to provide a more efficient surface area for components and cooling. A plurality of wedge-shaped server blades may, when installed on the radiator, form a complete disc or circle (or other shape) around the radiator. In some embodiments, a wedge-shaped motherboard allows for the CPU or other component to which the source heat sink is connected to be positioned on the motherboard near the radiator. Greater freedom in component location on the motherboard can allow for shorter heat pipes or other thermal conductor between the source heat sink and the radiator heat sink to efficiently transfer heat.

In some embodiments, the source heat sink is thermally conductively connected to the radiator by a plurality of heat pipes and a radiator heat sink. In some embodiments, the source heat sink is thermally conductively connected to the radiator by a vapor chamber and a radiator heat sink. The vapor chamber includes a working fluid therein that further improved heat transfer between the source heat sink and the radiator heat sink. In some embodiments, a source heat sink is thermally conductively connected to the radiator by a solid thermally conductive rod (or another solid element) and a radiator heat sink.

As described herein, the server blades may be selectively removable from or installable into the rack and/or radiator thermal management system. To facilitate the selective installation of the server blades, the thermal conductor may be disconnected from the heat-generating component, the radiator, or both. For example, the thermal conductor may be selectively connectable to the source heat sink and/or the radiator heat sink. In some examples, the radiator heat sink may be selectively connectable to the radiator. In some examples, the source heat sink may be selectively connectable to the heat-generating component.

In some embodiments, a spring-loaded mechanism connects the source heat sink to radiator heat sink. An elastically deformable thermal conductor may be a coiled heat pipe that allows the heat pipe to function as a spring. The heat pipe assembly between the source heat sink and the radiator heat sink follows a spring/helix structure that is elastically compressible at least 3 centimeters. In some embodiments, the source heatsink is attached to the heat-generating component while the surface in contact with the radiator heat sink is self-supported by one or more supports. The transverse action pushing in the server blade will convert to a pressure action on the spring via the elastic deformation thereof to provide the compression for thermal contact between both heat sinks.

In some embodiments, a thermal conductor and/or heat sinks are connected to and supported by a frame of the rack or server chassis, allowing the assembly to be mechanically moved into place after the server blade installation onto the rack. In some embodiments, a thermal management system with a movable thermal conductor and heat sinks are selectively connectable to the heat-generating component and the radiator.

A cantilever mechanism supports the thermal conductor (which may be elastic or inelastic, such as heat pipes) and can be extended and/or retracted by either manual or motorized operation. The cantilever or other movement mechanism provides for contact between the heat-generating component and the column radiator. In some embodiments, the cantilever or other movement mechanism moves the radiator heat sink and/or at least a portion of the thermal conductor toward the radiator after the server blade is inserted into the rack. For example, after the server blade and chassis are inserted into and connected to the rack, the server chassis may provide a rigid mechanical ground for the cantilever mechanism to apply a force to the radiator heat sink and compress the radiator heat sink against the radiator. In some embodiments, the cantilever or other movement mechanism moves the source heat sink and/or at least a portion of the thermal conductor toward the heat-generating component after the server blade is inserted into the rack. During storage, transport, or installation of the server blade, the heat-generating component may be at risk of damage from the mass of the source heat sink, thermal conductor, and radiator heat sink applying forces to the heat-generating component due to their mass. Disconnecting the source heat sink and/or the thermal conductor from the heat-generating component during storage, transport, or installation may protect the heat-generating component. After installation of the server blade into the rack, the server chassis may provide a rigid mechanical ground for the cantilever mechanism to apply a force to the source heat sink and compress the source heat sink against the heat-generating component after all individual components are stable and connected to the rack.

In some embodiments, the thermal conductor and/or radiator heat sink are connected to and/or integrally formed with the radiator. The thermal conductor may be selectively connected to the source heat sink, or the source heat sink may be coupled to the thermal conductor and selectively connected to the heat-generating component. In some embodiments, a thermal conductor that is connected to or part of the radiator itself can be mechanically lowered onto the heat-generating component after installation of the server blade.

In some embodiments, the radiator includes a thermal conductor that is deployable from proximate the radiator toward the heat-generating component to thermally conductively connect the heat-generating component to the radiator. In some embodiments, the thermal conductor is part of the column radiator that can be cantilevered onto the heat source. The joint of the movable thermal conductor may allow heat transfer via additional thermal conducting materials, such as copper mesh or elastic thermal conducting elements, that provide reduced thermal conduction between the heat plate and column radiator. In some embodiments, the thermal conductor is movable via manual or motorized operation.

The present disclosure relates to systems and methods for thermal management in a server rack according to at least the examples provided in the sections below:

[A1] In some embodiments, a thermal management system for cooling a computing device includes a cold aisle, a hot aisle, a radiator, and a plurality of source heat sinks thermally conductively connected to the radiator. The radiator connects the cold aisle to the hot aisle and flows a cooling fluid through an interior volume of the radiator. Each source heat sink is configured to connect to a heat-generating electronic component to thermally conductively connect the heat-generating component to a surface of the radiator.

[A2] In some embodiments, the thermal management system of [A1] includes a radiator heat sink contacting a surface of the radiator and thermally conductively connected to the source heat sink.

[A3] In some embodiments, the source heat sink of [A2] is thermally conductively connected to the radiator heat sink by a heat pipe.

[A4] In some embodiments, the radiator heat sink of [A2] or [A3] is selectively connected to the radiator to allow removal of the source heat sink thermally conductively connected to the radiator.

[A5] In some embodiments, the radiator of any of [A1] through [A4] is a column radiator.

[A6] In some embodiments, the cooling fluid of any of [A1] through [A5] is air.

[A7] In some embodiments, the radiator of any of [A1] through [A6] includes a plurality of thermal surface features on an inner surface thereof.

[A8] In some embodiments, the radiator of any of [A1] through [A7] includes a plurality of thermal surface features on an outer surface thereof.

[A9] In some embodiments, the thermal management system of any of [A1] through [A8] includes an ambient fan to blow ambient air toward the radiator.

[A10] In some embodiments, the thermal management system of any of [A1] through [A9] includes an ambient fan to blow ambient air away from the radiator.

[B1] In some embodiments, a thermal management system includes a cold aisle, a hot aisle, a radiator, a plurality of source heat sinks thermally conductively connected to the radiator, and an ambient fan positioned and configured to blow ambient air toward at least one of the source heat sinks. The radiator connects the cold aisle to the hot aisle and flows a cooling fluid through an interior volume of the radiator. Each source heat sink is configured to connect to a heat-generating electronic component to thermally conductively connect the heat-generating component to a surface of the radiator.

[B2] In some embodiments, the thermal management system of [B1] includes a thermal sensor and a controller in data communication with the thermal sensor and the ambient fan. The thermal sensor is positioned at the heat-generating electronic component to measure a temperature of the heat-generating electronic component. The controller is configured to adjust the ambient fan based at least partially on a measurement from the thermal sensor.

[B3] In some embodiments, the controller of [B2] is part of a rack manager.

[B4] In some embodiments, the controller of [B2] or [B3] is also in communication with a radiator blower and configured to adjust the radiator blower based at least partially on a measurement from the thermal sensor.

[B5] In some embodiments, the thermal management system of any of [B1] through [B4] includes a plurality of server blades. Each server blade includes one or more of the heat-generating electronic components. At least two of the plurality of server blades are positioned on opposite sides of the radiator.

[B6] In some embodiments, at least a portion of the plurality of server blades of [B5] is positioned radially around the radiator.

[B7] In some embodiments, the radiator of any of [B1] through [B6] is a round column radiator.

[C1] In some embodiments, a thermal management system for cooling computing devices includes a cold aisle, a hot aisle, a radiator, and a plurality of server blades. The radiator connects the cold aisle to the hot aisle and flows a cooling fluid through an interior volume of the radiator. Each server blade of the plurality of server blades includes a heat-generating component, a source heat sink, and an ambient fan. The source heat sink is positioned on the heat-generating component and thermally conductively connected to the radiator. The ambient fan is positioned and configured to blow ambient air toward at least one of the source heat sinks.

[C2] In some embodiments, the radiator of [C1] includes a plurality of columns, and each column connects the cold aisle to the hot aisle. A first source heat sink of a first server blade of the plurality of server blades is thermally conductively connected to a first column. A second source heat sink of a second server blade of the plurality of server blades is thermally conductively connected to a second column.

[C3] In some embodiments, the radiator of [C1] or [C2] includes at least one thermal surface feature on an inner surface thereof that changes in at least one dimension in a longitudinal direction of the radiator.

It should be understood that any directions or reference frames in the preceding description are merely relative directions or movements. For example, any references to “front” and “back” or “top” and “bottom” or “left” and “right” are merely descriptive of the relative position or movement of the related elements.

The present disclosure may be embodied in other specific forms without departing from its characteristics. The described embodiments are to be considered as illustrative and not restrictive. The scope of the disclosure is, therefore, indicated by the appended claims rather than by the foregoing description. Changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.