Hybrid liquid and air cooling in networking equipment

A networking hardware system includes a housing; a board located in the housing and comprising a plurality of components; a liquid cooled heat exchanger; and one or more fans disposed near the liquid cooled heat exchanger and configured to provide cool airflow from the liquid cooled heat exchanger to any of the plurality of components. The housing can be substantially sealed from an external environment and includes no air intake thereon, removing a need for higher powered fans and for air filtering for dust. The housing can include a faceplate with no air intake thereon, providing increased density for ports on the faceplate.

FIELD OF THE DISCLOSURE

The present disclosure generally relates to networking hardware. More particularly, the present disclosure relates to systems and methods for hybrid liquid and air cooling in networking equipment.

BACKGROUND OF THE DISCLOSURE

Networking equipment is physically realized as modules, chassis, shelves, pizza boxes, etc. that are mounted in a rack, frame, cabinet, etc. located in a data center, telecom hut, central office, point of presence, IT room, outside plant, etc. As described herein, networking equipment means any such equipment that performs electrical and/or optical functions and also includes computing, storage, and the like. Two of the main design constraints in networking equipment are space (i.e., real estate) and heat dissipation. Also, these two design constraints often clash as the conventional approach for heat dissipation is air cooling where an external housing of the networking equipment requires space for holes to allow external air to enter the housing. For example, typically, network elements are air cooled with front-to-rear airflow where fans inside the housing draw external air inward for cooling and expel the heated air from the rear. Of course, other implementations are also known, e.g., side-to-rear airflow or side-to-side airflow, but all of these implementations require space for air intake. The disadvantage of this approach is it takes up space on a faceplate which could otherwise be used for ports. A faceplate is a face of the network element—usually but not necessarily the front—where ports, such as pluggable optics are connected.

Another disadvantage of this approach is that pure air-cooled systems require larger and larger heat sinks as the networking equipment continues to push speed and capacity. Also, there is a requirement for improved air flow which often entails faster, louder fans and more faceplate area for air intake. The large airflow and pressures created draws dust through the network element and filtering such large amounts of air presents its own challenges. The required fan power scales with the cube of the air flow delivered, so a design improvement with 30% increase in air flow, for example, corresponds to 2.2 times increase in fan power. Furthermore, the space and cost requirements for the associated power systems become an increasingly significant part of an overall system.

Another approach is liquid cooling where liquid pipes provide the ingress/egress of fluid, and cold plates are connected directly to major heat-producing devices. However, there are smaller and lower-power devices that also need cooling, but liquid pipes to the complete set of those devices take too much room and add cost.

BRIEF SUMMARY OF THE DISCLOSURE

The present disclosure relates to systems and methods for hybrid liquid and air cooling in networking equipment. To address the design constraints associated with space and heat dissipation, the present disclosure utilizes sealed airflow within a network element housing, i.e., there is little or no air intake from the external environment, removing the space limitations for air intake on the external housing. To address heat dissipation of the sealed airflow, there is a liquid-cooled heatsink or radiator (evaporator) placed conveniently anywhere in the path of this airflow to move heat from the air to the liquid. Hence, this approach is described as hybrid liquid and air cooling. The benefits of this approach include fewer powered fans, lower total fan power, less acoustic noise, removal of the need for air filters and space for air intake, and increased density on the faceplate.

A networking hardware system includes a housing; a board located in the housing and comprising a plurality of components; a liquid cooled heat exchanger; and one or more fans disposed near the liquid cooled heat exchanger and configured to provide cool airflow from the liquid cooled heat exchanger to any of the plurality of components. The housing can be substantially sealed from an external environment and can includes no air intake thereon, removing a need for higher powered fans and for air filtering for dust. The housing can include a faceplate with no air intake thereon, providing increased density for ports on the faceplate. The housing can exclude air filtering for dust based on being substantially sealed from an external environment. The one or more fans can be configured to circulate the airflow in a closed loop. The closed loop can include pressure differential internal to the housing for the airflow.

The liquid cooled heat exchanger can include one of a heat sink with an optional base, a plurality of fins and a liquid cooling pipe therein, and the board and a liquid cooling pipe therein. The networking hardware system can further include one or more baffles located in an interior of the housing and positioned to direct the airflow within the housing. The one or more baffles can be positioned to exclude some of the plurality of components from the airflow. The networking hardware system can be a chassis having the housing and the board is a module inserted into the chassis.

The networking hardware system can be a rack-mounted unit. The one or more fans can operate at a variable speed based on ambient condition in the housing. The networking hardware system can further include a condensation detection and mitigation system. The condensation detection and mitigation system can include a temperature sensor that causes adjustments of the one or more fans to keep the liquid cooled heat exchanger above a dew point. The condensation detection and mitigation system can include a condensation collection system to maintain any condensate away from the plurality of components.

In another embodiment, a method of operating networking hardware includes steps of operating the networking hardware in a substantially sealed environment where, internal to a housing of the networking hardware, airflow is maintained and circulated therein; cooling the airflow at one or more paths internal to the housing, via one or more liquid cooled heat exchangers; and operating one or more fans disposed, each near the one or more liquid cooled heat exchangers, to provide cool airflow from the liquid cooled heat exchanger to any of a plurality of components internal to the housing. The steps can include directing the cool airflow from the one or more fans via one or more baffles located in the interior of the housing. The steps can include excluding any of more of a plurality of components from the airflow, via the one or more baffles. The steps can include operating the one or more fans at a variable speed based on ambient condition in the housing.

DETAILED DESCRIPTION OF THE DISCLOSURE

The present disclosure relates to systems and methods for hybrid liquid and air cooling in networking equipment. To address the design constraints associated with space and heat dissipation, the present disclosure utilizes sealed airflow within a network element housing, i.e., there is little or no air intake from the external environment, removing the space limitations for air intake on the external housing. To address heat dissipation of the sealed airflow, there is a liquid-cooled heatsink or radiator (evaporator) placed conveniently anywhere in the path of this airflow to move heat from the air to the liquid. Hence, this approach is described as hybrid liquid and air cooling. The benefits of this approach include fewer powered fans, lower total fan power, less acoustic noise, removal of the need for air filters and space for air intake, and increased density on the faceplate.

FIG.1is a perspective view of a networking module10.FIG.2is a front view of a networking shelf12, andFIG.3is a front view of a networking shelf14. These are presented to illustrate faceplates16,18,20. The networking module10includes significant space on its faceplate16for air intake, via holes22. The networking shelves14,16also require space on their faceplates18,20for various holes22. As can be seen, the holes22reduce the amount of space for ports.

The present disclosure includes a sealed housing for a networking module or networking shelf, without the holes, thereby increasing the space for ports or the like. The sealed housing can be on a module or card basis or for an entire shelf. For example, some deployments are referred to as pizza boxes which are generally 1-2 rack unit (RU) high integrated housings. Modules, cards, or blades (referred to herein as modules) are insertable into a shelf or chassis. The sealed housing refers to any of these physical embodiments.

FIG.4is a perspective view of a networking device50with a sealed housing52. Again, the networking device can be any physical embodiment of networking hardware, i.e., a module, a rack-mounted unit (pizza box), a chassis, a cabinet, etc.FIG.5is a top view of an interior of the networking device50. For illustration purposes, the networking device50is a horizontal module or a rack-mounted unit (pizza box). Those skilled in the art will recognize the hybrid air and liquid cooling described herein can apply to any physical embodiment.

Of note, the term sealed housing52does not necessarily require a perfectly air-hermetic module, i.e., a small amount of air exchange with the outside environment is acceptable, but there is no air intake necessary. Rather, the sealed housing52is one that does not require dedicated space, such as on a faceplate54, for air intake. As such, the faceplate54has more room for ports56, status indicators58(e.g., displays, light emitting diodes (LEDs), etc.), and the like.

Because air is only or mainly circulated within the sealed housing52, internal fans60are creating pressure differences internal to the device50, not external to it so holes in the device50do not necessarily draw in lots of air. Also, the fans60do not require the same amount of force. The closed system means that dust is not an issue. In the case where air exchange with the outside is undesirable; for example, in a weather-resistant application, the device50may be well-sealed, and the approach described herein will still function without compromise.

InFIG.5, the networking device50includes the ports54, various circuits62-72, and power74, which can be on board51, such as a printed circuit board (PCB). The fans60can include redundant fans to generate the interior airflow to increase reliability to be similar to the other components inside the device50. Individual modules are generally considered single points of failure. In an embodiment, the interior fans60can be field-removable by locating them along the front or rear edge of the housing52. For a pizza box (rack-mounted unit53), they can be located near the top and bottom covers too.

Also, a circuit pack can mate to an external rear fan module such that together they form an enclosure for circulation of air. In this case, the external fan modules can have much smaller fans that would normally be required in a purely air-cooled scenario and thus make room to act as liquid distribution modules. The liquid distribution modules could take building water and provide it to the circuit packs in a redundant manner.

The networking device50includes one or more air-to-liquid heatsinks80or radiators placed in the path of the airflow as necessary. The objective here is to cool internal air in the housing52after it has flowed over any of the circuits62-72. Of note,FIG.5illustrates a single air-to-liquid heatsink80, but practical embodiments can include more than one.

In one embodiment, the air-to-liquid heatsink80is the PCB itself. Components such as the circuits62-72couple a substantial amount of heat into PCB's. A large heatsink such as a PCB can help reduce condensation issues. In another embodiment, the air-to-liquid heatsink is the metal enclosure of the housing52. In another embodiment, an insulating layer is added between the enclosure and the exterior environment so that heating of the liquid lines by the exterior environment is minimized

The networking device50includes sealed airflow circulation coupled with heat removal via liquid. As air flows around the inside of the device50in a closed loop, it picks up heat from board components (e.g., the circuits62-72) and then releases that heat to a liquid-cooled heat exchanger (note, this term is used interchangeably with the air-to-liquid heatsink80) in the air path so that the air is cooled and ready for the next cycle; the heat exits the closed-loop air system via liquid, not air. Each component can have a heatsink or not, depending on the operating temperature requirement of the component. On the word ‘sealed’, as noted in herein, its ok to have a small amount of air exchange with the outside world.

The fans60can have a variable speed determined by internal airflow sensor in circuit air path. Fan speed can be adjusted based on measured airflow, which reduces fan aging and delays fan failure.

FIG.5shows the airflow inside the networking device50in a circular manner.FIG.6is a top view of an interior of the networking device50with a baffle82included therein.FIG.7is a top view of an interior of the networking device50with the baffle82and with a maze structure84included therein. Air baffles82can be arranged to encourage airflow, namely to force the airflow to flow in a set direction such that cool air leaves the air-to-liquid heatsink80and hot air enters the air-to-liquid heatsink80. The baffle can also serve to maximize local air speed in the region of one or more devices. The baffle82and the maze structure84can include any type of components, e.g., metal, the housing52itself, etc. In an embodiment, these components are part of the device50itself. That is, anything that can block air and direct it is contemplated.

Also, while the description has been to a sealed housing with an enclosed-loop airflow, this hybrid approach can also apply to non-enclosed airflow, such as a way to reduce the size and force of the fans in systems that rely on external airflow. In such an embodiment, the fresh cool air passes over some of the components, then passes over a liquid-cooled radiator, then over the rest of the components. Each component can have a heatsink or not, depending on the operating temperature requirement of the component. The radiator reduces pre-heating effects. By using external supply air, a dust filter would be part of that system. Also, it would require faceplate air holes, which consume faceplate room.

Also, the network device50is shown having a single housing52. In a chassis, shelf, cabinet, etc., there can be multiple modules such as the networking device50. In this embodiment, the networking device50can exclude the housing52, rather the housing52is the shelf, chassis, cabinet, etc. In this manner, multiple devices50can work together in a sealed housing. That is, two or more field-replaceable modules can be joined by a chassis to form a closed-airflow circuit. Here, the fans60can be on the device50or in the chassis, shelf, cabinet, etc. Also, the air-to-liquid heatsinks80can be on the devices50as well. Those skilled in the art recognize there can be various physical embodiments, all having different directions of airflow, etc. As such, the term networking hardware is used to cover any of physical embodiment.

Further, the air and liquid cooling may not be required for all components inside the networking device. For example, the ports54can have their own liquid cooling and/or air cooled heatsinks. Also, inFIG.7, the circuit62is shown excluded from the maze structure84. Here, the circuit62can be a high-powered circuit that requires its own dedicated liquid cooling solution. The hybrid air and liquid cooling can focus on the other components (circuits64-72) which it is not practical to dedicate their own cooling solution. Even further, this approach can reduce the size requirements for heat sinks on any of the circuits62-72, thereby gaining additional space inside the networking device50.

Condensation Control and Mitigation—In a non-sealed system, in the absence of mitigation, the air inside the networking device50will have the potential to absorb water vapor. If the temperature of the fins of the liquid-cooled heat exchanger80is below the dew point of the air inside the networking device50, condensation will occur on the heat exchanger80, and may pose a risk to electronics in the system.

Such risk is mitigated by one or more of the following approaches, including combinations:

The networking device50can include temperature and humidity monitoring to determine the dew point and to determine if condensation is at risk of occurring at any time

The networking device50can include a technique to regulate the flow rate of the liquid through the heat exchanger such that the heat exchanger80is never below the dew point.

The networking device50can include regulation of the flow rate of the air in the air circuit (e.g., fan speed control) such that the heat exchanger80is never below the dew point.

The enclosure is sealed from the external environment with the exception of controlled openings. These openings can be covered with a membrane which allows water vapor to leave the enclosure, but not to enter the enclosure (e.g., Gortex). The enclosure is assembled such that the water vapor content inside is known and below a threshold. The threshold is selected such that under the range of the expected operating environments, the dew point cannot be reached.

In the case that condensate is formed, the condensate location is bounded and controlled in a manner such that it never a risk to the electronics.FIG.8Ais a diagram of the heat sink80illustrating a close-up of one fin90and an approach for controlling condensation, andFIG.8Bis a diagram of another embodiment of the heat sink80. InFIG.8A, the array of fins is directly coupled to the liquid tubing. InFIG.8B, the array od fins or pins is coupled to a base plat in which the liquid tubing is embedded. Here, the cold surfaces of the heat exchanger—the tubing [FIG.8B], fins90, and baseplate (if used)—are coated with a hydrophilic wicking material94such as a mesh, powdered metal, parallel grooves, or other standard means to encourage the capture and movement of liquid via capillary action. The wick structure94runs continuously from the heat exchanger to the reservoir96. The reservoir96is sized and positioned such that it is generally drier than the heat exchanger so that when condensate92forms on any member of the heat exchanger, the condensate92is wicked from the heat exchanger to the reservoir. The reservoir is positioned where the temperature is relatively warmer than the heat exchanger such that additional condensate does not form at the reservoir. The reservoir is also coated or fully-constructed of a hydrophilic wicking material so that it has a certain capacity to hold water. After some period of condensation formation at the heat exchanger, a certain amount of water is captured in the reservoir. At a later time, If/when the heat exchanger is above the dew point, the captured water is free to wick back toward the heat exchanger and evaporate back into the air. It is illustrated here to give an idea that condensate can be addressed in at least one way; there might be other ways. Also, this approach inFIG.8can be broader and does not have to be tied to a closed-air-loop system.

FIG.9is a flowchart of a process100of operating networking hardware. The process100contemplates implementation in any networking, storage, computing, etc. platform. Further, the process100contemplates networking hardware that can include any of a shelf, a chassis, a cabinet, a module, multiple modules in a housing, a rack-mounted unit (i.e., a pizza box), and the like.

The process100includes steps of operating the networking hardware in a substantially sealed environment where, internal to a housing of the networking hardware, airflow is maintained and circulated therein (step102); cooling the airflow at one or more paths internal to the housing, via one or more liquid cooled heat exchangers (step104); and operating one or more fans disposed, each near the one or more liquid cooled heat exchangers, to provide cool airflow from the liquid cooled heat exchanger to any of a plurality of components internal to the housing (step106). The housing can include a faceplate with no air intake thereon.

The steps can further include directing the cool airflow from the one or more fans via one or more baffles located in the interior of the housing (step108). The steps can further include excluding any of more of a plurality of components from the airflow, via the one or more baffles (step110). The steps can further include operating the one or more fans at a variable speed based on ambient condition in the housing (step112). The steps can further include detecting and mitigating condensation in the housing (step114).

Also, of note, the hybrid liquid and air cooling does not necessarily perform better than external air cooling—as long as it performs comparable, the advantages significantly outweigh any of the performance decreases in terms of heat removal. That is, the benefits include lowered powered fans (less noise, less failure risk), no need for dust removal or air filtering, ability to place racks in a denser configuration, higher density on the housing for ports, displays, etc. It is further noted that external air-cooled systems require extra space in an office or data center for exhaust and air intake as well as high-powered air conditioning in these offices to cool the external air. The approached described herein reduces the air conditioning requirements as well as enables higher density rack placement, e.g., back-to-back and side-to-side.

FIG.10is a diagram of an example module200that is a 1U high rack-mounted unit (“pizza box”).FIG.11is a diagram of the example module200with baffles202and a cooling unit204including two fans206and a liquid cooled heat exchanger208.FIG.12is a close-up view of the cooling unit204including two fans206and a liquid cooled heat exchanger208.FIG.13is a top view of the example module200with the baffles202illustrating air flow at a mid-height of the enclosure.FIG.14is a top view of the example module200with the baffles202illustrating temperature at a mid-height.

FIGS.10-14are presented to illustrate an example module200and performance to show that the hybrid air and liquid cooling can serve to cool such equipment using readily available materials. In an embodiment, the module200is about a 750 W system including devices210,212,214,216,218,220, in a 1U (1.75″ tall) “pizza box” enclosure. 500 W comes from one high-power device220which can be independently cooled via a liquid-cooled cold-plate. The remaining power (˜250 W) is dissipated by the devices210-218ranging in power from 2.4 W to 80 W. These devices210-218are air-cooled within an air circuit inside the enclosure.

The air circuit is itself cooled by a sub-assembly herein called the cooling unit204, i.e., a Circulating Fan and Liquid-Cooled Heat Exchanger. A commercially available fan206was selected; two were used in parallel. Liquid supplied to the heat exchanger208is assumed to be 40° C.

InFIG.11, two different types of heat sinks230,232are used on the devices210-216, based on the baffle202configuration. The devices210,216include a plate fin heat sink230since they are located in a region between the baffles202where airflow is straight, i.e., through the plate fins. The devices212,214include a pin fin heat sink232since they are located in a region between the baffles202where the airflow is curved, i.e., around the pin fins.

The baffles202can be any material that blocks/directs airflow, e.g., injection-molded plastic, blown-foam, etc. These parts serve to constrain the flow to a circuit, with the circulating fan206as the beginning and end point.

InFIG.12, the cooling unit204can include one or more fans206and flow control parts240to ensure all air from the fans206is provided to the liquid cooled heat exchanger208. The liquid cooled heat exchanger208can include a cold plate and fond array.

InFIGS.13-14, a thermal simulation was run to model the thermal performance of the module200. Device temperatures were all verified to be cooler than their respective maximum operating temperature limits. And thus, the viability of this approach is demonstrated.

Although the present disclosure has been illustrated and described herein with reference to preferred embodiments and specific examples thereof, it will be readily apparent to those of ordinary skill in the art that other embodiments and examples may perform similar functions and/or achieve like results. All such equivalent embodiments and examples are within the spirit and scope of the present disclosure, are contemplated thereby, and are intended to be covered by the following claims. Moreover, it is noted that the various elements, operations, steps, methods, processes, algorithms, functions, techniques, etc. described herein can be used in any and all combinations with each other.