Patent Description:
Liquid submersion cooled electronic systems and devices are known. One example of an array of liquid submersion cooled electronic devices is an array of liquid submerged servers (LSS's) arranged in a rack system. An example of an array of LSS's in a rack system is disclosed in <CIT>,<CIT>, and<CIT>. Another example of an array of liquid submersion cooled electronic devices is disclosed in <CIT>. Further examples are reflected in <CIT> and <CIT>.

Liquid submersion cooling systems are described that use a cooling liquid, for example a dielectric cooling liquid, to submersion cool an array of electronic devices. In some embodiments described herein, rather than using pump pressure to return the cooling liquid back to a cooling liquid reservoir, gravity can be used to return the cooling liquid to a reservoir via a cooling liquid gravity return manifold, and a pump can only be used on the liquid supply side to deliver the cooling liquid to each electronic device.

In one example described herein, the electronic devices can each include a device housing that is formed by a tray which can have a completely or partially open top. A plurality of heat generating electronic components are disposed within the interior space of the device housing, with at least some of the heat generating electronic components being submerged within cooling liquid in the device housing. In addition to being submerged in the cooling liquid, a returning supply flow of the cooling liquid may also be directed onto one or more of the heat generating electronic components in what may be referred to as directed flow.

In one embodiment, a liquid submersion cooled electronic device includes a device housing defining an interior space and a maximum dielectric cooling liquid level. Heat generating electronic components are disposed within the interior space of the device housing, and a dielectric cooling liquid is in the interior space with the dielectric cooling liquid submerging and in direct contact with the heat generating electronic components. A dielectric cooling liquid inlet is in the device housing through which dielectric cooling liquid enters the interior space. In addition, a dielectric cooling liquid outlet weir is formed in the device housing from which dielectric cooling liquid exits the interior space without using pump pressure on the return side. The dielectric cooling liquid outlet weir is disposed at the maximum dielectric cooling liquid level of the device housing wherein the cooling liquid outlet weir establishes the level of the dielectric cooling liquid within the interior space and establishes a volumetric rate of flow of the dielectric cooling liquid within the interior space that is needed for the cooling of the heat generating electronic components.

A liquid submersion cooled electronic system can include a plurality of the liquid submersion cooled electronic devices, as well as a dielectric cooling liquid delivery manifold, a dielectric cooling liquid reservoir, a pump, and a dielectric cooling liquid gravity return manifold. The dielectric cooling liquid delivery manifold includes at least one inlet, and a plurality of delivery outlets, and each one of the delivery outlets is fluidly connected to the dielectric cooling liquid inlet of a respective one of the dielectric cooling liquid inlets to deliver the dielectric cooling liquid to the interior space of the respective device housing. The dielectric cooling liquid reservoir that is configured to supply the dielectric cooling liquid. The pump has a pump inlet that is fluidly connected to the dielectric cooling liquid reservoir and a pump outlet that is fluidly connected to the at least one inlet of the dielectric cooling liquid delivery manifold. In addition, each one of the dielectric cooling liquid outlet weirs are fluidly connected to the dielectric cooling liquid gravity return manifold, and the dielectric cooling liquid gravity return manifold is fluidly connected to the dielectric cooling liquid reservoir, whereby the dielectric cooling liquid that exits through the dielectric cooling liquid outlet weirs is returned by gravity to the dielectric cooling liquid reservoir by the dielectric cooling liquid gravity return manifold.

One example application of the liquid submersion cooling devices and systems described herein is for use with an array of LSS's arranged in a rack system. However, the concepts described herein can be used in other applications where arrays of electronic devices are liquid submersion cooled, including, but not limited to, blade servers, disk arrays/storage systems, solid state memory devices, storage area networks, network attached storage, storage communication systems, routers, telecommunication infrastructure/switches, wired, optical and wireless communication devices, cell processor devices, printers, power supplies, etc..

The liquid submersion cooling devices and systems described herein can be used in any area that could benefit from the advantages of liquid submersion cooling. In one example, the liquid submersion cooling devices and systems can be used in blockchain computing (cryptocurrency) applications, for example in either ASIC or GPU computer mining configurations. The liquid submersion cooling devices and systems can also be used in deep learning applications, for example in multi-GPU configurations supporting maximum bandwidth and direct memory access (DMA) of high performance GPUs. The liquid submersion cooling devices and systems can also be used in artificial intelligence and high-performance computing (HPC) clusters with multiple co-processor configurations, for example multi-GPU configurations supporting DMA capabilities of GPU co-processors. Many other applications and uses of the liquid submersion cooling devices and systems described herein are possible and contemplated.

The liquid submersion cooling devices and systems described herein do not require fully sealed electronic device housings. Instead, the trays that contain the electronics can be completely or partially open at the tops thereof helping to, when compared to systems that use fully sealed electronic device housings, reduce costs, and simplifying access to the electronics for service and modifications. Liquid submersion cooling also has superior cooling efficiency compared to air cooling.

Referring to <FIG>, one example of a liquid submersion cooled electronic system <NUM> is illustrated. The system <NUM> includes a housing <NUM> and at least one column of electronic devices <NUM> disposed within the housing <NUM>. A fluid delivery system is used to deliver cooling liquid through the system <NUM>. The fluid delivery system includes a cooling liquid delivery manifold <NUM> that delivers cooling liquid to the electronic devices <NUM> from a cooling liquid reservoir <NUM> for liquid submersion cooling of heat generating electronic components of the electronic devices <NUM>, and a cooling liquid gravity return manifold <NUM> that returns cooling liquid from the electronic devices <NUM> back into the cooling liquid reservoir <NUM> using gravity. A pump <NUM> pumps the cooling liquid from the reservoir <NUM> to the delivery manifold <NUM>.

The fluid delivery system delivers the cooling liquid through the system <NUM>. For example, the pump <NUM> pumps the cooling liquid from the reservoir <NUM> to the delivery manifold <NUM>. The cooling liquid is then delivered from the delivery manifold <NUM> into each electronic device <NUM>. The cooling liquid then exits the electronic devices <NUM> and is discharged into the gravity return manifold <NUM>, where gravity then returns the cooling liquid into the reservoir <NUM>.

The cooling liquid used to cool the electronics in the electronic devices <NUM> can be, but is not limited to, a dielectric liquid. The cooling liquid can be single phase or two-phase. Single phase dielectric cooling liquid is preferred. It is preferred that the cooling liquid have a high enough thermal transfer capability to handle the amount of heat being generated by the submerged heat generating electronic components so that the cooling liquid does not change state. Submersion cooling of the heat generating electronic components means that enough of the cooling liquid is present so that the heat generating electronic components are submerged in the cooling liquid in direct intimate contact with the cooling liquid.

The cooling liquid being returned back to the reservoir <NUM> is at a higher temperature than the cooling liquid delivered to the electronic devices <NUM> since the cooling liquid picks up heat from the heat generating electronic devices. In some embodiments, the returned cooling liquid may be sufficiently cooled through ambient heat exchange with the environment while sitting in the reservoir <NUM>. In other embodiments, where additional cooling is required, the cooling liquid can be directed through a heat exchanger prior to being delivered back to the electronic devices <NUM>.

For example, as illustrated in <FIG>, a heat exchanger <NUM> can be provided that has a heat exchanger inlet fluidly connected to an outlet of the pump <NUM> and an outlet of the heat exchanger <NUM> is connected to an inlet of the delivery manifold <NUM> so that the cooling liquid is cooled prior to entering the delivery manifold <NUM>. The heat exchanger <NUM> can be any heat exchanger that can reduce the temperature of the cooling liquid prior to being delivered to the electronic devices <NUM>. For example, the heat exchanger <NUM> can be a liquid-to-liquid heat exchanger where a heat exchange liquid, including but not limited to a water/glycol mix, is used to exchange heat with the cooling liquid prior to delivery to the delivery manifold <NUM>. The heat exchange liquid can be supplied to the heat exchanger <NUM> via a supply <NUM> and returned via a return <NUM>. In other embodiments, the heat exchanger <NUM> may be a liquid-air heat exchanger where the cooling liquid is cooled by exchanging heat with ambient air, optionally supplemented by a fan that can move air across the heat exchanger <NUM>. The heat exchanger <NUM> can be mounted within the reservoir <NUM>, or on the reservoir <NUM>, or the heat exchanger <NUM> can be separate from the housing <NUM>.

In the illustrated example, the delivery manifold <NUM>, the reservoir <NUM>, the gravity return manifold <NUM>, and the pump <NUM> are shown as being disposed within the housing <NUM>. However, one or more of the delivery manifold <NUM>, the reservoir <NUM>, the gravity return manifold <NUM>, and the pump <NUM> can be located outside of the housing <NUM> as long as the cooling liquid can be delivered to and returned from the electronic devices <NUM>.

The housing <NUM> can have any configuration that is suitable for enclosing the electronic devices <NUM>. The housing <NUM> is preferably enclosed in a manner that minimizes infiltration of dust, bugs, and other foreign contaminants into the interior of the housing <NUM> that might compromise the cooling liquid, the operation of the fluid delivery system, or the operation of the electronics of the electronic devices <NUM>. For example, the housing <NUM> can have a top panel <NUM>, a bottom panel <NUM> opposite the top panel <NUM>, a rear panel <NUM>, a front panel <NUM> opposite the rear panel <NUM>, and opposite side panels (not shown) defining a generally rectangular enclosure defining an interior space. The interior space of the housing <NUM> can include vertically spaced shelves or racks which support the electronic devices <NUM>.

The front panel <NUM> may be hinged to the housing <NUM> to act as a door that can be opened and closed to permit access to the electronic devices <NUM>, the fluid delivery system and other components of the system <NUM>. EMI gaskets or seals may be provided on the housing <NUM> to satisfy FCC emissions or susceptibility requirements. Sound proofing may also be added to the housing <NUM> to minimize system noise transmission to the surrounding environment. The bottom panel <NUM> may be provided with leveling features <NUM>, such as adjustable leveling feet or other leveling features, to permit the housing <NUM>, and the electronic devices <NUM> therein, to be leveled.

The delivery manifold <NUM> is illustrated as being oriented generally vertically with a longitudinal axis thereof parallel to a stacking direction of the electronic devices <NUM>. The gravity return manifold <NUM> is also illustrated as being oriented generally vertically with a longitudinal axis thereof parallel to the stacking direction of the electronic devices <NUM> and parallel to the longitudinal axis of the delivery manifold <NUM>.

The delivery manifold <NUM> extends substantially the height of the electronic devices <NUM> or greater, and includes at least one cooling liquid inlet <NUM> adjacent a base end thereof that is in fluid communication with an outlet <NUM> of the pump <NUM>, for example via the heat exchanger outlet <NUM>. The delivery manifold <NUM> also includes a plurality of liquid delivery outlets <NUM> spaced along the length thereof, such as at least one delivery outlet <NUM> for each electronic device <NUM>. Each one of the liquid delivery outlets <NUM> is fluidly connected to the associated electronic device <NUM> to deliver the cooling liquid to each electronic device <NUM>.

The reservoir <NUM> is illustrated as being located at a base of the housing <NUM>. An internal wall <NUM> separates the interior of the housing <NUM> into an upper space that contains the electronic devices <NUM> and a lower space that contains the reservoir <NUM>. The broken line in <FIG> depicts the level <NUM> of the cooling liquid within the reservoir <NUM>.

<FIG> also illustrates an optional upper cooling liquid reservoir <NUM> that can be provided above the housing <NUM>. The liquid reservoir <NUM> can be used in a gravity fed system where gravity is used to feed the cooling liquid to the electronic devices <NUM> instead of using the pump <NUM>. The pump <NUM> would move the cooling liquid from the lower reservoir <NUM> to the upper reservoir <NUM> which would then feed the vertical delivery manifold <NUM> by gravity. In another embodiment described further below, the upper reservoir <NUM> may serve as an accumulator/pressure tank to provide a buffer volume for a constant pressure cooling liquid delivery scheme.

The pump <NUM> is illustrated as being submerged in the cooling liquid in the reservoir <NUM>. However, the pump <NUM> can be located outside of the reservoir <NUM> as well. The pump <NUM> has a pump inlet (not shown) that is open to the cooling liquid in the reservoir <NUM> to intake cooling liquid, and the outlet <NUM> is fluidly connected to the cooling liquid inlet <NUM> of the delivery manifold <NUM>, for example via the heat exchanger <NUM>. An optional filter <NUM> can be provided to filter the cooling liquid prior to being delivered to the electronic devices <NUM>. The filter <NUM> can also be provided on the pump inlet.

The gravity return manifold <NUM> extends substantially the height of the electronic devices <NUM> or greater, with an upper end <NUM> located at about the uppermost one of the electronic devices <NUM>. In the example illustrated in <FIG>, a lower end <NUM> of the return manifold <NUM> is disposed within the reservoir <NUM> and submerged within the cooling liquid beneath the level <NUM> thereof. After cooling electronic components in the electronic devices <NUM>, cooling liquid is discharged from the electronic devices into the manifold <NUM>. The returning cooling liquid is then returned via gravity back into the bulk cooling fluid within the reservoir <NUM>.

The electronic devices <NUM> can be configured as any array of electronic devices that are liquid submersion cooled. For example, the electronic devices <NUM> can be LSS's, blade servers, disk arrays/storage systems, solid state memory devices, storage area networks, network attached storage, storage communication systems, routers, telecommunication infrastructure/switches, wired, optical and wireless communication devices, cell processor devices, printers, power supplies, and the like, and any combinations thereof.

An example configuration of one of the electronic devices <NUM> is illustrated in <FIG> and <FIG>. Each of the other electronic devices <NUM> can have a similar construction. The electronic device <NUM> includes a device housing <NUM> defining an interior space <NUM>. In the illustrated example, the device housing <NUM> is in the form of a tray that has a completely or partially open top. The tray can be round, triangular, square, rectangular, or any other shape. The illustrated tray is rectangular and has a bottom panel <NUM>, a first end panel <NUM>, a second end panel <NUM> opposite the first end panel <NUM>, a first side panel <NUM>, and a second side panel <NUM> opposite the first side panel. The top of the tray opposite the bottom panel <NUM> is completely or partially open meaning that a panel or other closure does not close the top. The bottom panel <NUM>, the end panels <NUM>, <NUM> and the side panels <NUM>, <NUM> form the interior space <NUM> in which electronic components and the cooling liquid will be contained. The tray is liquid tight to prevent unintended leakage of cooling liquid from the housing <NUM>.

At least one circuit board <NUM> having heat-generating electronic components <NUM> mounted thereon is disposed in the interior space <NUM> of the housing <NUM>. In some embodiments, a plurality of circuit boards, each with heat-generating electronic components mounted thereon can be disposed in the housing <NUM>. The circuit board <NUM> can be mounted in any suitable orientation within the housing <NUM>. In the illustrated example, the circuit board <NUM> is disposed in a horizontal orientation within the interior space <NUM> whereby a plane of the circuit board <NUM> is substantially perpendicular to a longitudinal axis of the delivery manifold <NUM> and the gravity return manifold <NUM>.

The heat-generating electronic components <NUM> can be mounted on the top surface of the circuit board <NUM>, on the bottom surface of the circuit board <NUM>, or on both the top and bottom surfaces of the circuit board <NUM> as illustrated (the broken lines in <FIG> illustrate electronic components <NUM> mounted on the bottom surface of the circuit board <NUM> and therefore not visible in the top view of <FIG>). The heat-generating electronic components <NUM> can be any electronic components that generate heat and that one may wish to cool by submerging the electronic components in the cooling liquid. For example, the electronic components <NUM> can include one or more processors, for example a CPU and/or a GPU, one or more power supplies, one or more switches, and other electronic components.

The electronic components <NUM> that are to be cooled are submerged in cooling liquid within the interior space <NUM>. The broken horizontal line <NUM> in <FIG> indicates a maximum level of the cooling liquid in the housing <NUM> whereby the circuit board <NUM> and all of the electronic components <NUM> mounted thereon are completely submerged in the cooling liquid in direct contact therewith.

The cooling liquid is introduced into the housing <NUM> via a cooling liquid supply line <NUM> that is fluidly connected to one of the liquid delivery outlets <NUM> of the supply manifold <NUM>. A valve <NUM> can be provided in the supply line <NUM> to control the incoming flow of the cooling liquid. The supply line <NUM> is fluidly connected to a multi-port distribution manifold <NUM> of the electronic device <NUM>. In the illustrated example, the distribution manifold <NUM> is disposed within the housing <NUM> and is submerged in the cooling liquid. However, in some embodiments the distribution manifold <NUM> can be located outside the housing <NUM> or located within the housing <NUM> but above the maximum cooling liquid level <NUM>. Alternative to, or in addition to, the distribution manifold <NUM>, the cooling liquid can be input directly into the housing <NUM> via an input line <NUM> shown in broken line in <FIG>.

The distribution manifold <NUM> includes a plurality of distribution outlet ports <NUM>. One or more of the outlets ports <NUM> can be closed, for example by a valve or by a removable cap or plug, to prevent the flow of cooling liquid therefrom and into the bulk cooling liquid in the housing <NUM>. In addition, one or more of the outlet ports <NUM> can be open so that cooling liquid is input directly into the bulk cooling liquid in the housing <NUM>.

In other embodiments, a fluid line(s) or tube <NUM> can be connected to one or more of the outlet ports <NUM>. The fluid line(s) <NUM> is used to direct the return cooling liquid directly onto a respective one of the heat generating electronic components <NUM> in what may be termed directed flow or directed liquid cooling where a flow of the return supply of cooling liquid is directed onto the heat generating electronic component <NUM> which is also submerged within the bulk cooling liquid within the housing <NUM>. In some embodiments, a dispersion plenum housing (not shown) can be disposed over the heat generating electronic component <NUM> and fluidly connected to the end of the fluid line <NUM> to constrain the flow of the returning cooling liquid as it flows over the heat generating electronic component <NUM>. The use of directed liquid cooling and dispersion plenum housings are disclosed in <CIT>.

Still referring to <FIG> and <FIG>, the housing <NUM> further includes at least one cooling liquid return outlet <NUM> through which cooling liquid can exit the housing <NUM> and flow into the gravity return manifold <NUM>. As depicted in <FIG>, the outlet <NUM> is disposed at the maximum cooling liquid level <NUM>. In addition, the outlet <NUM> establishes the level of the cooling liquid within the housing <NUM> and establishes a volumetric rate of flow of the cooling liquid within the housing <NUM> that is needed for the cooling of the heat generating electronic components <NUM>. If the cooling liquid is not at the maximum level <NUM>, then cooling liquid will not flow out of the outlet <NUM>. However, once the cooling liquid reaches the maximum level <NUM>, then the cooling liquid will flow out of the outlet <NUM>, and the cooling fluid will be maintained at the maximum level <NUM>. The outlet <NUM> can have any configuration that controls the level of the cooling liquid to maintain the maximum level <NUM> and allows discharge of the cooling liquid. For example, the outlet <NUM> can be a weir. <FIG> illustrates the weir or outlet <NUM> as extending across the entire width of the second end panel <NUM> of the housing <NUM>. However, the weir or outlet <NUM> can extend over a portion of the width of the second end panel <NUM>.

As used throughout this description and claims, a weir is an outlet for the cooling liquid where the cooling liquid exits the interior space via gravity without using pump pressure connected to the weir. The weir establishes the level of the dielectric cooling liquid within the interior space and establishes a volumetric rate of flow of the dielectric cooling liquid within the interior space that is needed for the cooling of the heat generating electronic components. A weir is different than, and distinct from, an outlet which during use is intended to be connected to a pump so that pump pressure causes the cooling liquid to exit through the outlet, such as the outlets 52b, <NUM> described in <CIT>.

Referring to <FIG> and <FIG>, another embodiment of a liquid submersion cooled electronic device <NUM> that can be used in the system <NUM> of <FIG> is illustrated. The electronic device <NUM> includes a device housing <NUM> defining an interior space <NUM>. In the illustrated example, the device housing <NUM> is in the form of a tray that has a completely or partially open top. The tray can be round, triangular, square, rectangular, or any other shape. The illustrated tray is rectangular and has a bottom panel <NUM>, a first end panel <NUM>, a second end panel <NUM> opposite the first end panel <NUM>, a first side panel <NUM>, and a second side panel <NUM> opposite the first side panel. The top of the tray opposite the bottom panel <NUM> is completely or partially open meaning that a panel or other closure does not close the top. The bottom panel <NUM>, the end panels <NUM>, <NUM> and the side panels <NUM>, <NUM> form the interior space <NUM> in which electronic components and the cooling liquid will be contained. The tray is liquid tight to prevent unintended leakage of cooling liquid from the housing <NUM>.

In another embodiment, a lid can be provided at the top of each housing/tray <NUM> (or the housing/tray <NUM>) to partially or fully close the open top thereof. The use of lids would reduce the potential for particulate and/or water condensation contamination of the cooling fluid. The lids need not be sealed/liquid tight/air tight with the remainder of the housing/tray <NUM> since, for example, there can be one or more openings for any wires to connect to the electronics within the housings/trays, and there may also be at least one outlet opening (described further below or described above as outlet <NUM>) at the housing/lid interface for the cooling liquid return path to the reservoir. The lids, if used, can be mounted to be removable from the housing <NUM> or hinged to the housing <NUM> to provide for easy access to the electronic components.

At least one circuit board <NUM> having various heat-generating electronic components mounted thereon is disposed in the interior space <NUM> of the housing <NUM>. The circuit board <NUM> can be mounted in any suitable orientation within the housing <NUM>. In the illustrated example, the circuit board <NUM> is disposed in a horizontal orientation within the interior space <NUM> whereby a plane of the circuit board <NUM> is substantially perpendicular to a longitudinal axis of the delivery manifold <NUM> and the gravity return manifold <NUM> of the system <NUM>.

A pair of CPU's (hidden by dispersion plenum housings <NUM> discussed further below) are mounted on the circuit board <NUM>. In addition, a plurality of graphics cards <NUM> each including a GPU (hidden by dispersion plenum housings <NUM> discussed further below, only one of which is visible) are mounted on the circuit board <NUM>. In addition, a power supply <NUM> and a data and/or program storage device <NUM> are disposed within the housing <NUM> but not mounted on the circuit board <NUM>.

The electronic components of the electronic device <NUM> that are to be cooled are submerged in cooling liquid within the interior space <NUM>. The line <NUM> in <FIG> indicates a maximum level of the cooling liquid in the housing <NUM> whereby the circuit board <NUM>, the CPUs and the plenum housings <NUM>, the GPUs and the plenum housings <NUM>, other electronic components on the circuit board <NUM> and the graphics cards <NUM> are completely submerged in the cooling liquid in direct contact therewith. In addition, the power supply <NUM> and the storage device <NUM> are also completely submerged in the cooling liquid in direct contact therewith.

The cooling liquid is introduced into the housing <NUM> via a cooling liquid supply inlet <NUM> that is fluidly connected to one of the liquid delivery outlets <NUM> of the supply manifold <NUM> of <FIG> via a cooling liquid supply line similar to the supply line <NUM>. A valve, similar to the valve <NUM> in <FIG>, can be provided to control the incoming flow of the cooling liquid through inlet <NUM>. In this embodiment, the inlet <NUM> is formed in the first end panel <NUM>. The inlet <NUM> is fluidly connected to a multi-port distribution manifold <NUM> of the electronic device <NUM>. In the illustrated example, the distribution manifold <NUM> is disposed within the housing <NUM> and is submerged in the cooling liquid. However, in some embodiments the distribution manifold <NUM> can be located outside the housing <NUM> or located within the housing <NUM> but above the maximum cooling liquid level <NUM>. Alternative to, or in addition to, the distribution manifold <NUM>, the cooling liquid can be input directly into the housing <NUM> via an input line similar to the input line <NUM> shown in <FIG>.

In other embodiments, a fluid line(s) or tube <NUM> can be connected to one or more of the outlet ports <NUM>. The fluid line(s) <NUM> is used to direct the return cooling liquid directly onto a respective one of the heat generating electronic components in what may be termed directed flow or directed liquid cooling where a flow of the return supply of cooling liquid is directed onto the heat generating electronic component which is also submerged within the bulk cooling liquid within the housing <NUM>. In the example illustrated in <FIG> and <FIG>, one of the fluid lines <NUM> extends from a respective one of the ports <NUM> to the dispersion plenum housings <NUM> over the CPUs and to the dispersion plenum housings <NUM> over the GPUs on the graphics cards <NUM>. The dispersion plenum housings <NUM>, <NUM> constrain the flow of the returning cooling liquid as it flows over the submerged CPUs and GPUs, thereby maximizing the cooling of the CPUs and GPUs. The use of directed liquid cooling and dispersion plenum housings are disclosed in <CIT>.

Still referring to <FIG> and <FIG>, the housing <NUM> further includes at least one cooling liquid return outlet <NUM> through which cooling liquid can exit the housing <NUM> and flow into the gravity return manifold <NUM> of <FIG>. The outlet <NUM> is disposed at the maximum cooling liquid level <NUM>. If the cooling liquid is not at the maximum level <NUM>, then cooling liquid will not flow out of the outlet <NUM>. However, once the cooling liquid reaches the maximum level <NUM>, then the cooling liquid will flow out of the outlet <NUM>, and the cooling fluid will be maintained at the maximum level <NUM>. The outlet <NUM> can have any configuration that controls the level of the cooling liquid to maintain the maximum level <NUM> and that allows discharge of the cooling liquid. For example, the outlet <NUM> can be a weir. <FIG> and <FIG> illustrate the weir or outlet <NUM> as being formed in the first end panel <NUM> and extending across only a portion of the end panel <NUM>. However, the weir or outlet <NUM> can be formed in other ones of the panels <NUM>, <NUM>, <NUM> and can extend over the entire length of the panel it is formed in.

Referring to <FIG> and <FIG>, another embodiment of an electronic device <NUM> that can be used in the system <NUM> of <FIG> is illustrated. In this embodiment, the device <NUM> includes a device housing <NUM> or tray that has a completely or partially open top. The tray can be round, triangular, square, rectangular, or any other shape. In this embodiment, the housing <NUM> or tray is much shallower (i.e. has less vertical depth) than the housings <NUM>, <NUM>.

A plurality of electronic devices <NUM> are mounted in the tray. In the illustrated example, three of the devices <NUM> are provided, although a smaller or larger number of systems can be used. In one non-limiting embodiment, each one of the devices <NUM> is an Antminer® system which is bitcoin mining ASIC hardware. Each Antminer® system can include printed circuit boards, for example <NUM> circuit boards, populated with a large number of ASICs which may have passive finned heat sinks bonded to them for additional heat dissipation. A power supply container <NUM> that contains a power supply <NUM> for each one of the electronic devices <NUM> is also mounted in the tray.

A multi-port distribution manifold <NUM> is provided that includes a plurality of distribution outlet ports <NUM>. The manifold <NUM> is fluidly connected to the delivery manifold <NUM> of <FIG> via an inlet <NUM>. Fluid lines <NUM> extend between the outlet ports <NUM> and inlet ports <NUM> of the electronic devices <NUM> and an inlet port <NUM> on the container <NUM>. The fluid lines <NUM> direct the return cooling liquid from the distribution manifold <NUM> into the interiors of the electronic devices <NUM> for submersion cooling of the electronics within the devices <NUM> and into the container <NUM> for submersion cooling of the power supplies <NUM>. The flow of the cooling liquid inside the devices <NUM> is more of a managed bulk flow that distributes the cooling liquid between partitions formed by the circuit boards, submerging the ASICs and other electronic components on the circuit boards in the cooling liquid.

Referring to <FIG>, fluid exits each one of the devices <NUM> via an exit opening or weir <NUM> formed in the side of the device <NUM> opposite the inlet port <NUM>. The devices <NUM> are otherwise generally liquid-tight so that the devices <NUM> fill with the cooling liquid up to the level of the exit openings <NUM>. Once the exit openings <NUM> are reached, the cooling liquid spills out through the exit openings <NUM> and into the tray. The container <NUM> can be open at the top thereof (as illustrated) or the top can be closed. An exit opening <NUM> is formed in a side of the container <NUM> opposite the inlet port <NUM>. The container <NUM> is otherwise generally liquid-tight so that the container <NUM> fills with the cooling liquid up to the level of the exit opening <NUM>. Once the exit opening <NUM> is reached, the cooling liquid spills out through the exit opening <NUM> and into the tray.

Still referring to <FIG> and <FIG>, the housing <NUM> or tray further includes at least one cooling liquid return outlet <NUM> (shown in <FIG>) through which cooling liquid can exit the housing <NUM> and flow into the gravity return manifold <NUM> of <FIG>. In this embodiment, the majority of the cooling liquid is disposed within the electronic devices <NUM> and the container <NUM>, and the tray, which is shallow, holds a relatively minimal amount of the cooling liquid compared to the trays in <FIG> and <FIG>. The outlet <NUM> can have any configuration that allow the cooling liquid to flow out of the tray and into the gravity return manifold. For example, the outlet <NUM> can be a weir. <FIG> illustrates the weir or outlet <NUM> as being formed in a side of the tray opposite the distribution manifold <NUM> and extending across only a portion of the tray. However, the weir or outlet <NUM> can be formed in other sides of the tray and can extend over the entire length of the side it is formed in.

Referring to <FIG>, another embodiment of a liquid submersion cooled electronic system <NUM> will now be described. In this embodiment, elements that are similar in construction and operation to elements in the system <NUM> are referenced using similar reference numbers. This embodiment includes a "positive constant-pressure" cooling liquid delivery system where the operation of the pump <NUM> (<FIG>) is governed to satisfy a min/max set of pressure limits.

For example, in the system <NUM>, a pressure tank <NUM> is connected to the top of the delivery manifold <NUM>. A pressure sensor <NUM> is located at an appropriate location in the system <NUM>, for example on the delivery manifold <NUM> at the approximate level of the <NUM>rd electronic device <NUM> down on the manifold <NUM>. Pressure readings from the sensor <NUM> are used to govern the operation of the pump <NUM> to satisfy a min/max set of pressure limits. The pressure tank <NUM> serves as an accumulator/pressure tank to provide a buffer volume of cooling liquid to ensure that large fluctuations in flow and large variations in pressure are not experienced by the electronic devices <NUM> as a result of time delays that might result as the pump <NUM> reacts to sensor control commands and then has to move the cooling liquid through the heat exchanger <NUM>, hoses, manifolds, etc. to satisfy the desired pressure conditions at the sensor <NUM> and all ports served by the cooling liquid supply.

Still referring to <FIG>, it is desirable that return flow of the cooling liquid from the outlets <NUM> of the housings/trays <NUM> not adversely impact the return flow path of any other housing/tray <NUM> nor feed returning cooling liquid into any housing/tray <NUM> that is at a lower elevation in the system <NUM>. In addition, it is desirable that the returning cooling liquid being returned to the reservoir <NUM> have minimal air entrainment in order to minimize the potential for cavitation events in the pump <NUM>. Further, it is desirable to avoid long vertical drops of the returning cooling liquid in the gravity return manifold <NUM> since long vertical drops would generate unwelcome acoustic noise and result in significant air entrainment as the returning cooling liquid impacts with the surface of the bulk volume of cooling liquid in the reservoir <NUM>.

One way to achieve these goals is illustrated in <FIG>. The outlets <NUM> from the electronic devices <NUM> extend a portion of the width of the electronic device <NUM> as illustrated in <FIG> (similar to the outlets <NUM>, <NUM> in <FIG> and <FIG>). As best seen in <FIG> and <FIG>, the outside surface of the return manifold <NUM> includes a series of cooling liquid outflow ramps <NUM>, one outflow ramp <NUM> for each electronic device <NUM>. The outflow ramps <NUM> have an inlet end <NUM> that receives cooling liquid from the respective outlet <NUM> and an outlet end <NUM> leading into the interior of the return manifold <NUM>. Between the inlet end <NUM> and the outlet end <NUM>, the ramps <NUM> slope gradually downwardly so that the cooling liquid flows from the inlet end <NUM> to the outlet end <NUM> where the cooling liquid then flows into the interior of the return manifold <NUM>. In the illustrated example, the outflow ramps <NUM> of the first group of four electronic devices <NUM> vary in length, with the second group of four electronic devices <NUM> having a similar variation as the first group of four. The inlet ends <NUM> are vertically located one-above the other, but the outlet ends <NUM> are laterally staggered from one another in each group of four electronic devices <NUM> so that the returning cooling liquid discharges into the return manifold <NUM> at staggered locations.

In addition, the interior space of the return manifold <NUM> is configured to prevent long free-falls of the returning cooling liquid. For example, as best seen in <FIG> and <FIG>, a series of flow interrupters <NUM> are provided on the interior of the manifold <NUM> from about the level of the topmost electronic device <NUM> and extending to below the lowermost electronic device <NUM>. The flow interrupters <NUM> are illustrated as a series of walls projecting from the facing front and rear walls of the return manifold <NUM>, with the walls projecting from the front wall alternating with the walls projecting from the rear wall. Cooling liquid entering the return manifold <NUM> through one of the outlet ends <NUM> flows onto the one of the flow interrupters <NUM> projecting from the front wall, which is then directed to flow onto the next flow interrupter <NUM> projecting from the rear wall, which then directs the liquid to flow onto the next flow interrupter <NUM> projecting from the front wall, etc. This cascading of the cooling liquid down the alternating flow interrupters <NUM> within the return manifold <NUM> ensures that the return flow from any one of the outlets <NUM> does not adversely impact the return flow path of any other housing/tray <NUM> nor feed returning cooling liquid into any housing/tray <NUM> that is at a lower elevation in the system <NUM>, helps to avoid long vertical drops of the returning cooling liquid, and minimizes air entrainment by minimizing splashing of the cooling liquid.

<FIG> (together with <FIG>) illustrate another example of a liquid submersion cooling system <NUM>. The system <NUM> can be similar in construction to the system <NUM> or to the system <NUM>. In this embodiment, elements that are similar in construction and operation to elements in the system <NUM> or the system <NUM> are referenced using similar reference numbers. In this embodiment, the reservoir <NUM> includes a contoured floor that controls where water condensate would accumulate if present. In particular, the cooling liquid is typically an oil that has a specific gravity that is less than water. The cooling liquid and the water are non-miscible, so the water will settle out at the bottom of the reservoir <NUM> in the lowest location so long as the slope of the floor of the reservoir <NUM> guides the water droplets to that lowest point. The cooling liquid will simply stratify above the condensed water in the reservoir <NUM>.

As shown in <FIG>, the contoured floor of the reservoir <NUM> is formed with a well or recess <NUM> that is intended to collect water condensate. The floor of the reservoir <NUM> is contoured toward the well <NUM> so that any water condensate accumulates in the well <NUM>. A water sensor <NUM> can be located in the well <NUM> for detecting accumulated water. A mechanism for draining water from the well <NUM> is also provided. For example, the mechanism can comprise a siphon tube <NUM> that can extend from the well <NUM> up and out of the reservoir <NUM> where a siphon pump (not shown) could be attached to draw the water out of the system. Alternatively, a drain port from the bottom of the well <NUM> could be provided with a manually or automatically operated shut off valve to drain water from the well <NUM>.

In the system <NUM>, the inlet of the pump <NUM> should be placed at a suitable location and elevation relative to the well <NUM> to avoid sucking water into the pump <NUM> and forcing the water into the electronic devices <NUM> where the water could contact the electronics. In addition, the inlet for the pump should be relatively low in the reservoir <NUM> from an elevation point of view to minimize bubbles being drawn into the pump <NUM> but not so low that pump <NUM> could be drawing water from the lowest portion of the reservoir <NUM> into the flow of cooling liquid. <FIG> illustrates one example relative location between the inlet <NUM> of the pump <NUM> and the well <NUM>. In this embodiment, the filter <NUM> is located on the inlet <NUM> to filter the cooling liquid before it enters the pump <NUM> rather than being located on the pump outlet.

In addition to the reservoir <NUM> having the well <NUM> to collect water, each of the housings/trays <NUM>, <NUM> that house the electronics can also be configured to accumulate water condensate in a similar manner. For example, like the reservoir <NUM>, the floor of the housings/trays <NUM>, <NUM> can be contoured to direct water condensate into a well formed in the floor of the housings/trays <NUM>, <NUM>. A mechanism for draining accumulated water from the wells in the housings/trays <NUM>, <NUM> can also be provided.

<FIG> illustrate another embodiment of a liquid submersion cooled electronic system <NUM> and liquid submersion cooled electronic devices <NUM> within the system <NUM>. In the system <NUM>, the devices <NUM> are supported on a rack <NUM> in an array composed of a plurality of vertically spaced rows <NUM> of the devices <NUM>. Each row <NUM> is illustrated as including a plurality of the devices <NUM>. However, each row <NUM> can include a single one of the devices <NUM>. In addition, vertical rows are not required, and a plurality of the devices <NUM> can be disposed in a single horizontal array.

The system <NUM> can include a vertical supply manifold <NUM>, a horizontal supply manifold <NUM> for each row <NUM>, a horizontal return manifold or gutter <NUM> for each row <NUM>, a vertical gravity return manifold or gutter <NUM> (best seen in <FIG>), a reservoir <NUM>, a pump <NUM> and a heat exchanger <NUM>. Power to each electronic device <NUM> can be delivered by a power distribution unit <NUM> associated with each row <NUM>.

Referring to <FIG>, the supply manifold <NUM> supplies cooled returning cooling liquid to each of the rows <NUM>. As seen in <FIG>, the supply manifold <NUM> extends upwardly along, and is fixed to, the frame <NUM>. Each of the supply manifolds <NUM> is fluidly connected to the supply manifold <NUM> to distribute the cooled returning cooling liquid to each of the electronic devices <NUM>. Cooling liquid exiting each of the electronic devices <NUM> is discharged into the return manifold <NUM> at each row <NUM>, and each return manifold <NUM> is fluidly connected to the gravity return manifold <NUM>. The gravity return manifold <NUM> is fluidly connected to the reservoir <NUM> which collects the returning cooling liquid which is returned by gravity after cooling the heat generating electronics in the devices <NUM>. An inlet of the pump <NUM> is connected to the reservoir <NUM> to pump cooling liquid from the reservoir to the heat exchanger <NUM> which cools the cooling liquid in any suitable manner including, but not limited to, liquid-to-liquid or liquid-air described above. The now cooled cooling liquid is then output from the heat exchanger <NUM> into the manifold <NUM> for distribution to each row <NUM>.

Referring to <FIG>, <FIG>, each electronic device <NUM> is fluidly connected to its respective supply manifold <NUM> via a supply line or supply tube <NUM> extending from the manifold <NUM> to a cooling liquid inlet <NUM> of the device <NUM>. Each of the devices <NUM> can be similar in construction to the electronic devices <NUM>. For example, each one of the devices <NUM> includes a housing <NUM> defining an interior space with heat generating electronics disposed therein and submerged within the cooling liquid. In one non-limiting embodiment, each one of the devices <NUM> is configured as an Antminer® system which is bitcoin mining ASIC hardware. Each Antminer® system can include printed circuit boards, for example <NUM> circuit boards, populated with a large number of ASICs which may have passive finned heat sinks bonded to them for additional heat dissipation. Some electronics <NUM> of the devices <NUM> can be disposed at the top of each device <NUM> and not submerged in the cooling liquid. Power supplies <NUM> for the devices <NUM> can be disposed next to each device <NUM> and the power supplies <NUM> can be air cooled or cooled by liquid submersion.

The inlet <NUM> is disposed in a first or front side of the housing <NUM> to input cooling liquid into the interior space of the housing <NUM>. An exit opening or weir <NUM> is formed in a second or rear side of the housing <NUM> opposite the side with the inlet <NUM>. Cooling fluid exits each one of the devices <NUM> via the exit opening or weir <NUM> formed in the side of the device <NUM> opposite the inlet port <NUM>. As best seen in <FIG>, the exit openings <NUM> are connected to the return manifolds <NUM> so that the exiting cooling liquid is collected by the return manifolds <NUM>, which in turn are fluidly connected to the gravity return manifold <NUM> to return the cooling liquid to the reservoir <NUM>. The top of the housing <NUM> is open (i.e. there is no lid that seals the housing <NUM>). The housings <NUM> are otherwise generally liquid-tight so that the devices <NUM> fill with the cooling liquid up to the level of the exit openings <NUM>.

<FIG> illustrate another embodiment of a liquid submersion cooled electronic system <NUM> that is similar to the electronic system <NUM>, and elements in the system <NUM> that are similar to the system <NUM> are referenced using the same reference numerals. The system <NUM> uses a smaller reservoir <NUM> compared to the reservoir <NUM> used in the system <NUM> with the smaller reservoir <NUM> mounted beneath the rack <NUM>. In addition, an auxiliary reservoir 468a is provided that is fluidly connected to the reservoir <NUM>. The auxiliary reservoir provides additional capacity for holding the cooling liquid when filling and draining the devices <NUM> in the entire rack <NUM>. The configuration of the system <NUM> allows room for an additional row <NUM> of the devices <NUM> on the rack <NUM> compared to the system <NUM> in <FIG>.

<FIG> illustrate another embodiment of a liquid submersion cooled electronic system <NUM> that is similar to the electronic systems <NUM> and <NUM>, and elements in the system <NUM> that are similar to the systems <NUM> and <NUM> are referenced using the same reference numerals. In the system <NUM>, each of the rows <NUM> includes its own pump <NUM> and heat exchanger <NUM>. Referring to <FIG>, each row <NUM> includes the supply manifold <NUM> and the return manifold <NUM>. The supply manifold <NUM> supplies the cooling liquid to each one of the devices <NUM> in that row, and after cooling the electronics, the cooling liquid exits each device through the exit opening <NUM> and into the return manifold <NUM> of the row. The inlet of the pump <NUM> is connected to the return manifold <NUM> to pump the returning cooling liquid from the manifold <NUM>, through the heat exchanger <NUM>, and into the supply manifold <NUM>. The heat exchanger <NUM> can be air cooled, or a secondary fluid loop for liquid-to-liquid cooling can be connected to each heat exchanger <NUM> via a vertical manifold (not shown).

The configuration of the system <NUM> eliminates the need for the vertical supply and return manifolds <NUM>, <NUM> of the systems <NUM>, <NUM>. In addition, the configuration of the system <NUM> allows room for an additional row <NUM> of the devices <NUM> on the rack <NUM> compared to the system <NUM> in <FIG>. In addition, keeping the cooling fluid loops in each row <NUM> allows the pumping by the pumps <NUM> to be more efficient and simplifies flow proportioning to the devices <NUM>.

<FIG> illustrate another embodiment of a liquid submersion cooled electronic system <NUM>. The system <NUM> includes a plurality of device housings <NUM> supported on a rack <NUM> in an array composed of a plurality of vertically spaced rows <NUM> of the housings <NUM>. Each row <NUM> is illustrated as including a plurality of the housings <NUM>. However, each row <NUM> can include a single one of the housings <NUM>. In addition, vertical rows are not required, and a plurality of the housings <NUM> can be disposed in a single horizontal array.

Each of the housings <NUM> includes a liquid tight bottom tray <NUM> that defines an interior space that is intended to contain the cooling liquid, and a removable top cover <NUM> disposed over the tray <NUM> that closes off the interior space and prevents ingress of contaminants into the cooling liquid in the tray <NUM>.

As best seen in <FIG>, a plurality of electronic devices <NUM> are disposed within each of the housings <NUM>. Each of the devices <NUM> can be configured similar to the devices <NUM>. In particular, each of the devices <NUM> can be configured as an Antminer® system which is bitcoin mining ASIC hardware as described above. In the illustrated example, ten of the devices <NUM> are illustrated in each one of the housings <NUM>. However, a smaller or larger number of the devices <NUM> can be disposed in each of the housings <NUM>.

Each of the devices <NUM> includes an open top device housing <NUM> defining an interior space containing heat generating electronic components <NUM>. The device housings <NUM> are configured to contain the cooling liquid therein with at least some of the electronic components <NUM> submerged therein. As seen in <FIG>, some of the electronic components can be covered by a dispersion plenum housing <NUM> whose function is similar to the dispersion plenum housings <NUM>, <NUM> in <FIG>. Cooling liquid can enter each of the housings <NUM> via an inlet line <NUM> connected to each plenum housing <NUM> which constrains the entering cooling liquid as it flows over a heat generating component <NUM>, such as a CPU or a GPU. The entering cooling liquid then enters the bulk cooling liquid contained in the housing <NUM>. Cooling liquid exits each housing <NUM> via an exit opening or weir <NUM> (similar to the exit openings <NUM> in <FIG>) formed in each housing <NUM>, with the exiting cooling liquid then falling by gravity into the bulk liquid contained in the interior space of the tray <NUM>.

In addition, at least some of the electronics of the devices <NUM> are submerged in the bulk cooling liquid that is contained within the bottom tray <NUM> of the housing <NUM>. For example, as illustrated in <FIG>, a power supply <NUM> for each device <NUM> can be disposed beneath each device <NUM>, with each power supply <NUM> partially or fully submerged within the bulk cooling liquid in the tray <NUM>. The bulk cooling liquid in the tray <NUM> can be maintained at a level <NUM> to ensure that the power supplies <NUM> or other electronics are adequately submerged.

The cooling liquid is circulated in the system <NUM> by a pump <NUM>. The pump <NUM> can be disposed within the housing <NUM> or at any suitable location for performing its pumping function. The pump <NUM> has an inlet <NUM> that is disposed within the bulk cooling liquid within the tray <NUM>, and pumps the cooling liquid to a heat exchanger <NUM> for cooling the cooling liquid. The heat exchanger <NUM> can be located within the housing <NUM> or at any suitable location for performing its heat exchange function. The heat exchanger <NUM> can be connected to an external cooling fluid loop <NUM> for example for liquid-to-liquid heat exchange, or the heat exchanger <NUM> can cool the cooling liquid via liquid-to-air heat exchange. The outlet of the pump <NUM> is connected to a supply manifold <NUM> within the housing <NUM> to which the inlet lines <NUM> are fluidly connected to return the cooled cooling liquid to each of the devices <NUM>.

In the illustrated system <NUM>, the cooling liquid never leaves the housings <NUM>. The cooling liquid enters each of the housings <NUM> of the devices <NUM>, cools the electronics therein, then exits the exit opening or weir <NUM> and falls by gravity into the bulk cooling liquid within the tray <NUM>, where the cooling liquid is then pumped to the heat exchanger for cooling, and then pumped back into the housings <NUM> of each device <NUM>. At the same time, the bulk cooling liquid in the tray <NUM> cools the power supplies <NUM> and any other portions of the system <NUM> in contact with the cooling liquid. The configuration of the system <NUM> eliminates the need for a separate fluid reservoir as well as supply and return manifolds.

<FIG> illustrates another embodiment of a liquid submersion cooled electronic system <NUM> that is similar to the system <NUM> and like elements are referenced using the same reference numerals. In the system <NUM>, the housings <NUM> in each row <NUM> share a pump and heat exchange module <NUM> that contains a pump <NUM> and a heat exchanger <NUM>. The cooling liquid from the tray <NUM> of each housing <NUM> is pumped by the pump <NUM> to the heat exchanger <NUM> which cools the cooling liquid before returning the now cooled cooling liquid to the devices <NUM> via the supply manifolds <NUM>. The heat exchanger <NUM> can be configured to cool the cooling liquid via heat exchange with a secondary coolant loop carrying a secondary coolant liquid, or if determined to be adequate the cooling can be achieved using air. The arrows in <FIG> illustrate the flow of the cooling liquid through the module <NUM>. Since the housings <NUM> share the module <NUM>, the cooling liquid can be shared between the two housings <NUM>.

<FIG> illustrate another embodiment of a liquid submersion cooled electronic system <NUM>. In this embodiment, the system <NUM> is configured as a dense GPU or CPU array with a plurality of electronic device housings <NUM> suitably supported on a rack. As seen in <FIG>, each housing <NUM> comprises an open top, liquid tight tray <NUM> that can be covered by a removable cover if desired. The tray <NUM> is configured to contain a cooling liquid therein. A plurality of cards or circuit boards <NUM> are disposed within the tray <NUM>. In the illustrated example, the cards <NUM> are arranged vertically and parallel to one another. The cards <NUM> are configured as graphics processing units (GPUs) or as central processing units (CPUs), with each card <NUM> having one or more processors mounted thereon.

A supply manifold <NUM> supplies returning cooling liquid, after being cooled, to each card <NUM>. The supply manifold <NUM> is illustrated as being disposed outside of the interior space of the tray <NUM>. However, the supply manifold <NUM> can be positioned at other locations. The opposite side of the tray <NUM> includes an exit opening or weir <NUM> through which cooling liquid exits the tray <NUM>. As best seen in <FIG>, cooling liquid that exits the openings <NUM> falls by gravity into return gutters or manifolds <NUM> that return the cooling liquid to a reservoir, where the liquid is cooled by a heat exchanger (not shown) and pumped by a pump (not shown) back to each row of housings <NUM> on the rack via a return manifold and ultimately into the supply manifolds <NUM> for return to the trays <NUM>.

Claim 1:
A liquid submersion cooled electronic device, comprising:
a device housing (<NUM>; <NUM>; <NUM>; <NUM>; <NUM>, <NUM>, <NUM>) defining an interior space (<NUM>; <NUM>) and a maximum dielectric cooling liquid level (<NUM>; <NUM>);
heat generating electronic components (<NUM>; <NUM>, <NUM>, <NUM>; <NUM>; <NUM>) disposed within the interior space of the device housing;
a dielectric cooling liquid in the interior space (<NUM>; <NUM>), the dielectric cooling liquid submerging and in direct contact with the heat generating electronic components; and
a dielectric cooling liquid inlet (<NUM>; <NUM>; <NUM>, <NUM>; <NUM>; <NUM>) in the device housing through which dielectric cooling liquid enters the interior space (<NUM>; <NUM>);
characterized by a dielectric cooling liquid outlet weir (<NUM>; <NUM>; <NUM>, <NUM>; <NUM>; <NUM>, <NUM>) in the device housing from which dielectric cooling liquid exits the interior space (<NUM>; <NUM>); wherein the dielectric cooling liquid outlet weir is disposed at the maximum dielectric cooling liquid level of the device housing (<NUM>; <NUM>; <NUM>; <NUM>; <NUM>, <NUM>, <NUM>); and wherein the cooling liquid outlet weir establishes the level of the dielectric cooling liquid within the interior space (<NUM>; <NUM>) and establishes a volumetric rate of flow of the dielectric cooling liquid within the interior space (<NUM>; <NUM>) that is needed for the cooling of the heat generating electronic components (<NUM>; <NUM>, <NUM>, <NUM>; <NUM>; <NUM>).