Data center challenges are mounting fast. These are fueled both by the rise of next-gen applications, super-hot, largely GPU-driven applications like AI, and the fact that business and IT growth must increasingly move in lockstep. Not surprisingly, as temperatures rise inside data centers, operators are scrambling to find workable, more power efficient alternatives to traditional air-cooling. In this second in a series of blogs comparing data center cooling technologies we’ll discuss arguably the most viable and practical of these – liquid immersion cooling, and contrast the two very different approaches of this seemingly similar method: two phase versus single-phase immersion cooling.
Our Goal: help you understand the differences and find the best cooling technology to grow and future-proof your data center.
Matchup: How These
Competing Cooling Technologies Work
The underlying principle behind both single – and two-phase immersion cooling is that liquid is a much better heat conductor than air. The other reason that makes both these immersion-based systems superior is the proximity of the liquid to the heat source. (Wet is about as close as you can get.) Furthermore, both options require no chillers, CRACs, CRAHs or raised floors. Apart from that there are significant differences in the two technologies.
Two-Phase Cooling
In a two-phase immersion cooled system, servers are sealed inside a bath of specially engineered fluorocarbon-based liquid. Because the fluid has a low boiling point (often below 50°C vs. 100°C for water), heat from the servers easily boils the surrounding fluid. The boiling of the liquid causes a phase change (from liquid to gas), which gives two-phase immersion cooling its name.
The vapor is then condensed back to the liquid form via water-cooled condenser coils, which are integrated into the top of the sealed racks. The condensed liquid drips back into the bath of fluid to be recycled through the system.
Single-Phase Cooling
With single-phase immersion cooling, servers are installed vertically in coolant bath of a hydrocarbon-based dielectric fluid that’s similar to mineral oil. Like its two-phase counterpart, heat is transferred to the coolant through direct contact with server components. But, unlike two-phase immersion cooling, the coolant does not boil off. Instead, it remains in the liquid phase (hence the name) and is cooled via a heat exchanger in a cooling distribution unit (CDU).
Many installations that have switched to single-phase immersion cooling have been impressed with its simplicity, which translates into greater upfront affordability, easier operations, and less maintenance.
Compare, Contrast and Be Cool
To help with your decision making we’ll now break down each of these technologies and see how they compare across four important categories.
Complexity & Upfront Costs
Two-Phase Cooling
Two-phase immersion cooling systems are a bit more complex in their engineering and present higher up-front costs. While pumps are not needed to circulate fluid around the servers, sealed racks with integrated condenser coils are required to make it all work.
The biggest cost difference between single – and two-phase relates to the coolant. Fluorocarbon-based two-phase coolants such as 3MTM NovecTM typically cost an order of magnitude more than hydrocarbon-based ones like ElectroSafe® from GRC. When you’re using over 300 gallons of the fluid in a single rack, those additional costs can be significant.
What’s more, the enclosure in which racks are immersed in the Novec must be completely sealed to avoid losing that extremely expensive coolant. Since servers often require maintenance, this can create annoying access issues and add operating costs related to topping up fluid that evaporates. In fact, a study by the Lawrence Berkeley National Lab (LBNL) found that the cost of replacing evaporated fluid negated any energy savings two-phase cooling offered.
Apart from the cost of evaporation, prolonged exposure to the vapor can cause health and environmental concerns. Fluorocarbon-based coolants often have extremely high GWP (Global Warming Potential) and are highly regulated.
Single-Phase Cooling
Although it requires a pump to keep the coolant circulating through the system, single-phase immersion cooling wins the complexity category. For example, GRC’s ICEraQ™ micro-modular immersion cooling solution features an open rack design allowing free access to servers. The synthetic hydrocarbon-based fluids, such as GRC’s ElectroSafe, in this process are clear, odorless, non-toxic, and widely used in domestic products such as cosmetics. They also have well-established material handling practices, minimal regulations, and no direct GWP (Global Warming Potential).
Efficiency & Operating Expenses
Two-Phase Cooling
Both single– and two-phase immersion cooling systems blow the doors off air-cooled data centers when it comes to cooling efficiency. But two-phase just squeezes by to claim first prize with a PUE of 1.01 to 1.02 versus 1.02 to 1.03 for single-phase.
However, unlike ElectroSafe™, fluorocarbon-based coolants must be regularly replenished – again at hundreds of dollars per gallon. That’s enough to raise overall operating expenses and offset any small power efficiency gains, to say nothing of total cost of ownership (TCO).
While two-phase systems are energy-efficient, pulling maintenance on them can be complicated. To access the servers, operators must unseal the enclosure while trying to minimize coolant loss and inhalation of vapor.
Single-Phase Cooling
At an impressive 1.02-1.03 PUE, single-phase utterly crushes the relative energy inefficiency of non-immersion-cooled systems which, according to the Uptime Institute, averages out to a 1.67 PUE.1
Where consumables are concerned, ElectroSafe is far less expensive upfront, does not evaporate, and will easily last 15 or more years. Regular lab tests are performed on the coolants to test for any variance in critical properties over time. Commercial deployments dating back to as early as 2010 have been running efficiently and reliably, with no degradation in the coolant’s performance.
Plus, compared to two-phase systems, server maintenance on GRC’s ICEraQ micro-modular immersion cooling solutions is very easy. You just open the lid, lift out the server and set it on integrated service rails, which are placed at a convenient workbench height.
Cooling Capacity & High-Density Performance
Two-Phase Cooling
Two-phase coolants can typically support extreme rack densities, with some providers claiming to support up to 250 kW in a single rack! While the claim sounds extremely impressive, it’s hard to find commercially available hardware that will get you to even 100 kW.
Single-Phase Cooling
By contrast, GRC’s ICEraQ is capable of handling 200 kW per rack when attached to a chilled water system. In fact, we’ve had a customer support over 130 kW within a 52U rack with cooling tower water. Given the anticipated long-term “heat trend” for incorporating next-gen applications like AI, IoT, facial recognition and others, that’s more than enough cooling performance for the foreseeable technology evolution.
Reliability & Location Flexibility
Two-Phase Cooling
Both single – and two-phase immersion cooling systems on the market today give information and communications technology (ICT) leaders tremendous flexibility in locating their data centers. This is due to the fact that neither requires the extensive infrastructure traditional air-cooled installations do.
However, studies have shown that fluorocarbon-based fluids used in two-phase immersion cooling present several safety and reliability issues GRC’s ElectroSafe doesn’t. For one, fluorocarbon-based fluids have a high Global Warming Potential (GWP). While the 2016 Lawrence Berkeley National Lab (LBNL) study found no harmful breathing zone concentrations during normal operations,2 some industry pundits have rightly expressed concerns about exposing data center personnel to this substance in any amounts.
That same study also found that “repeated logic board…failures were responsible for the high number of service requests.” Indeed, these amounted to 368% of the total cost of IT equipment energy consumption.3
Next, some experts have suggested that the boiling action on which two-phase cooling depends may harm IT assets over time through a process known as cavitation. Seals for the coolant bath can also fail. Plus, the expensive Novec evaporates quickly, leaving the rack empty and without cooling.
Finally, it should be noted that the condenser that’s placed directly over the servers in two-phase systems is water-cooled. While the two are unlikely to come in contact, there are widespread concerns about having a conductive liquid directly above the IT rack.
Single-Phase Cooling
Since pioneering single-phase immersion cooling back in 2009, GRC has made reliability one of the hallmarks of our ICEraQ and ICEtank solutions. Posing virtually no maintenance issues, both these systems can consistently cool 200 kW/rack using our dielectric coolant, ElectroSafe, which is totally inert, non-conductive, non-flammable, non-corrosive, and does not need replacement over the life of a typical data center.
Hardware reliability data for our systems show a significant increase in the Mean Time Between Failures (MTBF) over traditional air-cooled data centers. This increase, in part, is attributable to our close collaboration with key OEM partners such as Dell, Hewlett Packard Enterprise, 2CRSI, iXSystems, AMAX, and Supermicro.
Conclusion: Single-Phase Wins by Keeping its Cool
Without Boiling Away Costs
While it boasts some impressive theoretical performance metrics and considerable potential, two-phase immersion cooling is very complex, far too expensive and overly engineered for today’s data center cooling needs.
To that point, back in 2016 Lawrence Berkeley Laboratory concluded that two-phase immersion cooling was not viable, with liquid cost and IT equipment failures as primary obstacles.4 Since then, there have been no large-scale, public-enterprise deployments of the technology to disprove those findings.
Single-phase immersion, on the other hand, has been proven cost-effective and highly reliable by GRC over the course of a decade. Our ICEraQ and ICEtank solutions are now helping some of the world’s largest cloud, enterprise, government, education and telecom organizations grow while future-proofing their data centers.
1 Andy Lawrence, “Is PUE actually going UP?,” https://journal.uptimeinstitute.com, (May 2019)
2 Coles, Henry and Herrlin, Magnus. Immersion Cooling of Electronics in DoD Installations. Lawrence Berkeley National Laboratory. May 2016
3 Coles, Henry and Herrlin, Magnus. Immersion Cooling of Electronics in DoD Installations
4 Coles, Henry and Herrlin, Magnus. Immersion Cooling of Electronics in DoD Installations
Single-Minded About Growing Your Data Center? Let’s Talk!
Send us an email at info@www.grcooling.com or call us 1.512.692.8003. A GRC associate will reach out to talk details with you.
In the meantime, be sure to read Data Center Cold Wars — Part 1: Air-Based Cooling Versus Single-Phase Immersion Cooling and watch for the next installment.
