Liquid cooling – Will telecom engineers need plumbing courses?

A bit of history

Liquid cooling itself is quite old technology – it’s been around since the late 1800s, originally used for insulating and cooling extra-high-voltage transformers. Auto industry is another example of wide application of liquid cooling – most internal combustion engines use liquid cooling. Liquids, in general, are excellent at transferring heat, and with a bit of chemical engineering, you can fine-tune their boiling and condensation points to optimize heat transfer. But bringing liquids close to expensive electronics was not considered as a good idea.

But Why Now?

Artificial Intelligence and machine learning have created an immense demand for computing power and, consequently, for ultra-high-performance computer chips. Chip makers are racing to develop new AI-grade chips, but with current technology, this entails denser semiconductor packaging and higher clock frequencies, which comes with challenges – power and thermal management.

AI-grade chips consume substantially more energy and generate significantly more heat. The more powerful the AI, the more energy it takes – for example, training GPT-4 required over 50 gigawatt-hours of electricity—approximately 0.02% of California’s annual electricity generation. Many industry professionals are sounding the alarm, warning that AI Is Pushing The World Toward An Energy Crisis. According to Goldman Sachs, data centers currently consume 1-2% of the world’s total power, but this percentage is expected to rise to 3-4% by the end of the decade. The increase in electricity consumption driven by AI, along with its global consequences, deserves a separate discussion. However, it’s also important to note that a significant portion of this energy is converted into heat.

PUE & Cooling

Data centers are continuously working to enhance energy efficiency, with PUE (Power Usage Effectiveness) being the most widely used key performance indicator (KPI). PUE is a simple yet effective metric: it is calculated by dividing the total energy consumed by the data center by the energy used specifically for IT equipment such as servers, storage, and networking devices.

A PUE value below 1.5 is generally considered efficient. Any energy not used by IT equipment is seen as wasteful, prompting efforts to minimize it. This includes energy consumed by security systems, lighting, backup power supplies, and, most significantly, cooling and HVAC systems, which account for about 40% of non-IT energy use.

Over the last decade, air-cooling solutions have dominated data center cooling, effectively managing equipment racks with power densities of up to 20kW. However, as we enter the AI era, power densities are increasing. According to a survey by The Register, 17% of IT professionals report rack power densities between 20kW and 50kW, with some exceeding 100kW.

For such high-power densities, liquid cooling is emerging as the solution of the future. So, what are the advantages and types of liquid cooling?

Advantages of Liquid Cooling

For this we need to refresh our high-school basic physics lessons in our memory, but in general – liquid is much better heat conductor and has much greater Volumetric heat capacity – 1.5m3 of water can accumulate same amount of heat as 5380m3 of air. A liquid can handle significantly higher temperatures as well– the density of a liquid as a material is much higher than that of a gas, such as air. As a result, it can absorb excess heat energy much faster, allowing manufacturers of IT equipment to concentrate heat-emitting components in a much smaller space.

Heat energy can be stored in a liquid and transported – in the case of air cooling, all the dissipated heat energy is released into the atmosphere (which is neither environmentally friendly nor conducive to combating global warming). In contrast, with liquid cooling, this heat energy can be stored and transported for further use. The pleasant side effects of liquid cooling solutions include quiet operation—air cooling fans generate a lot of noise, whereas liquid flows through pipes almost silently—and a much more compact installation. A significantly smaller mass of liquid is needed to dissipate the same amount of energy, whereas air cooling equipment and ducts tend to be quite bulky.

Types of Liquid Cooling

Liquid cooling in the IT industry is a new trend, as many previously did not consider the proximity of liquid and expensive electronics a good idea, nor was there much need for it – power densities of up to 20kW per rack could be managed using traditional air cooling. Currently, various approaches to liquid cooling are rapidly developing, and these are its main types:

• In-Row and In-Rack Liquid Cooling – With the help of various types and technological solutions of Cooling Distribution Units (CDU), cooled water is supplied to racks or rows of racks, absorbs heat, and is then discharged. A major advantage of these solutions is that they do not affect the design of the equipment installed in the racks. Hot air from the equipment racks is collected and dissipated with the help of cooling liquid, while the equipment itself remains of a traditional design. This makes the solution backwards compatible with virtually all IT equipment currently used in the industry. This type of solution is well-suited for co-location data centers, where space is rented in the data center but the exact equipment being used is not known in advance. While traditional air-cooling solutions allow power densities of 10 to 20 kW per rack, such liquid cooling solutions can support densities of 50 kW or more per rack.

• Liquid on Board or Direct-to-Chip Cooling – Also known as cold plate liquid cooling or direct liquid cooling (DLC). This method involves cooling servers by directly transferring heat from their components. In this approach, the IT equipment design is adapted: heat sinks and fans are replaced with cold plates, allowing liquid coolant to be applied directly to the surface of the processor. Instead of water, the cooling liquid typically consists of a water/glycol mixture or a special dielectric fluid (which, although safer, has poorer thermodynamic properties than water). This liquid is circulated from the cold plates to a cooling distribution unit within each rack. This cooling method is primarily used in High-Performance Computing (HPC) systems. Additionally, it is often combined with traditional air cooling, as the liquid cooling technology targets heat extraction from critical areas, while other components still require conventional cooling. This approach allows for power densities of 50 to 100+ kW per rack, with some high-end custom designs exceeding these figures. However, in most real-world data centers, rack densities are maintained around 40–80 kW to ensure operational stability. A significant disadvantage of this technology is the requirement for close collaboration between the customer and the equipment manufacturer to develop custom designs, as there are currently few standardized solutions available on the market.

• Immersion Cooling– This approach involves submerging IT equipment in immersion tanks, which resemble horizontally oriented equipment racks. These tanks contain a special dielectric fluid that fully surrounds the equipment, efficiently dissipating heat. In this case, equipment modifications are also required: fans are removed, thermal grease (commonly used in processor cooling) is cleaned off, and device casings are often removed. These changes are made to ensure the liquid can freely access the equipment. Once prepared, the equipment can be submerged in the liquid, which can vary in type. A useful side effect of Immersion Cooling is that equipment immersed in liquid is completely protected from dust and environmental factors. This technological solution offers an exceptionally high potential for rack power density, with capabilities reaching 200 kW or more per rack. Several drawbacks are worth mentioning. First, racks filled with liquid are heavy and may not be suitable for the load-bearing capacity of all building structures. Second, some of the liquids used contain molecules that, when released into the atmosphere, have a significantly greater potential to contribute to global warming than CO₂.

When will Liquid cooling become mainstream?

Answer is – very soon! Various IT market trend analyst reports predict that the liquid cooling market will grow rapidly in the coming years. Currently (2024), it is valued at approximately $4.48 billion, but by 2029, it is projected to reach $12.76 billion, reflecting significant growth (CAGR 23%).

However, today liquid cooling solutions like Direct Liquid Cooling (DLC) and Immersion Cooling are primarily relevant in the High-Performance Computing (HPC) environment. These solutions are mostly used in hyperscale data centers for large AI businesses and some cryptocurrency mining plants.

Silicon processors continue to develop following Moore’s Law, and their energy consumption is increasing. What is considered high-performance today will soon become the “new normal” at the enterprise level. This shift could be driven by the desire and need of many organisations to implement self-hosted AI solutions for reasons of data security and customization possibilities.

Another factor currently hindering wider adoption and application is the lack of unified standards and interoperability between various components. However, the market often resolves this on its own – the best solutions become so popular that other market players follow their lead.

The great opportunity – Symbiosis and Heat Reuse

Let’s be honest—data centers and AI contribute to global warming – significant portion of consumed energy being released into the air. Liquid cooling technologies share a key advantage: they allow excess energy to be accumulated in a liquid and transported for further use.

Our country – Latvia offers an example. The local telecom operator TET recently announced the construction of a new data center. TET has partnered with the municipality to repurpose the heat generated by the center to warm multi-apartment buildings.

Latvia’s northern climate makes utilizing excess heat for heating one example, but there are other possibilities as well. Many industries, such as chemical manufacturing, food processing, and textile manufacturing, require heat. Additionally, excess heat can be used in agriculture to create optimal growth conditions for plants and crops in greenhouses and indoor farming.

When planning future data centers of any significant size, the aspect of symbiosis and efficient utilization of excess heat will be crucial. The widely used PUE (Power Usage Effectiveness) metric, often highlighted in marketing and for comparing data center efficiency, is no longer the most relevant measure. Efficient reuse of excess heat is becoming increasingly important, which could render PUE obsolete. It may be replaced by metrics such as ERE (Energy Reuse Efficiency), which is like PUE but accounts for effectively reused energy.

Will telecom engineers need plumbing courses?

One thing is clear—working in IT and telecommunications requires continuous learning. While liquid cooling solutions are currently mostly utilized in large hyperscale data centers, their adoption is expected to expand into enterprise and operator infrastructure in the coming years. When that happens, IT and telecommunications engineers will need to familiarize themselves with several previously unfamiliar areas, including:

• Proficiency with piping and pump systems– a skill traditionally associated with plumbing. This is crucial because liquid-cooled systems operate at significantly higher temperatures, and any disruption in cooling circulation can quickly cause irreversible damage to sensitive components. Engineers will need to monitor and optimize heat transfer and energy use effectively.

• Expertise in handling and maintaining electronic components designed for liquid cooling– including installing them in racks, connecting them to cold plate circulators, or immersing them in cooling tanks without contamination. During maintenance, equipment will need to be carefully disconnected or removed from immersion tanks and cleaned of any residual fluid.

• Understanding dielectric fluids, their characteristics, and handling procedures– since these fluids can evaporate or require adjustments when new equipment is installed in a tank. It’s critical to prevent mixing incompatible fluids to avoid unwanted chemical or thermodynamic reactions. Engineers must also manage fluid leaks and mitigate environmental impacts.

These skills will be essential for maintaining and optimizing liquid-cooled IT infrastructure in the near future – so all engineers – there’s a quite a journey in front of us !