The Complete Guide to Data Center Cooling

down arrow

Introduction

Many factors go into designing and building a reliable and efficient data center, one of which is cooling. Cooling plays a critical role due to the immense heat IT equipment produces. This equipment must be kept cool to function properly
throughout its intended life.

In this guide, we’ll explore the most common forms of data center cooling technology available today and discuss how you can make the best selection for your application.

Data center owners have more cooling options available today than at any time in the past. During data center design or redesign, companies must consider which cooling method or combination of methods will best serve their application. This starts with a basic understanding of various available technologies.

First, it’s helpful to understand that IT cooling solutions fall into two broad categories: air cooling and liquid cooling. Let’s take a closer look at the cooling technologies within these categories.

Man and woman talking in front of a server rack

 

Download a PDF version of this guide by filling out the form below, or keep scrolling to read.

The Complete Guide to Data Center Cooling

Chapter 1

Data Center Cooling Methods

Air cooling remains the most common method for data center cooling. These solutions use chilled air, either cooled by a refrigerant or a chilled water system, to remove the heat generated by the IT equipment. Room-based and close-coupled cooling are two forms of air cooling used in data centers.

Air Cooling Methods

1. Room Cooling

Computer room air conditioners (CRAC) and computer room air handlers (CRAH) are the most traditional options for data center cooling. These
legacy systems are still a popular choice in many instances.

CRAC/CRAH Systems

Although these terms are used interchangeably in the industry, there are technical differences between CRAC systems—which use refrigerant,
typically referred to as direct expansion (DX), and generally are of lower capacity—and CRAH systems, which use chilled water and generally are of higher capacity.

Both systems typically involve placing cooling units around the perimeter of the data center to cool the entire IT space. They can be configured to
blow air overhead, but most employ a raised-floor setup. IT systems are mounted in racks on top of the raised floor, with the underfloor plenum
pressurized with cool air. These racks are arranged into “hot aisles” and “cold aisles.”

Perforated floor tiles are placed in the cold aisles to allow cold air from under the floor to be distributed to the server inlets, whereas the hot aisles
return hot exhaust air to the perimeter units. Varying levels of partitioning may be used to contain and segregate the hot and cold air streams.
Partitioning is important because short-circuiting hot air into the cold aisle can affect server performance and energy efficiency.

Other best practices include using filler panels in the IT racks to prevent air recirculation through the rack and sealing cable cutouts in the raised
floor to prevent air mixing and ensure cold air is delivered only to the server inlets. CRAC/CRAH systems are typically used for rack densities up
to 10 kilowatts (kW) per rack.

Perforated floor tiles are placed in cold aisles to allow cold air from under the floor to be distributed to the server inlets.

Free Air Cooling

When it comes to air cooling methods, you may see that some data centers use some form of “free air cooling.” This method takes advantage
of cooler ambient temperatures to feed outside air into the facility. In most cases, this is a building-level system, but we include it here because of its popularity among large-scale data center providers.

Free air cooling is an eco- and budget-friendly means of cooling. However, this arrangement is subject to air quality and humidity issues,
necessitating a filtration system. Such systems require the owner or operator to accept the risk of running the data center at higher
temperatures on occasion. However, these risks are accepted in the interest of energy conservation and sustainability.

2. Close-Coupled Cooling

As IT density increases, perimeter room cooling becomes more difficult to manage. Another approach to air cooling is close-coupled cooling, which—as the name implies—places cooling closer to the source of the IT heat.

In addition to higher densities, close-coupled cooling is a good option for small- and medium-sized facilities. According to Mission Critical
Magazine, close-coupled cooling technology can support densities of anywhere from 1-50 kW per rack, depending on the design. Some data
centers may employ a room cooling system and add close-coupled cooling solutions only in higher-density areas. Close-coupled cooling solutions can be placed within the rows of servers or attached directly to the IT racks.

In-Row Cooling

In-row cooling units are positioned between server cabinets and commonly come in chilled water or direct expansion (DX) varieties. These
systems operate using a horizontal airflow pattern and can be configured with hot aisle containment to prevent air from mixing. In-row setups can be used for densities up to approximately 20 kW, but higher capacity systems take up more space.

In-row units are typically narrower than server racks and fit compactly in between racks.

Rear-Door Heat Exchangers

Rear-door heat exchangers (RDHx) move the cooling even closer to the IT equipment, attaching it directly to the backs of racks. As the name suggests, this technology replaces standard rack doors with heat exchangers. As air flows through the rack, it is heated by the equipment and immediately cooled by a liquid medium flowing through the RDHx before reentering the room.

There are two main types of rear-door heat exchangers:

  • Passive RDHx:
    • Passive heat exchangers are typically used for densities of about 20-30 kW per rack. These systems do not use fans beyond those in the servers to move air through the system.
  • Active RDHx:
    • Higher densities call for active heat exchangers, which include high-efficiency fans to assist the air through the system.
      These solutions may be suitable for densities of up to 50 kW per cabinet.

Compared to in-row cooling solutions, RDHx offers a more space-efficient solution, especially at higher densities.

3. Liquid Cooling Methods

As density increases, data centers must consider liquid cooling. Liquid cooling methods are more advanced to support high-
density cooling.

Data centers can take two main approaches to liquid cooling: direct-to-chip and immersion. Let’s take a closer look at these
options and how they differ.

Direct-to-Chip Cooling

The direct-to-chip cooling method uses a liquid coolant that runs through cold plates attached to select high-powered electronic devices. Heat transfer occurs via conduction between the electronic package and the cold plate. The liquid transfers the heat to a cooling system that ultimately rejects the heat to the environment.

Because direct-to-chip cooling targets specific device(s) on the server boards, the remaining heat-generating components still require cool air to remove their heat. Therefore, this cooling method must be combined with room or close-coupled cooling to remove all generated heat.

Immersion Cooling

One of the latest developments in data center cooling solutions is liquid immersion cooling. The market for this cooling technology is expected to register a compound annual growth rate (CAGR) of more than 26 percent over the next five years. This growing popularity can be attributed to the impressive efficiency immersion cooling offers in high-density deployments.

Immersion cooling device
Immersion cooling is unique from all other cooling technologies we’ve covered thus far because it involves fully submerging servers and other computing equipment directly into dielectric fluid. This fluid bath can remove all heat from the IT equipment without shorting out the boards.

There are two different approaches to immersion cooling to be aware of:

  • Single-phase
    • In single-phase immersion cooling, hydraulic pumps push fluid through large tanks. After the fluid passes around the server boards and absorbs heat, the warmed fluid exits the tank and passes through a heat exchanger, where it is cooled before cycling back into the tank.
  • Two-phase
    • Two-phase immersion cooling does not move fluid in and out of tanks. Instead, the fluid in the tank boils as it heats up, causing the liquid to vaporize. This vapor then condenses on a heat exchanger where another fluid carries the residual heat to the environment. Gravity returns the condensed vapor to the bath, where the process repeats.

Although both options are considered highly efficient systems, two-phase immersion cooling offers a more self-contained and compact design,
making it a more desirable choice for some data centers. It also typically comes at a lower cost due to fewer components.

In two-phase immersion cooling, the fluid in the tank boils as it heats up, causing the liquid to vaporize.

Chapter 2

Cooling Systems Outside the Data Center

Though the purpose of this guide is to focus on IT cooling solutions, it’s worth noting that a data center’s total cooling solution extends beyond this equipment.

Outside the data center’s whitespace, you’ll find the portion of the overall cooling system that rejects IT residual heat to the outside environment. For large data centers, this commonly consists of large chillers and evaporative cooling towers or arrays of dry chillers.

For small data centers utilizing DX cooling, heat rejection typically consists of condenser units similar to those utilized in residential air conditioning systems. These technologies are used in an array of applications and are not unique to data centers.

For large data centers, this is commonly composed of large chillers and evaporative cooling towers or arrays of dry chillers.

 

Chapter 3

Choosing the Right Cooling Solution for Your Data Center

Now that we’ve discussed the variety of data center cooling systems available, it’s time to turn our attention to your data center. How can you choose the best cooling solution for your application?

Because cooling is such a critical aspect of your data center, you must determine your cooling needs early in the data center design process. Consider these factors to choose the right solution for your application:

Data Center Cooling Factors

1. Density

A data center's density is a function of the number of servers contained and the amount of power consumed for each square foot of floor space. Higher densities present a greater challenge for removing the residual IT heat.

As a rule of thumb, room cooling methods work best at low densities. As density increases, data centers must find ways to bring cooling solutions closer to the IT equipment, whether that is a close-coupled cooling solution or a form of liquid cooling.

As a rule of thumb, room cooling methods work best at low densities.

2. Size

The size of your data center is another critical factor and must be considered in conjunction with data center density. A prudent
cooling solution for a small modular data center (MDC) may be a poor choice for a large hyperscale data center.

In addition to impacting cooling requirements, the size and layout of your data center will dictate how much space is available for cooling equipment. You should always consider the overall size of the real estate parcel and, if space is a concern, opt for a compact cooling solution. This is often a priority for colocation providers because less space given to cooling infrastructure means more usable space for IT racks.

You should always consider the overall size of the real estate parcel and, if space is a concern, opt for a compact cooling solution.

3. Total Cost of Ownership

Cost is always a driving factor in decision-making. Some cooling technologies are generally more cost-prohibitive than others, but the most cost-effective solution comes down to far more than
the initial sticker price.

Consider the total cost of ownership, including:
• The initial investment
• Ongoing operating costs
• Maintenance costs

In some cases, a more expensive system, such as immersion cooling, may ultimately be more cost-effective in the long run thanks to its energy efficiency.

The most cost-effective solution comes down to far more than the initial sticker price.

4. Sustainability

Lower cost isn’t the only benefit of an energy-efficient cooling solution. Energy efficiency is also key in helping data centers meet their environmental goals.

The power your cooling infrastructure consumes is an important determinant of your facility’s overall power usage effectiveness (PUE). Cooling infrastructure is also a major component of a data center’s Scope 1 carbon emissions, so prioritizing an energy-efficient cooling solution can impact your data center’s environmental footprint.

In addition to the cooling technology, building in a location with a mild climate and taking advantage of free cooling options when possible can contribute to an eco-friendly approach to cooling.

Cooling standards have changed over the years to prioritize energy efficiency. Data centers now operate at higher temperatures than in previous decades. You can ensure you stay within an appropriate
temperature window by following The American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) standards.

 

Chapter 4

Moving Forward with Modular Cooling Solutions

Modern data centers can utilize any cooling technologies discussed above to keep IT
equipment cool and running while balancing other crucial priorities.

Modular construction can help you optimize the design of your cooling infrastructure in relation to the rest of your facility.

Modular cooling devices

 

Icon showing modular data centersModular Data Centers

One option is to use MDCs, which are all-in-one data center solutions with built-in cooling.. These units make it easy to establish a new data center or build out capacity without designing and building a facility from scratch.

Icon showing modular mechanical skids

Modular Mechanical Skids

For more traditional data center facilities, mechanical skids offer modular convenience in a different way. These units house cooling equipment within a skidded system. They are highly scalable and can simplify your data center design and deployment.

Want to learn more about using skids as data center cooling equipment? Check out The Complete Guide to Modular Data Center Skids.

 

Don't start your next project until you've discovered the benefits of modular data center skids! Download the 'Process for Modular Data Center Skids' infographic.
close chapters modal

Download a PDF version of this guide by filling out this form

Simply fill out this form to receive a PDF version of The Complete Guide to Data Center Cooling

The Complete Guide to Data Center Cooling