Creating a Simulation-Ready Data Center Model

Data centers can be large spaces with a lot of equipment. The data center needs to be characterized well in Simulation CFD to properly represent its performance. Here the process of preparing a data center for simulation will be discussed.

 

1. Define Data Center Design Goals

Simulation CFD can be used to help design an efficient air management system in data centers.  The high energy density of a data center requires extra consideration while developing the air management system.  Avoiding recirculation and conditioning hot exhaust air as soon as possible will lower operating temperatures and reduce energy costs.

Further information on data center design practices can be found online. Due to the rapid growth in this industry, it is recommended to research recent design best practices similar to what can be found here.  

2. Characterize Data Center Components

Data centers all share the following primary components which need to be considered in the characterization strategy, and are discussed in detail below:

  •  Air Handlers
  • Server Racks
  • Diffusers and Tiles
  • Walls, Ceiling, and Floor

The complexity of these components, combined with the large number of components, will require strategic characterization methods as detailed in the Component Characterization section.

Air Handlers

In this closed loop configuration (left), a heat exchanger device (1) is used to remove energy and circulate the air through the sub-floor and into the cold aisle inlets.  A simpler alternative (right) uses a flow boundary condition (2) and a pressure ope

In this closed loop configuration (left), a heat exchanger device (1) is used to remove energy and circulate the air through the sub-floor and into the cold aisle inlets.  A simpler alternative (right) uses a flow boundary condition (2) and a pressure opening (3) to avoid modeling the full sub-floor air domain.

What it is: Circulate air and remove the thermal energy from the system. Can be a centralized air conditioning system or dedicated Computer Room Air Conditioner (CRAC) units.

How it is simulated: Represented with either with heat exchanger devices or with boundary conditions.

The first step in characterizing the air handler is to determine if a closed loop model will be simulated or if a simpler set of boundary conditions will be used instead. 

A closed loop model is more complex since it requires modeling the air space below the sub-floor or above the drop-down ceiling.  When using boundary conditions (a process also known as short-circuiting), these areas do not need to be explicitly modeled, reducing overall complexity.  However, the boundary conditions method will not directly account for any losses in the sub-floor space between the heat exchanger and the cold aisle inlets.

(1) CRAC unit with housing hidden to reveal heat exchanger material assigned to a simple block.  This is an example of a closed loop configuration.  (2) This small data center contains a total of 6 CRAC units.

(1) CRAC unit with housing hidden to reveal heat exchanger material assigned to a simple block.  This is an example of a closed loop configuration.

(2) This small data center contains a total of 6 CRAC units.

Air handlers for data centers are characterized according to the best practices detailed here:

Server Racks

What it is: From a thermal and flow standpoint, the server racks both circulate air and add thermal energy to the system.

How it is simulated: Represented with internal fan devices with a heat generation boundary condition.

Server racks contain server blades, each of which have individual fan flows and heat loads.  For simulating a data center system, the blades are all added up to determine a total flow rate and heat load for each rack.

When simulating data centers, at least 2 internal fan devices will be needed to account for the opposite direction of flow for each row of servers.  When looking down from the ceiling, the servers (grey) pull air from the cold aisles (blue) and push it in

When simulating data centers, at least 2 internal fan devices will be needed to account for the opposite direction of flow for each row of servers.  When looking down from the ceiling, the servers (grey) pull air from the cold aisles (blue) and push it into the hot aisles (red).

The server racks are characterized using an internal fan device which also has a heat generation boundary condition.  Note that a minimum of 2 internal fan devices will be required since each row of servers will move air in opposite directions to create the typical alternating hot aisle/cold aisle pattern.

Rows of servers can be represented with either single or multiple fans.  Using a single fan reduces complexity and would be valid if all of the server racks have the same flow rate and heat load.  Multiple fans would be the better choice if individual racks have varying flow rates or heat loads.  Note that in actual data centers, some racks are intentionally kept empty to store wiring and other components.

(1) Entire row of server racks characterized as a single fan device.  (2) Individual racks characterized as fans.

(1) Entire row of server racks characterized as a single fan device.

(2) Individual racks characterized as fans. 

 

Server racks for data centers essentially act as heat exchangers (they add heat) in the system;  more details on using internal fan devices to represent heat exchangers can be found here:

Diffusers and Perforated Tiles

What it is: Some data centers may have diffusers in the walls or ceiling to supply air from a central conditioning system.  Perforated tiles are a type of diffuser used in the cold aisle supplies to evenly distribute the air across the server inlets. 

How it is simulated: Diffusers should be simplified and represented with boundary conditions, especially if there are a large quantity of them. Perforated tiles can be represented with a resistance material.

A detailed data center floor tile (1) next to a simpler characterization (2) using a resistance material.  Note that a large data center can contain hundreds of these tiles.

A detailed data center floor tile (1) next to a simpler characterization (2) using a resistance material.  Note that a large data center can contain hundreds of these tiles.

If there are a large number of supply diffusers in the data center, they should be simplified by following the best practices for characterization detailed on the diffuser page in Component Characterization section.

Perforated tiles contain a lot of small holes or slots which let air pass through.  The combination of small features and the overall number of tiles make them impractical to model explicitly.  To account for the pressure drop of air moving through this restriction, a resistance material can be used.  Since the tiles are relatively thin compared to the overall model,  the best practice is to use surface resistances for tiles to reduce mesh requirements.   For more information on surface resistances, please refer to the Devices page of Materials section.


Above shows the tiles in front of a row of servers in a data center.  Notice that they are just simple blocks that are the thickness of the floor with a surface resistance assigned to one side (zoomed in on the right).

TIP: Similar to the server racks, the square tiles (typically 2’ x 2’) can be merged together in CAD to reduce overall volume count and streamline material and boundary condition assignments.

Walls, Ceiling, and Floor

What it is: The elements that bound and enclose the data center.

How it is simulated: The geometry of these elements can be suppressed, but they define the shape of the air volume. External walls can be represented with U-factor boundary conditions on the air volume.

If the bounding objects (e.g., walls, ceiling, floor) of a data center are not expected to transfer thermal energy, then they can be omitted from the simulation.  The primary air domains can be created directly in CAD, or if the bounding objects are modeled, they can be suppressed from the simulation.

If any of the bounding objects will transfer heat (e.g., one side of the data center is a glass wall exposed to outside air), then U-factor boundary conditions can be used to represent the heat load as detailed on the Walls and Windows page of the Component Characterization section.

The data center above is shown in the meshing task.  Note that all of the blue volumes are suppressed from the meshing.  This includes all of the exterior walls, interior ceiling, floor and CRAC housings.

The data center above is shown in the meshing task.  Note that all of the blue volumes are suppressed from the meshing.  This includes all of the exterior walls, interior ceiling, floor and CRAC housings.

 

3. Solver Settings

The large number of fluid momentum sources from the air handlers and server rack make this a forced convection scenario.  For some data centers, thermal stratification from natural convection may also play a role, which would require a mixed convection scenario. 

For best practices on adjusting solver settings for forced and mixed convection scenarios, please refer to the AEC Solver Settings page of the Solver section.