Exercise: Data Centers in Simulation CFD
- Observe how to set up a data center efficiently in Simulation CFD.
- Provide experience working with multiple data center components in Simulation CFD.
- Validate simulation inputs and evaluate data center performance.
The simulation will have several simplifying assumptions to reduce overall complexity and expedite completion of this data center tutorial.
- All exterior walls will be considered adiabatic (i.e., perfectly insulated).
- Radiation will be assumed to be negligible.
- Raised floor and ceiling will be considered adiabatic forcing airflow to control temperature distribution.
- System will be considered as a closed loop with no air being added or removed from the simulation.
- Heat exchanger capacity:
- 15,000 CFM each
- Supply temperature of 61F
- Server racks
- 800 CFM each
- 10,000 Watts each
NOTE: Setup will begin from a support file for which some settings have been predefined to maintain focus on topics specific to data centers. As this an advanced exercise, it is assumed that fundamental skills required to interact with the Simulation CFD interface have already been developed.
1. Download the Exercise-data-center.cfz share file to a local hard drive folder.
2. Load the share file
a. Start the Simulation CFD interface and click Open in the upper left ribbon, then navigate to and open the Exercise-data-center.cfz share file.
3. Define internal fan devices for the server racks:
a. In the Design Study Bar, expand the Groups node, right click on the +X_Servers node and then click on Show only this Group to select the servers which will flow air in the positive X direction and hide all other volumes.
TIP: Groups can be useful in the setup and results review of models with many components; more details about groups can be found here.
b. Assign an Internal Fan device to the selected volumes with the following attributes:
Save to Database
+X 42U Rack
c. Make certain to set the Flow Direction to 1,0,0 (+X) before clicking on the Apply button.
4. Repeat previous step for the servers on the opposite side of the hot aisle:
a. Right click on the Groups:-X_Servers node and then click on Show only this Group to select the servers which will flow air in the negative X direction and hide all other volumes.
b. Define Fan Name as -X 42U Rack
c. Make certain to set the Flow Direction to -1,0,0 (-X) before clicking on the Apply button.
5. Assign Heat Exchanger devices to represent the CRAC units:
a. Right click on the Groups:CRACs node and then click on Show only this Group to select the CRAC volumes and hide all other volumes.
b. Click directly on one of the selected volumes to deselect it (heat exchangers have to be defined one at a time).
c. Click on Edit in the Ribbon and change the material Type to Heat Exchanger.
d. Click on the “...” button next to Inlet Surfaces.
e. Click Select Surface and select the top surface (either surface id 41 or 42 depending on which volume is being assigned) of the selected volume (rotate model if needed).
f. Click on the “...” button next to Outlet Surfaces.
g. Click Select Surface and select the bottom surface of the selected volume.
h. Bring up the material editor by clicking Edit… in the Materials dialog.
i. Change the Save to database: to Local.
j. For the Name, type in CRAC 1.
k. Set the Flow to Constant at 15000 ft3/min.
l. Select Apply to see the settings update in the properties section.
m. Select the Heat Transfer button from the properties.
n. Select Air Conditioner from the Variation method: dropdown.
o. Make the Set Point Temperature a value of 61 Fahrenheit.
p. Click the Apply button.
q. Click the OK button in the Material Editor dialog.
r. Click the Apply button in the Materials dialog.
6. Repeat step 5 for the other volume representing a CRAC unit:
a. Enter a name of CRAC 2 for this device.
7. Assign surface resistance regions to the floor tiles to represent the pressure drop going through the actual tiles.
a. Right click on the Groups:NoAirCeiling node and then click on Show only this Group which will hide the primary air volumes and the ceiling volume.
b. Left click on the Groups:FloorTiles node to select the top surfaces of the floor tiles.
c. Define a custom surface resistance material with the following attributes:
Save to Database
Free Area Ratio
d. Click the Apply button and then the OK button.
e. Set the Shell Thickness to 1 inch in the Materials dialog.
f. Click Apply.
8. Repeat step 7 for the ceiling tiles:
a. Left click on the Groups:CeilingTiles node to select only the bottom surface of the volume for the ceiling tiles.
b. Create another custom resistance material with the following attributes:
i. Name: 80 Open
ii. Free Area Ratio: 0.8
9. Assign a heat generation boundary condition to each server to represent the heat load from the server components (16 servers with 10,000 Watts each for a total of 160,000 Watts added to the system).
a. Go into the Boundary Conditions setup task.
b. Left click on the Groups:AllServers node to select all of the servers.
c. Select Edit.
d. Enter 10000 W into the Total Heat Generation quantity.
e. Hit Apply.
NOTE: Initial mesh settings have already been defined. Note that conduction through the ceiling, wall, and the housing volumes for the CRAC units and servers is assumed to be negligible. To save mesh, all of these volumes have been suppressed and will show up as blue in the Mesh Sizing task.
10. Solve the simulation. Settings for a forced convection analysis have already been defined (as detailed here in the AEC Solver Settings page).
a. Solve the analysis for 100 iterations.
11. Verify the model inputs with a few quick sanity checks.
a. When the simulation has completed, review the energy balance in the Summary File:
i. Click on Summary File in the Ribbon.
ii. Scroll down to the Fluid Energy Balance Information section in the Summary File (see Sanity Checks page). The Heat Transfer Due To Sources In Fluid should state 1.6e+005 Watts. If this number is incorrect, then check the volumetric heat generation assignments in the Boundary Conditions section of the setup.
iii. Now scroll down the Summary File a little further to the Data for heat exchanger devices section. The Flow Rate for each heat exchangers should be about 25,000 m^3/hr (15,000 cfm). If it is not, then the heat exchanger device has the wrong inputs.
b. Check the global thermal results.
i. Set the Global Result to Temperature and hide the air volumes to see the server racks. The minimum temperature in the model should be 61F (16.11C), which was the set point for the CRAC units. The hot aisle should be in between the 2 rack aisles. If it is not, then the fan directions were not defined correctly.
12. Check for recirculation in the server racks
a. Zoom in on the server racks and note some areas of potential recirculation which appear at the edges of the end units.
Using the options detailed on the Evaluating Data Center Results page, the results can be further interrogated, using Planes and Particle Traces (depicted below), to confirm the extent of recirculation around the servers or in other areas of the data center.
The primary focus of this tutorial was to provide practice in the proper setup of a data center simulation. To keep the tutorial manageable, the data center was intentionally limited to 16 servers. However, the same fundamental best practices demonstrated here will directly apply to the largest of data centers, which may contain hundreds of servers. With larger data centers, the higher number of components will increase the probability of user input errors, making model validation a top priority. Finally, as the number of servers increase, so does the mesh size and solver run time.
|Dataset - exercise data center.cfz||588.97 KB|