Somewhere around the current generation of dense compute, air cooling stops being sufficient at rack level. Where exactly depends on the hardware, the room, and how honest the facility’s figures are, but the transition is happening now in ordinary enterprise data halls.
What replaces it is one of three answers, and the choice is made with facilities and finance in the room, not by an infrastructure architect alone.
Air, pushed further
Containment, higher supply temperatures, careful airflow management, blanking panels taken seriously. There is more headroom in a well-managed air-cooled hall than most organisations are using, because most halls are not well managed.
Before concluding that air is exhausted, find out whether it has actually been tried properly. Hot aisle containment and a supply temperature raised to what the equipment actually tolerates buys real capacity for modest money.
The ceiling is real though. Beyond a certain per-rack figure, the air volume required becomes impractical regardless of management.
Rear door heat exchangers
A water-cooled radiator on the back of the rack. Air still moves through the servers in the normal way; the heat is removed into water at the rack exit rather than dumped into the room.
The appeal: servers are unmodified, the failure mode is a rack that reverts to air cooling and gets hot slowly rather than immediately, and you can deploy it rack by rack rather than rebuilding the hall.
The cost: water to each rack position, which means pipework, a facility water loop, and leak detection. It is a building project, smaller than the alternative.
This is the pragmatic middle and it is where most enterprises adding their first dense racks should look.
Direct liquid cooling
Cold plates on the hot components, coolant delivered to each server. Removes the majority of the heat directly and supports densities air cannot approach.
The cost is a commitment. Servers are specific to it, the facility needs a full coolant distribution system, and the operating model changes in ways that surprise IT teams: fluid quality management, leak procedures, connector handling during maintenance, and somebody competent available when a connection fails at 3am.
It is also increasingly not optional at the top of the hardware range, which means the decision may be made for you by what you need to buy.
The questions that decide it
What is the actual current per-rack capacity? Measured, not the design figure from the original build.
What is the largest density you will need in five years? Because the facility work has a longer life than the servers.
Is there a water loop in the building, or would you be creating one? The marginal cost of extending an existing loop is a fraction of creating one.
Who maintains it? If the answer is a facilities team that has never run a technical water loop near live equipment, that is a hiring or contracting question, and it belongs in the business case.
Is colocation cheaper than retrofitting? Very often, yes. A facility built for this already has it, and the comparison should be run properly rather than assumed away.
The thing to do first
Get the facilities team into the design review at the start, not at the point of ordering. The lead times here are the longest in the project — electrical capacity, mechanical work, permits — and an infrastructure design that arrives at facilities as a fait accompli will be delayed by months.
The most common failure in accelerated infrastructure projects that I see is not technical. It is that the hardware was ordered before anyone asked whether the room could take it.