Rail-Optimized or Full CLOS: How Should You Build a 2,000-GPU AI Fabric?

Not all GPU clusters are built the same way.

And the topology decision you make early in the design process will follow you for years.

Most network engineers instinctively reach for something familiar. Spine and leaf. CLOS. Any-to-any connectivity. Equal-cost paths everywhere.

That is a reasonable starting point.

But AI training fabrics introduce a question that traditional data center design rarely had to answer:

What if the network already knows who is going to talk to whom?

That changes the design conversation.

AI training traffic is not random.

When thousands of GPUs work on the same training job, they repeatedly synchronize through collective operations such as AllReduce. GPUs collectively combine and redistribute gradient updates before the next computation step can continue.

The important part from a network perspective is that this communication is structured, repetitive, and tied to a known set of GPUs participating in the same job.

Traditional data center networks are designed around uncertainty.

Any server might talk to any other server. Traffic patterns shift. East-west communication can appear almost anywhere.

A GPU training cluster is more structured than that.

And rail optimization is designed to take advantage of it.

The idea is straightforward.

Imagine every server has eight GPUs.

GPU 0 across the servers is associated with one network rail. GPU 1 uses another. GPU 2 another, and so on.

Instead of treating every network path as completely interchangeable, the physical topology is aligned with the logical structure of the GPU cluster.

Collective communication can then use predictable portions of the fabric rather than depending entirely on ECMP hashing across a generic any-to-any topology.

The result is more deterministic pathing and bandwidth utilization.

The fabric and the workload are intentionally designed to work together.

But rail optimization is not free.

When the network is optimized around GPU placement, where a job runs begins to matter.

The scheduler needs to understand topology.

Which GPUs are close to each other? Which GPUs share a rail? Which switches will their communication traverse? How should ranks be assigned inside the distributed job?

Now the network is influencing workload placement.

That is one of the biggest differences between designing a traditional data center and designing infrastructure for large-scale AI training.

The complexity does not disappear.

It moves.

A full CLOS fabric pushes much of that complexity back into the network.

Any GPU can communicate with any other GPU with high available bandwidth regardless of placement.

The scheduler has more freedom.

Jobs can grow and shrink without being as tightly coupled to the physical topology.

Different workload types can share the same cluster more easily.

Training and inference can coexist without requiring the scheduler to care as much about how GPUs map onto the network.

But that flexibility costs ports, optics, power, switches, and fabric capacity.

At 2,000 GPUs, that becomes a serious architectural and financial decision.

You are effectively paying for workload-placement freedom.

So which approach is right?

A dedicated training cluster with predictable workload sizes and strong control over job placement can benefit significantly from rail optimization.

The communication patterns are known. The scheduler can understand the topology. Collective algorithms can take advantage of it. And the infrastructure can be designed around the dominant workload.

But imagine a different environment.

Jobs vary dramatically in size. Training and inference share the same infrastructure. GPU allocation changes constantly. The operations team does not want workload placement to depend heavily on the physical network.

In that environment, paying for greater any-to-any flexibility may be the better decision.

The real question is not simply rail or CLOS.

Rail optimization can exist inside a CLOS architecture.

The more useful question is how much should the network depend on topology-aware workload placement.

And underneath that is the question that actually drives the architecture:

Where do you want the complexity to live?

A highly flexible fabric puts more capability and capacity into the network.

A topology-aware design can use the fabric more efficiently, but now the scheduler, placement strategy, rank assignment, and collective algorithms need to understand the topology.

Neither approach eliminates complexity.

It places that complexity in a different part of the system.

At 2,000 GPUs, the network is no longer just infrastructure underneath the compute system.

It becomes part of the compute system.

GPU utilization depends on communication performance. Communication performance depends on topology. Topology influences placement. Placement influences collective efficiency. And collective efficiency determines how long thousands of extremely expensive GPUs spend computing instead of waiting for the network.

That is why the first question should not be which switch to buy or how many spines you need.

The first question should be what communication pattern are you actually building this network for.

Once you understand the workload, the topology decision becomes much easier to justify.