6 hours ago
5
Cerebras' SRAM-packed wafer-scale engines (WSEs) have carved out a niche in the AI model serving space for extremely low-latency, high-throughput inference, enabling services like OpenAI's ChatGPT-5.6 Sol Ultrafast tier. At Hot Chips 2026, the company revealed the next two generations of its wafer-scale accelerator roadmap. It also discussed the benefits of its new Nexus rack design for the CS-4 rack-scale accelerator and the performance of the three WS-3T wafer-scale engines contained within.
The integration of a huge coherent processor on a single massive slice of silicon is a unique feat in the industry. But that approach also comes with limitations. AI demands for memory are only increasing due to growing model sizes (the memory occupancy of which can be amortized across multiple inference sessions) and ever-lengthening contexts stored in large KV caches (which are also unique to each inference session).
Traditional GPU makers have addressed those pressures, in part by working with memory makers to stack HBM higher and by using more of it per accelerator to expand that precious resource in proximity to the processor. But on a wafer-scale design whose area is already 100% utilized by logic and memory, adding more of a particular resource requires giving up area that might have been used for some other purpose. Since silicon production will continue to take place on 300mm wafers for the foreseeable future, Cerebras must look in other directions to scale up the on-chip resources available to its processors.
Cerebras revealed that it will start expanding its wafer-scale engines into stacked designs with its CS-6 system’s WSE, currently two generations out on its roadmap. For the first time, Cerebras will attempt 3D stacking of DRAM on top of its logic and SRAM wafer, a move it claims will maintain the company's performance lead for inference while reducing the area required for the overall chip.
The goal of stacking wafer-scale logic and memory chips on top of one another is certainly ambitious, but it’s only one potentially important change in the CS-6 system. The concurrent reduction in area Cerebras foresees suggests the company might be able to increase the overall number of WSEs it produces, which could relax a crucial constraint as the company seeks to scale its business amid a world of ever-increasing wafer demand.
In the present, Cerebras is boosting the performance of its existing wafer-scale platform with its new CS-4 rack-scale system and its Nexus rack design. CS-4 incorporates three of the company's refreshed WS-3T wafers into self-contained "backpacks" that incorporate power delivery, scale-up networking, and liquid cooling infrastructure into a single pluggable module.
Cerebras notes that because these modules are self-contained, future wafer-scale engines built with this architecture can be swapped in without exchanging the entire rack in the process.
Cerebras chief system architect JP Fricker had choice words when describing the 5,000 cables that are used to connect the Rubin NVL72 NVLink scale-up domain within each of those racks, calling it "a mess" and contrasting it with the cleaner and less failure-prone design provided by the on-die interconnects and self-contained compute module design of the Nexus system.
The Nexus backpack design also disaggregates the I/O interfaces of the WSE from the rest of the backpack's components. Two I/O modules now connect to the edges of the wafer, providing RoCE v2 RDMA connections for interoperability with other systems, alongside a direct connection to other wafers in the rack. Because these modules are also interchangeable, they provide another potential route for future upgrades, independent of the core compute wafer.
The Nexus design situates up to 10 rack power delivery units for each backpack at the front side of the rack, each group of which can be configured for varying levels of redundancy in accordance with an operator’s needs. The rack also provides air cooling for the backpack components that need it.
Mounting the wafer-scale engines vertically in the backpack modules lets Cerebras do away with a PCB or substrate for the wafer to handle all its supporting infrastructure. Instead, the backpack connects the large copper busbar that delivers juice to the chip directly to its back side. This close contact is important, as it minimizes power losses that occur on the way to the chip, as happens with a BGA GPU chip mounted on a PCB module with all of its power delivery circuitry located around the die.
Cerebras translates the power saved this way directly into performance in the WS-3T. The company says the losses avoided by the Nexus backpack design allow it to deliver twice as much power to the wafer-scale engine as in past designs, which leads directly to increased clock speeds and up to twice the performance of the WS-3.
Using the same base silicon as the WS-3, each WS-3T delivers twice as many sparse FP16 petaFLOPS and twice as much memory bandwidth from its SRAM. But the WS3-T is still limited to 44GB of memory across the entire wafer, and three such wafers in a CS-4 rack only scale up to 132 GB, far less than the 20.7 TB of HBM in the Vera Rubin NVL72 system and the 31 TB of AMD’s Helios.
The company doesn’t publish dense PFLOPS figures for these engines, possibly because the dataflow architectural design of the chips is specifically built to derive advantage from sparsity in a way a traditional GPU usually isn’t.
In any event, to accommodate the larger models of today and tomorrow, Cerebras will need to scale up and out. But unlike other rack-scale systems that rely on Ethernet for scale-out, Cerebras can simply connect CS-4 systems together using the same wafer-to-wafer interconnect that connects wafer-scale engines together in the Nexus rack. The company claims 2.4 Tb/s of direct scale-up bandwidth per wafer within the rack for a total of 7.2 Tb/s of inter-chip bandwidth at 2 μs latencies.
Cerebras notes that with its architecture, only the model activations need to pass between wafer-scale engines, so the relatively low bandwidth of the direct wafer connection isn't the obstacle to scaling out the system that it might seem when evaluated against the hundreds of terabytes per second of scale-up bandwidth of a system like Vera Rubin NVL72 or AMD's Helios. (The on-die fabric of the WSE-3T boasts 53.4 PB/s of bandwidth, regardless.)
The CS-4 system architecture lays the groundwork for the next-generation CS-5 accelerator, which will use new WSE silicon in 2027. For smaller models, the company says the next-generation WSE will deliver up to 10,000 tokens per second per user, while larger frontier models from labs like DeepSeek or OpenAI could run at 5,000 tokens per second per user.
As Nvidia CEO Jensen Huang has said, AI agents are impatient, and the ability to provide such vast numbers of tokens per second using specialized accelerators like the CS-4 will likely continue to be an important niche for Cerebras to exploit alongside its partners at OpenAI and AMD going forward.
Image 1 of 42
English (US) ·