The Fastest AI Chip will Greatly Accelerate AI Progress

Whether you view AI as an extraordinary tool with immense advantages or a societal detriment that favors only the powerful, a groundbreaking new chip can now train them at unprecedented speeds. Cerebras Systems has introduced the Wafer Scale Engine 3 (WSE-3), the world’s fastest AI chip, powering the Cerebras CS-3 AI supercomputer with a peak performance of 125 petaFLOPS, and its scalability is remarkable.

Prior to an AI system creating an endearing yet slightly eerie video of a cat waking its owner, it must undergo training with a truly remarkable volume of data, consuming energy equivalent to that of over 100 households in the process. However, the new chip, along with computers constructed using it, will enhance the speed and efficiency of this process.

Revolutionary Chip Performance and System Compactness

Each WSE-3 chip, roughly the size of a pizza box, contains an astonishing four trillion transistors, delivering double the performance of the company’s previous model, which held the previous world record, all at the same cost and power consumption. When assembled into the CS-3 system, these chips reportedly offer the performance equivalent to a room filled with servers, condensed into a single unit the size of a mini-fridge.

Cerebras claims that the CS-3 system houses 900,000 AI cores and 44 GB of on-chip SRAM, delivering up to 125 petaFLOPS of peak AI performance. In theory, this should position it among the top 10 supercomputers globally, although formal benchmark testing is yet to confirm its performance.

Unprecedented Data Capacity and Model Handling

To accommodate the vast amount of data, external memory options range from 1.5 TB to a colossal 1.2 Petabytes (1,200 TB). The CS-3 surpasses current standards by handling AI models with up to 24 trillion parameters, far exceeding the projected maximum of around 1.8 trillion parameters for models like GPT-4. Cerebras asserts that the CS-3 can effortlessly train a one-trillion-parameter model, equivalent to the workload of current GPU-based computers handling a mere one-billion-parameter model.

The wafer manufacturing method used for WSE-3 chips enables the CS-3 to be engineered for scalability. It allows for clustering up to 2,048 units into a single supercomputer, which can achieve up to 256 exaFLOPS, surpassing the capabilities of current leading supercomputers. This level of power could purportedly train a Llama 70B model from scratch in just one day, according to the company.

The rapid advancement of AI models is already palpable, but this technology is set to accelerate progress even further. Regardless of one’s profession or hobbies, it appears AI systems will continue to encroach upon various domains at an unprecedented pace.

Read the original article on: New Atlas

Read more: Expert Warns of Mental Health Risks from AI Ghosts