Enlarge / The Cerebras Andromeda is a 13.5 Million core AI supercomputer.

Cerebras Systems has unveiled its core Andromeda AI Supercomputer for deep Learning on Monday. reports Reuters. Cerebras claims that Andromeda has more than one exaflop (1 quintillion operations every second) of AI computational power with 16-bit half precision.

The Andromeda itself is a 16-member cluster. Cerebras C-2 Computers linked together. Each CS-2 has one Wafer Scale Engine chip (often called “WSE-2″It is currently at largest silicon chip At 8.5 inches square, it contains 2.6 trillion transistors in 850,000 cores.

For $35 million, Cerebras purchased Andromeda from Santa Clara’s data center. It’s tuned for applications like large language models and has already been in use for academic and commercial work. “Andromeda delivers near-perfect scaling via simple data parallelism across GPT-class large language models, including GPT-3, GPT-J and GPT-NeoX,” writes Cerebras in press release

The Cerebras WSL2 chip is roughly 8.5-inches square and packs 2.6 trillion transistors.
Enlarge / The Cerebras WSL2 Chip is approximately 8.5 inches square and contains 2.6 trillion transistors.

Cerebras

The phrase “Near-perfect scaling” Cerebras will add more CS-2 computer unit to Andromeda. This means that training time on neural networks is decreased as a result. “near perfect proportion,” According to Cerebras. One might use the GPU-based systems to increase the compute power of a deep-learning model. diminishing returns As hardware costs rise. Cerebras claims its supercomputer is capable of performing tasks that GPU-based system cannot.

GPU impossible work was demonstrated by one of Andromeda’s first users, who achieved near perfect scaling on GPT-J at 2.5 billion and 25 billion parameters with long sequence lengths—MSL of 10,240. Users attempted the same work with Polaris, a 2,000 Nvidia A100 Cluster. However, the GPUs could not handle the work due to memory bandwidth and GPU memory limitations.

Although it remains to be determined if those claims will hold up under external scrutiny, Cerebras offers an alternative in an age where deep-learning models are often trained on large clusters Nvidia GPUs.

How does Andromeda fare against other supercomputers like the IBM Xerox? At the moment, Andromeda is the fastest supercomputer in the world. FrontierOak Ridge National Labs’ computer resides at Oak Ridge National Labs. It is capable of performing at 1.103 exaflops, with 64-bit precision at 64-bit. It cost $600 million for the computer to be built.

Multiple remote users have access to Andromeda. Commercial writing assistants are already using it. JasperAI and Argonne National Laboratory, University of Cambridge for Research.