Elon Musk's Memphis Project Colossus
Dojo or Gigacenter of compute.
Hello Everyone,
If xAI can get an additional $5 Billion from Tesla, after a huge series B, it will be fairly interesting in xAI trying to catch up to the likes of OpenAI, who are under financial pressure. If you remember what Elon calls a Gigacenter of compute, is just a bigger AI datacenter.
On June 5, 2024, the Greater Memphis Chamber of Commerce announced that Elon Musk's xAI would build the world's largest supercomputer in Memphis. This project, dubbed the 'Gigafactory of Compute,' represents the largest capital investment by a new-to-market company in Memphis's history.
Elon Musk is using Tesla and his own leverage to try to fast-track xAI and Grok’s development.
On July 22nd, Elon Musk announced the birth of this facility in an X post.
“Nice work by xAI team, X team, Nvidia & supporting companies getting Memphis Supercluster training started at ~4:20am local time. With 100k liquid-cooled H100s on a single RDMA fabric, it’s the most powerful AI training cluster in the world!” - Elon Musk
xAI’s supercomputer cluster is expected to be used to train the company’s large language model, Grok, which is currently rolled out as a feature for X Premium subscribers.
Grok 3 could arrive as early as December, 2024, but knowing Elon Musk and timelines, likely sometime in early 2025.
Of course Dojo is not the same as the Memphis gigacenter of compute. Memphis Supercluster: A supercomputer cluster in Memphis, Tennessee, using Nvidia H100 GPUs, developed by xAI.